Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20250115 03:06:09.681093 14059 test_util.cc:274] Using random seed: -174625098
2025-01-15T03:06:09Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-01-15T03:06:09Z Disabled control of system clock
I20250115 03:06:09.695153 14059 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu
/tmp/dist-test-taskE2XgYg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.186.254:39901
--webserver_interface=127.13.186.254
--webserver_port=0
--builtin_ntp_servers=127.13.186.212:37407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.186.254:39901,127.13.186.253:35411,127.13.186.252:41855
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20250115 03:06:09.791471 14067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250115 03:06:09.791653 14067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250115 03:06:09.791690 14067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250115 03:06:09.794658 14067 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250115 03:06:09.794705 14067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250115 03:06:09.794725 14067 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250115 03:06:09.794762 14067 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250115 03:06:09.798511 14067 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.186.212:37407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.13.186.254:39901,127.13.186.253:35411,127.13.186.252:41855
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.186.254:39901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.186.254
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.14067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 75b75ca2f58f1a5aba69207830e3558011033d99
build type DEBUG
built by None at 15 Jan 2025 02:43:26 UTC on 24a791456cd2
build id 4181
I20250115 03:06:09.799381 14067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250115 03:06:09.800346 14067 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250115 03:06:09.807125 14067 server_base.cc:1034] running on GCE node
W20250115 03:06:09.807433 14075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 03:06:09.807636 14072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 03:06:09.807804 14073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250115 03:06:09.808221 14067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250115 03:06:09.809052 14067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250115 03:06:09.810194 14067 hybrid_clock.cc:648] HybridClock initialized: now 1736910369810170 us; error 36 us; skew 500 ppm
I20250115 03:06:09.814859 14067 webserver.cc:458] Webserver started at http://127.13.186.254:37481/ using document root <none> and password file <none>
I20250115 03:06:09.815347 14067 fs_manager.cc:362] Metadata directory not provided
I20250115 03:06:09.815397 14067 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250115 03:06:09.815557 14067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250115 03:06:09.817148 14067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data/instance:
uuid: "4225fe2fbfa84937bb41fed702ca66e4"
format_stamp: "Formatted at 2025-01-15 03:06:09 on dist-test-slave-c74p"
I20250115 03:06:09.817566 14067 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/wal/instance:
uuid: "4225fe2fbfa84937bb41fed702ca66e4"
format_stamp: "Formatted at 2025-01-15 03:06:09 on dist-test-slave-c74p"
I20250115 03:06:09.820478 14067 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20250115 03:06:09.822338 14081 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250115 03:06:09.823232 14067 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:09.823328 14067 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data,/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/wal
uuid: "4225fe2fbfa84937bb41fed702ca66e4"
format_stamp: "Formatted at 2025-01-15 03:06:09 on dist-test-slave-c74p"
I20250115 03:06:09.823398 14067 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250115 03:06:09.835734 14067 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250115 03:06:09.836226 14067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250115 03:06:09.836359 14067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250115 03:06:09.842119 14067 rpc_server.cc:307] RPC server started. Bound to: 127.13.186.254:39901
I20250115 03:06:09.842125 14134 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.186.254:39901 every 8 connection(s)
I20250115 03:06:09.842952 14067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data/info.pb
I20250115 03:06:09.845513 14135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:09.848157 14135 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } has no permanent_uuid. Determining permanent_uuid...
I20250115 03:06:09.849495 14059 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu as pid 14067
I20250115 03:06:09.849615 14059 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/wal/instance
I20250115 03:06:09.851007 14059 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu
/tmp/dist-test-taskE2XgYg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.186.253:35411
--webserver_interface=127.13.186.253
--webserver_port=0
--builtin_ntp_servers=127.13.186.212:37407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.186.254:39901,127.13.186.253:35411,127.13.186.252:41855
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20250115 03:06:09.858134 14135 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } has no permanent_uuid. Determining permanent_uuid...
W20250115 03:06:09.858840 14083 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.186.253:35411: connect: Connection refused (error 111)
W20250115 03:06:09.859647 14135 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.186.253:35411: Network error: Client connection negotiation failed: client connection to 127.13.186.253:35411: connect: Connection refused (error 111)
I20250115 03:06:09.908890 14135 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } attempt: 1
W20250115 03:06:09.910292 14135 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.186.253:35411: Network error: Client connection negotiation failed: client connection to 127.13.186.253:35411: connect: Connection refused (error 111)
W20250115 03:06:09.949120 14139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250115 03:06:09.949294 14139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250115 03:06:09.949329 14139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250115 03:06:09.952243 14139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250115 03:06:09.952286 14139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250115 03:06:09.952304 14139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250115 03:06:09.952320 14139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250115 03:06:09.955885 14139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.186.212:37407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.13.186.254:39901,127.13.186.253:35411,127.13.186.252:41855
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.186.253:35411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.186.253
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.14139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 75b75ca2f58f1a5aba69207830e3558011033d99
build type DEBUG
built by None at 15 Jan 2025 02:43:26 UTC on 24a791456cd2
build id 4181
I20250115 03:06:09.956733 14139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250115 03:06:09.957712 14139 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250115 03:06:09.963793 14145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 03:06:09.964170 14148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 03:06:09.964398 14146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250115 03:06:09.964794 14139 server_base.cc:1034] running on GCE node
I20250115 03:06:09.965212 14139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250115 03:06:09.966020 14139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250115 03:06:09.967149 14139 hybrid_clock.cc:648] HybridClock initialized: now 1736910369967136 us; error 28 us; skew 500 ppm
I20250115 03:06:09.968750 14139 webserver.cc:458] Webserver started at http://127.13.186.253:41043/ using document root <none> and password file <none>
I20250115 03:06:09.969261 14139 fs_manager.cc:362] Metadata directory not provided
I20250115 03:06:09.969312 14139 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250115 03:06:09.969448 14139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250115 03:06:09.970903 14139 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-1/data/instance:
uuid: "147c86b98f604d518e71f2ba3121fc21"
format_stamp: "Formatted at 2025-01-15 03:06:09 on dist-test-slave-c74p"
I20250115 03:06:09.971313 14139 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-1/wal/instance:
uuid: "147c86b98f604d518e71f2ba3121fc21"
format_stamp: "Formatted at 2025-01-15 03:06:09 on dist-test-slave-c74p"
I20250115 03:06:09.974282 14139 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20250115 03:06:09.976142 14154 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250115 03:06:09.977052 14139 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250115 03:06:09.977139 14139 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-1/data,/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-1/wal
uuid: "147c86b98f604d518e71f2ba3121fc21"
format_stamp: "Formatted at 2025-01-15 03:06:09 on dist-test-slave-c74p"
I20250115 03:06:09.977205 14139 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250115 03:06:09.978470 14135 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } attempt: 2
W20250115 03:06:09.979677 14135 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.186.253:35411: Network error: Client connection negotiation failed: client connection to 127.13.186.253:35411: connect: Connection refused (error 111)
I20250115 03:06:09.992435 14139 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250115 03:06:09.992918 14139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250115 03:06:09.993072 14139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250115 03:06:09.998971 14139 rpc_server.cc:307] RPC server started. Bound to: 127.13.186.253:35411
I20250115 03:06:09.998981 14206 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.186.253:35411 every 8 connection(s)
I20250115 03:06:09.999814 14139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-1/data/info.pb
I20250115 03:06:10.002403 14207 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:10.005111 14207 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } has no permanent_uuid. Determining permanent_uuid...
I20250115 03:06:10.005976 14059 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu as pid 14139
I20250115 03:06:10.006059 14059 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-1/wal/instance
I20250115 03:06:10.006747 14059 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu
/tmp/dist-test-taskE2XgYg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.186.252:41855
--webserver_interface=127.13.186.252
--webserver_port=0
--builtin_ntp_servers=127.13.186.212:37407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.186.254:39901,127.13.186.253:35411,127.13.186.252:41855
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20250115 03:06:10.013401 14207 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } has no permanent_uuid. Determining permanent_uuid...
I20250115 03:06:10.019074 14207 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } has no permanent_uuid. Determining permanent_uuid...
W20250115 03:06:10.019752 14156 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.186.252:41855: connect: Connection refused (error 111)
W20250115 03:06:10.020696 14207 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.186.252:41855: Network error: Client connection negotiation failed: client connection to 127.13.186.252:41855: connect: Connection refused (error 111)
I20250115 03:06:10.069933 14207 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } attempt: 1
I20250115 03:06:10.070878 14135 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } attempt: 3
W20250115 03:06:10.071470 14207 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.186.252:41855: Network error: Client connection negotiation failed: client connection to 127.13.186.252:41855: connect: Connection refused (error 111)
I20250115 03:06:10.074930 14135 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } has no permanent_uuid. Determining permanent_uuid...
W20250115 03:06:10.075937 14135 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.186.252:41855: Network error: Client connection negotiation failed: client connection to 127.13.186.252:41855: connect: Connection refused (error 111)
I20250115 03:06:10.107062 14135 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } attempt: 1
W20250115 03:06:10.107086 14210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250115 03:06:10.107282 14210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250115 03:06:10.107340 14210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250115 03:06:10.108162 14135 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.186.252:41855: Network error: Client connection negotiation failed: client connection to 127.13.186.252:41855: connect: Connection refused (error 111)
W20250115 03:06:10.110354 14210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250115 03:06:10.110401 14210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250115 03:06:10.110435 14210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250115 03:06:10.110466 14210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250115 03:06:10.113991 14210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.186.212:37407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.13.186.254:39901,127.13.186.253:35411,127.13.186.252:41855
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.186.252:41855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.186.252
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.14210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 75b75ca2f58f1a5aba69207830e3558011033d99
build type DEBUG
built by None at 15 Jan 2025 02:43:26 UTC on 24a791456cd2
build id 4181
I20250115 03:06:10.114851 14210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250115 03:06:10.115815 14210 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250115 03:06:10.122179 14217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 03:06:10.122347 14218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 03:06:10.122530 14220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250115 03:06:10.122886 14210 server_base.cc:1034] running on GCE node
I20250115 03:06:10.123301 14210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250115 03:06:10.124095 14210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250115 03:06:10.125231 14210 hybrid_clock.cc:648] HybridClock initialized: now 1736910370125223 us; error 25 us; skew 500 ppm
I20250115 03:06:10.126899 14210 webserver.cc:458] Webserver started at http://127.13.186.252:39163/ using document root <none> and password file <none>
I20250115 03:06:10.127439 14210 fs_manager.cc:362] Metadata directory not provided
I20250115 03:06:10.127497 14210 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250115 03:06:10.127652 14210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250115 03:06:10.129189 14210 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/data/instance:
uuid: "2db8cff032af46db986bf52b21b52246"
format_stamp: "Formatted at 2025-01-15 03:06:10 on dist-test-slave-c74p"
I20250115 03:06:10.129611 14210 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/wal/instance:
uuid: "2db8cff032af46db986bf52b21b52246"
format_stamp: "Formatted at 2025-01-15 03:06:10 on dist-test-slave-c74p"
I20250115 03:06:10.132560 14210 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20250115 03:06:10.134419 14226 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250115 03:06:10.135339 14210 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:10.135449 14210 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/data,/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/wal
uuid: "2db8cff032af46db986bf52b21b52246"
format_stamp: "Formatted at 2025-01-15 03:06:10 on dist-test-slave-c74p"
I20250115 03:06:10.135522 14210 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250115 03:06:10.139622 14207 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } attempt: 2
W20250115 03:06:10.140699 14207 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.186.252:41855: Network error: Client connection negotiation failed: client connection to 127.13.186.252:41855: connect: Connection refused (error 111)
I20250115 03:06:10.150573 14210 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250115 03:06:10.151046 14210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250115 03:06:10.151178 14210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250115 03:06:10.156801 14210 rpc_server.cc:307] RPC server started. Bound to: 127.13.186.252:41855
I20250115 03:06:10.156811 14278 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.186.252:41855 every 8 connection(s)
I20250115 03:06:10.157608 14210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/data/info.pb
I20250115 03:06:10.160073 14279 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:10.161370 14059 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu as pid 14210
I20250115 03:06:10.161463 14059 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/wal/instance
I20250115 03:06:10.162798 14279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } has no permanent_uuid. Determining permanent_uuid...
I20250115 03:06:10.170459 14279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } has no permanent_uuid. Determining permanent_uuid...
I20250115 03:06:10.174962 14279 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } has no permanent_uuid. Determining permanent_uuid...
I20250115 03:06:10.180265 14279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246: Bootstrap starting.
I20250115 03:06:10.182368 14279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:10.183185 14279 log.cc:826] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246: Log is configured to *not* fsync() on all Append() calls
I20250115 03:06:10.183312 14135 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } attempt: 2
I20250115 03:06:10.184751 14279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246: No bootstrap required, opened a new log
I20250115 03:06:10.187341 14279 raft_consensus.cc:357] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } }
I20250115 03:06:10.187549 14279 raft_consensus.cc:383] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:10.187647 14279 raft_consensus.cc:738] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2db8cff032af46db986bf52b21b52246, State: Initialized, Role: FOLLOWER
I20250115 03:06:10.188066 14279 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } }
I20250115 03:06:10.188333 14135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4: Bootstrap starting.
I20250115 03:06:10.188426 14288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:10.188542 14288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [sys.catalog]: This master's current role is: FOLLOWER
I20250115 03:06:10.189062 14279 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [sys.catalog]: configured and running, proceeding with master startup.
I20250115 03:06:10.190341 14135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:10.190987 14135 log.cc:826] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4: Log is configured to *not* fsync() on all Append() calls
I20250115 03:06:10.193401 14135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4: No bootstrap required, opened a new log
W20250115 03:06:10.194135 14299 catalog_manager.cc:1559] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250115 03:06:10.194191 14299 catalog_manager.cc:874] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250115 03:06:10.192682 14210 master_runner.cc:186] Error getting master registration for 127.13.186.254:39901: OK, instance_id { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" instance_seqno: 1736910369840453 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20250115 03:06:10.194923 14210 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20250115 03:06:10.195855 14135 raft_consensus.cc:357] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } }
I20250115 03:06:10.195984 14135 raft_consensus.cc:383] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:10.196015 14135 raft_consensus.cc:738] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4225fe2fbfa84937bb41fed702ca66e4, State: Initialized, Role: FOLLOWER
I20250115 03:06:10.196337 14135 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } }
I20250115 03:06:10.196700 14302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:10.196811 14302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [sys.catalog]: This master's current role is: FOLLOWER
I20250115 03:06:10.197276 14135 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [sys.catalog]: configured and running, proceeding with master startup.
W20250115 03:06:10.203619 14313 catalog_manager.cc:1559] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250115 03:06:10.203737 14313 catalog_manager.cc:874] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250115 03:06:10.203703 14067 master_runner.cc:186] Error getting master registration for 127.13.186.253:35411: OK, instance_id { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" instance_seqno: 1736910369997304 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20250115 03:06:10.206331 14067 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20250115 03:06:10.231842 14207 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } attempt: 3
I20250115 03:06:10.236635 14302 raft_consensus.cc:491] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:10.236804 14302 raft_consensus.cc:513] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } }
I20250115 03:06:10.237896 14302 leader_election.cc:290] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 147c86b98f604d518e71f2ba3121fc21 (127.13.186.253:35411), 2db8cff032af46db986bf52b21b52246 (127.13.186.252:41855)
I20250115 03:06:10.238724 14207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Bootstrap starting.
I20250115 03:06:10.238673 14253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4225fe2fbfa84937bb41fed702ca66e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2db8cff032af46db986bf52b21b52246" is_pre_election: true
I20250115 03:06:10.238996 14253 raft_consensus.cc:2463] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4225fe2fbfa84937bb41fed702ca66e4 in term 0.
I20250115 03:06:10.239487 14083 leader_election.cc:304] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2db8cff032af46db986bf52b21b52246, 4225fe2fbfa84937bb41fed702ca66e4; no voters:
I20250115 03:06:10.239781 14302 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:10.239846 14302 raft_consensus.cc:491] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:10.239885 14302 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:10.240978 14302 raft_consensus.cc:513] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } }
I20250115 03:06:10.241318 14302 leader_election.cc:290] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [CANDIDATE]: Term 1 election: Requested vote from peers 147c86b98f604d518e71f2ba3121fc21 (127.13.186.253:35411), 2db8cff032af46db986bf52b21b52246 (127.13.186.252:41855)
I20250115 03:06:10.241591 14207 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:10.241956 14253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4225fe2fbfa84937bb41fed702ca66e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2db8cff032af46db986bf52b21b52246"
I20250115 03:06:10.242060 14253 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:10.242398 14207 log.cc:826] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Log is configured to *not* fsync() on all Append() calls
I20250115 03:06:10.243093 14253 raft_consensus.cc:2463] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4225fe2fbfa84937bb41fed702ca66e4 in term 1.
I20250115 03:06:10.243465 14083 leader_election.cc:304] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2db8cff032af46db986bf52b21b52246, 4225fe2fbfa84937bb41fed702ca66e4; no voters:
I20250115 03:06:10.243691 14302 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:10.243888 14302 raft_consensus.cc:695] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [term 1 LEADER]: Becoming Leader. State: Replica: 4225fe2fbfa84937bb41fed702ca66e4, State: Running, Role: LEADER
I20250115 03:06:10.244184 14302 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } }
I20250115 03:06:10.245926 14302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4225fe2fbfa84937bb41fed702ca66e4. Latest consensus state: current_term: 1 leader_uuid: "4225fe2fbfa84937bb41fed702ca66e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:10.246088 14302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [sys.catalog]: This master's current role is: LEADER
I20250115 03:06:10.246691 14207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: No bootstrap required, opened a new log
I20250115 03:06:10.249845 14207 raft_consensus.cc:357] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } }
I20250115 03:06:10.250007 14207 raft_consensus.cc:383] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:10.250058 14207 raft_consensus.cc:738] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 147c86b98f604d518e71f2ba3121fc21, State: Initialized, Role: FOLLOWER
I20250115 03:06:10.250470 14207 consensus_queue.cc:260] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } }
I20250115 03:06:10.250962 14319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:10.251190 14319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [sys.catalog]: This master's current role is: FOLLOWER
I20250115 03:06:10.251621 14207 sys_catalog.cc:564] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [sys.catalog]: configured and running, proceeding with master startup.
I20250115 03:06:10.251673 14181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4225fe2fbfa84937bb41fed702ca66e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147c86b98f604d518e71f2ba3121fc21" is_pre_election: true
I20250115 03:06:10.251884 14181 raft_consensus.cc:2463] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4225fe2fbfa84937bb41fed702ca66e4 in term 0.
I20250115 03:06:10.252413 14180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4225fe2fbfa84937bb41fed702ca66e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "147c86b98f604d518e71f2ba3121fc21"
I20250115 03:06:10.252537 14180 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:10.253687 14180 raft_consensus.cc:2463] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4225fe2fbfa84937bb41fed702ca66e4 in term 1.
I20250115 03:06:10.254294 14318 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250115 03:06:10.255220 14318 catalog_manager.cc:1485] Initializing Kudu cluster ID...
W20250115 03:06:10.267321 14330 catalog_manager.cc:1559] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250115 03:06:10.267396 14330 catalog_manager.cc:874] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250115 03:06:10.275884 14253 raft_consensus.cc:1270] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [term 1 FOLLOWER]: Refusing update from remote peer 4225fe2fbfa84937bb41fed702ca66e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:10.276518 14180 raft_consensus.cc:1270] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [term 1 FOLLOWER]: Refusing update from remote peer 4225fe2fbfa84937bb41fed702ca66e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:10.277283 14316 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:10.277657 14316 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:10.283766 14288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4225fe2fbfa84937bb41fed702ca66e4. Latest consensus state: current_term: 1 leader_uuid: "4225fe2fbfa84937bb41fed702ca66e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:10.283874 14288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [sys.catalog]: This master's current role is: FOLLOWER
I20250115 03:06:10.284559 14319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4225fe2fbfa84937bb41fed702ca66e4. Latest consensus state: current_term: 1 leader_uuid: "4225fe2fbfa84937bb41fed702ca66e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:10.284667 14319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [sys.catalog]: This master's current role is: FOLLOWER
I20250115 03:06:10.292114 14302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4225fe2fbfa84937bb41fed702ca66e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:10.292210 14302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [sys.catalog]: This master's current role is: LEADER
I20250115 03:06:10.293195 14319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4225fe2fbfa84937bb41fed702ca66e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:10.293282 14319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [sys.catalog]: This master's current role is: FOLLOWER
I20250115 03:06:10.293655 14331 mvcc.cc:204] Tried to move back new op lower bound from 7114384876643414016 to 7114384876523597824. Current Snapshot: MvccSnapshot[applied={T|T < 7114384876643414016}]
I20250115 03:06:10.293834 14333 mvcc.cc:204] Tried to move back new op lower bound from 7114384876643414016 to 7114384876523597824. Current Snapshot: MvccSnapshot[applied={T|T < 7114384876643414016}]
I20250115 03:06:10.294657 14316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4225fe2fbfa84937bb41fed702ca66e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:10.294767 14316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [sys.catalog]: This master's current role is: LEADER
I20250115 03:06:10.295822 14288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4225fe2fbfa84937bb41fed702ca66e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:10.295912 14288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [sys.catalog]: This master's current role is: FOLLOWER
I20250115 03:06:10.296504 14334 mvcc.cc:204] Tried to move back new op lower bound from 7114384876643414016 to 7114384876523597824. Current Snapshot: MvccSnapshot[applied={T|T < 7114384876643414016}]
I20250115 03:06:10.299234 14318 catalog_manager.cc:1348] Generated new cluster ID: 6933a79146ea47b6911c92899078d551
I20250115 03:06:10.299289 14318 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250115 03:06:10.325213 14318 catalog_manager.cc:1371] Generated new certificate authority record
I20250115 03:06:10.326385 14318 catalog_manager.cc:1505] Loading token signing keys...
I20250115 03:06:10.346220 14318 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4: Generated new TSK 0
I20250115 03:06:10.346988 14318 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250115 03:06:10.364838 14059 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu
/tmp/dist-test-taskE2XgYg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.186.193:0
--local_ip_for_outbound_sockets=127.13.186.193
--webserver_interface=127.13.186.193
--webserver_port=0
--tserver_master_addrs=127.13.186.254:39901,127.13.186.253:35411,127.13.186.252:41855
--builtin_ntp_servers=127.13.186.212:37407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
W20250115 03:06:10.482296 14340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250115 03:06:10.482508 14340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250115 03:06:10.482575 14340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250115 03:06:10.485587 14340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250115 03:06:10.485700 14340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.186.193
I20250115 03:06:10.489260 14340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.186.212:37407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--fs_data_dirs=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.186.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.186.193
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.13.186.254:39901,127.13.186.253:35411,127.13.186.252:41855
--never_fsync=true
--heap_profile_path=/tmp/kudu.14340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.186.193
--log_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 75b75ca2f58f1a5aba69207830e3558011033d99
build type DEBUG
built by None at 15 Jan 2025 02:43:26 UTC on 24a791456cd2
build id 4181
I20250115 03:06:10.490193 14340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250115 03:06:10.491206 14340 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250115 03:06:10.498279 14346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 03:06:10.498672 14348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 03:06:10.499023 14345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250115 03:06:10.499369 14340 server_base.cc:1034] running on GCE node
I20250115 03:06:10.499683 14340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250115 03:06:10.500137 14340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250115 03:06:10.501267 14340 hybrid_clock.cc:648] HybridClock initialized: now 1736910370501254 us; error 25 us; skew 500 ppm
I20250115 03:06:10.502858 14340 webserver.cc:458] Webserver started at http://127.13.186.193:36373/ using document root <none> and password file <none>
I20250115 03:06:10.503338 14340 fs_manager.cc:362] Metadata directory not provided
I20250115 03:06:10.503392 14340 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250115 03:06:10.503551 14340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250115 03:06:10.505039 14340 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/data/instance:
uuid: "e8df7247ee9c43e0aa18216da2661f75"
format_stamp: "Formatted at 2025-01-15 03:06:10 on dist-test-slave-c74p"
I20250115 03:06:10.505461 14340 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/instance:
uuid: "e8df7247ee9c43e0aa18216da2661f75"
format_stamp: "Formatted at 2025-01-15 03:06:10 on dist-test-slave-c74p"
I20250115 03:06:10.508355 14340 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:10.510254 14354 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250115 03:06:10.511135 14340 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:10.511229 14340 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/data,/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal
uuid: "e8df7247ee9c43e0aa18216da2661f75"
format_stamp: "Formatted at 2025-01-15 03:06:10 on dist-test-slave-c74p"
I20250115 03:06:10.511309 14340 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250115 03:06:10.544545 14340 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250115 03:06:10.545055 14340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250115 03:06:10.545189 14340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250115 03:06:10.545641 14340 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250115 03:06:10.546484 14340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250115 03:06:10.546531 14340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250115 03:06:10.546579 14340 ts_tablet_manager.cc:610] Registered 0 tablets
I20250115 03:06:10.546607 14340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250115 03:06:10.554203 14340 rpc_server.cc:307] RPC server started. Bound to: 127.13.186.193:45107
I20250115 03:06:10.554250 14467 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.186.193:45107 every 8 connection(s)
I20250115 03:06:10.554999 14340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/data/info.pb
I20250115 03:06:10.561020 14059 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu as pid 14340
I20250115 03:06:10.561133 14059 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/instance
I20250115 03:06:10.564039 14059 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu
/tmp/dist-test-taskE2XgYg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.186.194:0
--local_ip_for_outbound_sockets=127.13.186.194
--webserver_interface=127.13.186.194
--webserver_port=0
--tserver_master_addrs=127.13.186.254:39901,127.13.186.253:35411,127.13.186.252:41855
--builtin_ntp_servers=127.13.186.212:37407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20250115 03:06:10.564338 14468 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:10.564522 14468 heartbeater.cc:463] Registering TS with master...
I20250115 03:06:10.564990 14468 heartbeater.cc:509] Master 127.13.186.252:41855 requested a full tablet report, sending...
I20250115 03:06:10.565290 14470 heartbeater.cc:346] Connected to a master server at 127.13.186.253:35411
I20250115 03:06:10.565377 14470 heartbeater.cc:463] Registering TS with master...
I20250115 03:06:10.565562 14470 heartbeater.cc:509] Master 127.13.186.253:35411 requested a full tablet report, sending...
I20250115 03:06:10.566016 14469 heartbeater.cc:346] Connected to a master server at 127.13.186.254:39901
I20250115 03:06:10.566089 14469 heartbeater.cc:463] Registering TS with master...
I20250115 03:06:10.566236 14469 heartbeater.cc:509] Master 127.13.186.254:39901 requested a full tablet report, sending...
I20250115 03:06:10.566362 14243 ts_manager.cc:194] Registered new tserver with Master: e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:10.566421 14171 ts_manager.cc:194] Registered new tserver with Master: e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:10.567102 14097 ts_manager.cc:194] Registered new tserver with Master: e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:10.568625 14097 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.186.193:47161
W20250115 03:06:10.661579 14475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250115 03:06:10.661753 14475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250115 03:06:10.661799 14475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250115 03:06:10.664705 14475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250115 03:06:10.664796 14475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.186.194
I20250115 03:06:10.668244 14475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.186.212:37407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--fs_data_dirs=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.186.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.186.194
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.13.186.254:39901,127.13.186.253:35411,127.13.186.252:41855
--never_fsync=true
--heap_profile_path=/tmp/kudu.14475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.186.194
--log_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 75b75ca2f58f1a5aba69207830e3558011033d99
build type DEBUG
built by None at 15 Jan 2025 02:43:26 UTC on 24a791456cd2
build id 4181
I20250115 03:06:10.669072 14475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250115 03:06:10.670010 14475 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250115 03:06:10.677326 14480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 03:06:10.677409 14483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 03:06:10.677850 14481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250115 03:06:10.678234 14475 server_base.cc:1034] running on GCE node
I20250115 03:06:10.678521 14475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250115 03:06:10.678954 14475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250115 03:06:10.680090 14475 hybrid_clock.cc:648] HybridClock initialized: now 1736910370680077 us; error 26 us; skew 500 ppm
I20250115 03:06:10.681716 14475 webserver.cc:458] Webserver started at http://127.13.186.194:45125/ using document root <none> and password file <none>
I20250115 03:06:10.682196 14475 fs_manager.cc:362] Metadata directory not provided
I20250115 03:06:10.682247 14475 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250115 03:06:10.682405 14475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250115 03:06:10.683854 14475 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/data/instance:
uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
format_stamp: "Formatted at 2025-01-15 03:06:10 on dist-test-slave-c74p"
I20250115 03:06:10.684293 14475 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/instance:
uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
format_stamp: "Formatted at 2025-01-15 03:06:10 on dist-test-slave-c74p"
I20250115 03:06:10.687234 14475 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20250115 03:06:10.689105 14489 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250115 03:06:10.690014 14475 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:10.690097 14475 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/data,/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal
uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
format_stamp: "Formatted at 2025-01-15 03:06:10 on dist-test-slave-c74p"
I20250115 03:06:10.690160 14475 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250115 03:06:10.732239 14475 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250115 03:06:10.732728 14475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250115 03:06:10.732848 14475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250115 03:06:10.733314 14475 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250115 03:06:10.734117 14475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250115 03:06:10.734161 14475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250115 03:06:10.734196 14475 ts_tablet_manager.cc:610] Registered 0 tablets
I20250115 03:06:10.734215 14475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250115 03:06:10.742197 14475 rpc_server.cc:307] RPC server started. Bound to: 127.13.186.194:34693
I20250115 03:06:10.742238 14602 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.186.194:34693 every 8 connection(s)
I20250115 03:06:10.743021 14475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/data/info.pb
I20250115 03:06:10.749770 14059 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu as pid 14475
I20250115 03:06:10.749881 14059 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/instance
I20250115 03:06:10.752496 14059 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu
/tmp/dist-test-taskE2XgYg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.186.195:0
--local_ip_for_outbound_sockets=127.13.186.195
--webserver_interface=127.13.186.195
--webserver_port=0
--tserver_master_addrs=127.13.186.254:39901,127.13.186.253:35411,127.13.186.252:41855
--builtin_ntp_servers=127.13.186.212:37407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20250115 03:06:10.752673 14605 heartbeater.cc:346] Connected to a master server at 127.13.186.253:35411
I20250115 03:06:10.752862 14605 heartbeater.cc:463] Registering TS with master...
I20250115 03:06:10.752987 14604 heartbeater.cc:346] Connected to a master server at 127.13.186.254:39901
I20250115 03:06:10.753059 14604 heartbeater.cc:463] Registering TS with master...
I20250115 03:06:10.753372 14604 heartbeater.cc:509] Master 127.13.186.254:39901 requested a full tablet report, sending...
I20250115 03:06:10.753376 14605 heartbeater.cc:509] Master 127.13.186.253:35411 requested a full tablet report, sending...
I20250115 03:06:10.753420 14603 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:10.753526 14603 heartbeater.cc:463] Registering TS with master...
I20250115 03:06:10.753665 14603 heartbeater.cc:509] Master 127.13.186.252:41855 requested a full tablet report, sending...
I20250115 03:06:10.754278 14097 ts_manager.cc:194] Registered new tserver with Master: 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:10.754349 14171 ts_manager.cc:194] Registered new tserver with Master: 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:10.754650 14243 ts_manager.cc:194] Registered new tserver with Master: 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:10.754959 14097 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.186.194:47299
W20250115 03:06:10.850907 14610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250115 03:06:10.851073 14610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250115 03:06:10.851117 14610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250115 03:06:10.854135 14610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250115 03:06:10.854238 14610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.186.195
I20250115 03:06:10.857745 14610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.186.212:37407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--fs_data_dirs=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.186.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.186.195
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.13.186.254:39901,127.13.186.253:35411,127.13.186.252:41855
--never_fsync=true
--heap_profile_path=/tmp/kudu.14610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.186.195
--log_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 75b75ca2f58f1a5aba69207830e3558011033d99
build type DEBUG
built by None at 15 Jan 2025 02:43:26 UTC on 24a791456cd2
build id 4181
I20250115 03:06:10.858574 14610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250115 03:06:10.859520 14610 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250115 03:06:10.866703 14616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 03:06:10.866847 14615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 03:06:10.867084 14618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250115 03:06:10.867525 14610 server_base.cc:1034] running on GCE node
I20250115 03:06:10.867835 14610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250115 03:06:10.868299 14610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250115 03:06:10.869429 14610 hybrid_clock.cc:648] HybridClock initialized: now 1736910370869421 us; error 26 us; skew 500 ppm
I20250115 03:06:10.870958 14610 webserver.cc:458] Webserver started at http://127.13.186.195:41267/ using document root <none> and password file <none>
I20250115 03:06:10.871438 14610 fs_manager.cc:362] Metadata directory not provided
I20250115 03:06:10.871490 14610 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250115 03:06:10.871637 14610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250115 03:06:10.873100 14610 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/data/instance:
uuid: "5c0dae7aff4b44c699275b95ba639770"
format_stamp: "Formatted at 2025-01-15 03:06:10 on dist-test-slave-c74p"
I20250115 03:06:10.873521 14610 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/instance:
uuid: "5c0dae7aff4b44c699275b95ba639770"
format_stamp: "Formatted at 2025-01-15 03:06:10 on dist-test-slave-c74p"
I20250115 03:06:10.876430 14610 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20250115 03:06:10.878307 14624 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250115 03:06:10.879192 14610 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:10.879285 14610 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/data,/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal
uuid: "5c0dae7aff4b44c699275b95ba639770"
format_stamp: "Formatted at 2025-01-15 03:06:10 on dist-test-slave-c74p"
I20250115 03:06:10.879355 14610 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250115 03:06:10.895943 14610 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250115 03:06:10.896432 14610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250115 03:06:10.896564 14610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250115 03:06:10.897078 14610 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250115 03:06:10.897873 14610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250115 03:06:10.897918 14610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250115 03:06:10.897951 14610 ts_tablet_manager.cc:610] Registered 0 tablets
I20250115 03:06:10.898000 14610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250115 03:06:10.905444 14610 rpc_server.cc:307] RPC server started. Bound to: 127.13.186.195:40215
I20250115 03:06:10.905472 14737 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.186.195:40215 every 8 connection(s)
I20250115 03:06:10.906229 14610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/data/info.pb
I20250115 03:06:10.908859 14059 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu as pid 14610
I20250115 03:06:10.909000 14059 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/instance
I20250115 03:06:10.916009 14739 heartbeater.cc:346] Connected to a master server at 127.13.186.254:39901
I20250115 03:06:10.916006 14738 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:10.916059 14740 heartbeater.cc:346] Connected to a master server at 127.13.186.253:35411
I20250115 03:06:10.916230 14738 heartbeater.cc:463] Registering TS with master...
I20250115 03:06:10.916231 14739 heartbeater.cc:463] Registering TS with master...
I20250115 03:06:10.916231 14740 heartbeater.cc:463] Registering TS with master...
I20250115 03:06:10.916759 14739 heartbeater.cc:509] Master 127.13.186.254:39901 requested a full tablet report, sending...
I20250115 03:06:10.916752 14740 heartbeater.cc:509] Master 127.13.186.253:35411 requested a full tablet report, sending...
I20250115 03:06:10.916738 14738 heartbeater.cc:509] Master 127.13.186.252:41855 requested a full tablet report, sending...
I20250115 03:06:10.917760 14243 ts_manager.cc:194] Registered new tserver with Master: 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:10.917760 14097 ts_manager.cc:194] Registered new tserver with Master: 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:10.917783 14171 ts_manager.cc:194] Registered new tserver with Master: 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:10.918273 14097 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.186.195:57963
I20250115 03:06:10.927831 14059 external_mini_cluster.cc:934] 3 TS(s) registered with all masters
I20250115 03:06:10.939029 14059 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu with pid 14067
W20250115 03:06:10.945233 14747 connection.cc:537] client connection to 127.13.186.254:39901 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20250115 03:06:10.945660 14747 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.13.186.254:39901)
I20250115 03:06:11.070398 14469 heartbeater.cc:501] Master 127.13.186.254:39901 was elected leader, sending a full tablet report...
W20250115 03:06:11.071208 14469 heartbeater.cc:646] Failed to heartbeat to 127.13.186.254:39901 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.186.254:39901: connect: Connection refused (error 111)
I20250115 03:06:11.195209 14299 catalog_manager.cc:1260] Loaded cluster ID: 6933a79146ea47b6911c92899078d551
I20250115 03:06:11.195284 14299 catalog_manager.cc:1553] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246: loading cluster ID for follower catalog manager: success
I20250115 03:06:11.197021 14299 catalog_manager.cc:1575] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246: acquiring CA information for follower catalog manager: success
I20250115 03:06:11.198288 14299 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250115 03:06:11.231976 14784 raft_consensus.cc:491] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4225fe2fbfa84937bb41fed702ca66e4)
I20250115 03:06:11.232062 14784 raft_consensus.cc:513] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } }
I20250115 03:06:11.232841 14784 leader_election.cc:290] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4225fe2fbfa84937bb41fed702ca66e4 (127.13.186.254:39901), 147c86b98f604d518e71f2ba3121fc21 (127.13.186.253:35411)
I20250115 03:06:11.233198 14180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2db8cff032af46db986bf52b21b52246" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "147c86b98f604d518e71f2ba3121fc21" is_pre_election: true
W20250115 03:06:11.233319 14229 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.186.254:39901: connect: Connection refused (error 111)
W20250115 03:06:11.234205 14229 leader_election.cc:336] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4225fe2fbfa84937bb41fed702ca66e4 (127.13.186.254:39901): Network error: Client connection negotiation failed: client connection to 127.13.186.254:39901: connect: Connection refused (error 111)
I20250115 03:06:11.234282 14229 leader_election.cc:304] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2db8cff032af46db986bf52b21b52246; no voters: 147c86b98f604d518e71f2ba3121fc21, 4225fe2fbfa84937bb41fed702ca66e4
I20250115 03:06:11.234457 14784 raft_consensus.cc:2743] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250115 03:06:11.256593 14604 heartbeater.cc:501] Master 127.13.186.254:39901 was elected leader, sending a full tablet report...
W20250115 03:06:11.257344 14604 heartbeater.cc:646] Failed to heartbeat to 127.13.186.254:39901 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.186.254:39901: connect: Connection refused (error 111)
I20250115 03:06:11.268379 14330 catalog_manager.cc:1260] Loaded cluster ID: 6933a79146ea47b6911c92899078d551
I20250115 03:06:11.268433 14330 catalog_manager.cc:1553] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: loading cluster ID for follower catalog manager: success
I20250115 03:06:11.270186 14330 catalog_manager.cc:1575] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: acquiring CA information for follower catalog manager: success
I20250115 03:06:11.271399 14330 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250115 03:06:11.282990 14788 raft_consensus.cc:491] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4225fe2fbfa84937bb41fed702ca66e4)
I20250115 03:06:11.283087 14788 raft_consensus.cc:513] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } }
I20250115 03:06:11.283809 14788 leader_election.cc:290] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4225fe2fbfa84937bb41fed702ca66e4 (127.13.186.254:39901), 2db8cff032af46db986bf52b21b52246 (127.13.186.252:41855)
I20250115 03:06:11.284159 14253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "147c86b98f604d518e71f2ba3121fc21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2db8cff032af46db986bf52b21b52246" is_pre_election: true
I20250115 03:06:11.284273 14253 raft_consensus.cc:2463] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 147c86b98f604d518e71f2ba3121fc21 in term 1.
I20250115 03:06:11.284611 14156 leader_election.cc:304] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 147c86b98f604d518e71f2ba3121fc21, 2db8cff032af46db986bf52b21b52246; no voters:
I20250115 03:06:11.284828 14788 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250115 03:06:11.284893 14788 raft_consensus.cc:491] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4225fe2fbfa84937bb41fed702ca66e4)
I20250115 03:06:11.284957 14788 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [term 1 FOLLOWER]: Advancing to term 2
W20250115 03:06:11.284999 14157 leader_election.cc:336] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4225fe2fbfa84937bb41fed702ca66e4 (127.13.186.254:39901): Network error: Client connection negotiation failed: client connection to 127.13.186.254:39901: connect: Connection refused (error 111)
I20250115 03:06:11.285804 14788 raft_consensus.cc:513] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } }
I20250115 03:06:11.286087 14788 leader_election.cc:290] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [CANDIDATE]: Term 2 election: Requested vote from peers 4225fe2fbfa84937bb41fed702ca66e4 (127.13.186.254:39901), 2db8cff032af46db986bf52b21b52246 (127.13.186.252:41855)
I20250115 03:06:11.286383 14253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "147c86b98f604d518e71f2ba3121fc21" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "2db8cff032af46db986bf52b21b52246"
I20250115 03:06:11.286507 14253 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [term 1 FOLLOWER]: Advancing to term 2
W20250115 03:06:11.286963 14157 leader_election.cc:336] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4225fe2fbfa84937bb41fed702ca66e4 (127.13.186.254:39901): Network error: Client connection negotiation failed: client connection to 127.13.186.254:39901: connect: Connection refused (error 111)
I20250115 03:06:11.287258 14253 raft_consensus.cc:2463] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 147c86b98f604d518e71f2ba3121fc21 in term 2.
I20250115 03:06:11.287524 14156 leader_election.cc:304] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 147c86b98f604d518e71f2ba3121fc21, 2db8cff032af46db986bf52b21b52246; no voters: 4225fe2fbfa84937bb41fed702ca66e4
I20250115 03:06:11.287672 14788 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [term 2 FOLLOWER]: Leader election won for term 2
I20250115 03:06:11.287827 14788 raft_consensus.cc:695] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [term 2 LEADER]: Becoming Leader. State: Replica: 147c86b98f604d518e71f2ba3121fc21, State: Running, Role: LEADER
I20250115 03:06:11.288064 14788 consensus_queue.cc:237] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } }
I20250115 03:06:11.288758 14791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 147c86b98f604d518e71f2ba3121fc21. Latest consensus state: current_term: 2 leader_uuid: "147c86b98f604d518e71f2ba3121fc21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:11.288854 14791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [sys.catalog]: This master's current role is: LEADER
I20250115 03:06:11.289160 14792 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250115 03:06:11.289872 14792 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250115 03:06:11.290191 14792 catalog_manager.cc:1260] Loaded cluster ID: 6933a79146ea47b6911c92899078d551
I20250115 03:06:11.290236 14792 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250115 03:06:11.290707 14792 catalog_manager.cc:1505] Loading token signing keys...
I20250115 03:06:11.291102 14792 catalog_manager.cc:5910] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Loaded TSK: 0
I20250115 03:06:11.291714 14792 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250115 03:06:11.296221 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_fd521b8459734272a3dabed5fea57790"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.296376 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_035c2cbf92ca438ea4e6034f81146326"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.296579 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d37e680ce96b4e32ba9fffce8733415f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.296253 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_8ca8b30c518a45a9a5b634a9149c39be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.296221 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_bef8903aa780435ab033ed6629db6cec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.296593 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_8380beb77f8d436f8a45d11164889ea5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.296798 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_3d0c951d6ea7491798b994cfe256cca4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.296403 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_34e1332f5bbc4fd9a9a586dc613d2fe7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.296774 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_aae0be0952964e618e5aabba82ecc215"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.296691 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0d1a416fff2a4f07bc52910954d44e6d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:11.298945 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8ca8b30c518a45a9a5b634a9149c39be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:11.298965 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aae0be0952964e618e5aabba82ecc215 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:11.299072 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bef8903aa780435ab033ed6629db6cec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:11.299077 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8380beb77f8d436f8a45d11164889ea5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:11.299183 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3d0c951d6ea7491798b994cfe256cca4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:11.299400 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0d1a416fff2a4f07bc52910954d44e6d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:11.299518 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_34e1332f5bbc4fd9a9a586dc613d2fe7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:11.299554 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d37e680ce96b4e32ba9fffce8733415f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:11.300074 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_035c2cbf92ca438ea4e6034f81146326 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:11.301545 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fd521b8459734272a3dabed5fea57790 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.302333 14253 raft_consensus.cc:1270] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [term 2 FOLLOWER]: Refusing update from remote peer 147c86b98f604d518e71f2ba3121fc21: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 6. (index mismatch)
I20250115 03:06:11.302769 14788 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20250115 03:06:11.303082 14157 consensus_peers.cc:487] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 -> Peer 4225fe2fbfa84937bb41fed702ca66e4 (127.13.186.254:39901): Couldn't send request to peer 4225fe2fbfa84937bb41fed702ca66e4. Status: Network error: Client connection negotiation failed: client connection to 127.13.186.254:39901: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:11.304493 14784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 147c86b98f604d518e71f2ba3121fc21. Latest consensus state: current_term: 2 leader_uuid: "147c86b98f604d518e71f2ba3121fc21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:11.304584 14784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [sys.catalog]: This master's current role is: FOLLOWER
I20250115 03:06:11.305910 14791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "147c86b98f604d518e71f2ba3121fc21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:11.306062 14791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [sys.catalog]: This master's current role is: LEADER
I20250115 03:06:11.306998 14784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "147c86b98f604d518e71f2ba3121fc21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:11.307092 14784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [sys.catalog]: This master's current role is: FOLLOWER
I20250115 03:06:11.308550 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0d1a416fff2a4f07bc52910954d44e6d" } new_table_name: "default.Table_d46d2affc6e74c7a8d3cc2a8ec0233e6" modify_external_catalogs: true
I20250115 03:06:11.310231 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_aae0be0952964e618e5aabba82ecc215" } new_table_name: "default.Table_ff7c7c1250d74aec9ace68037a6d6a86" modify_external_catalogs: true
I20250115 03:06:11.311658 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d37e680ce96b4e32ba9fffce8733415f" } new_table_name: "default.Table_9928f846023c45a0bc9c92d5447958cb" modify_external_catalogs: true
I20250115 03:06:11.312242 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_035c2cbf92ca438ea4e6034f81146326" } new_table_name: "default.Table_017cebf1769e47a997604ce80cd74d9b" modify_external_catalogs: true
I20250115 03:06:11.312361 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8380beb77f8d436f8a45d11164889ea5" } modify_external_catalogs: true
I20250115 03:06:11.312393 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_34e1332f5bbc4fd9a9a586dc613d2fe7" } modify_external_catalogs: true
I20250115 03:06:11.312436 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8380beb77f8d436f8a45d11164889ea5" } modify_external_catalogs: true
I20250115 03:06:11.312469 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_34e1332f5bbc4fd9a9a586dc613d2fe7" } modify_external_catalogs: true
I20250115 03:06:11.312251 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3d0c951d6ea7491798b994cfe256cca4" } new_table_name: "default.Table_5dec8aa2c23146558b4c71b9ab0a48b5" modify_external_catalogs: true
I20250115 03:06:11.315585 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8ca8b30c518a45a9a5b634a9149c39be" } modify_external_catalogs: true
I20250115 03:06:11.315676 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8ca8b30c518a45a9a5b634a9149c39be" } modify_external_catalogs: true
I20250115 03:06:11.318629 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_fd521b8459734272a3dabed5fea57790" } modify_external_catalogs: true
W20250115 03:06:11.318632 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cc88653cdfec4120b37097b361f441d4 has no consensus state
I20250115 03:06:11.318713 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_bef8903aa780435ab033ed6629db6cec" } modify_external_catalogs: true
W20250115 03:06:11.318810 14163 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 44585e70c3434fa1915d31c46b1c0d50 consensus state has no leader
I20250115 03:06:11.318930 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_fd521b8459734272a3dabed5fea57790" } modify_external_catalogs: true
I20250115 03:06:11.318794 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_bef8903aa780435ab033ed6629db6cec" } modify_external_catalogs: true
I20250115 03:06:11.320510 14165 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet 274a20c7b1944ece99decf418366f893
W20250115 03:06:11.320583 14167 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5006eb8055b04bf79773830de5974ef6 has no consensus state
W20250115 03:06:11.321128 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fa0228294e0c42f3bd637838cf58c692 consensus state has no leader
W20250115 03:06:11.321655 14162 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 94c861b4ff2244deb37a850340f71a3d has no consensus state
I20250115 03:06:11.321918 14164 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet e3c573873ed34ee8a8ee9d6a1b230d8e
I20250115 03:06:11.324664 14170 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet 6d7a650e0d0946c299a9d4cbb8d304e8
I20250115 03:06:11.325060 14169 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet f046668d406449e5b3e3aecadcdd8b82
I20250115 03:06:11.326396 14171 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet ac06c0dc508a4560a2a30d0fa09a556d
I20250115 03:06:11.326167 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 44585e70c3434fa1915d31c46b1c0d50 (DEFAULT_TABLE table=default.Table_0d1a416fff2a4f07bc52910954d44e6d [id=d9fda51f2bed4c1ca720fe30ec97f440]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.327096 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44585e70c3434fa1915d31c46b1c0d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.329093 14401 tablet_service.cc:1467] Processing CreateTablet for tablet fa0228294e0c42f3bd637838cf58c692 (DEFAULT_TABLE table=default.Table_d37e680ce96b4e32ba9fffce8733415f [id=39abf3d98dba4d63979ccc4c27af3a97]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.329342 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa0228294e0c42f3bd637838cf58c692. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.329576 14671 tablet_service.cc:1467] Processing CreateTablet for tablet fa0228294e0c42f3bd637838cf58c692 (DEFAULT_TABLE table=default.Table_d37e680ce96b4e32ba9fffce8733415f [id=39abf3d98dba4d63979ccc4c27af3a97]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.330400 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa0228294e0c42f3bd637838cf58c692. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.331341 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 44585e70c3434fa1915d31c46b1c0d50 (DEFAULT_TABLE table=default.Table_0d1a416fff2a4f07bc52910954d44e6d [id=d9fda51f2bed4c1ca720fe30ec97f440]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.332151 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44585e70c3434fa1915d31c46b1c0d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.333200 14536 tablet_service.cc:1467] Processing CreateTablet for tablet fa0228294e0c42f3bd637838cf58c692 (DEFAULT_TABLE table=default.Table_d37e680ce96b4e32ba9fffce8733415f [id=39abf3d98dba4d63979ccc4c27af3a97]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.333649 14809 tablet_bootstrap.cc:492] T fa0228294e0c42f3bd637838cf58c692 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.335808 14669 tablet_service.cc:1467] Processing CreateTablet for tablet cc88653cdfec4120b37097b361f441d4 (DEFAULT_TABLE table=default.Table_ff7c7c1250d74aec9ace68037a6d6a86 [id=96a7774dedae4e7fb54cb70b3e8a85fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.335808 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 94c861b4ff2244deb37a850340f71a3d (DEFAULT_TABLE table=default.Table_5dec8aa2c23146558b4c71b9ab0a48b5 [id=1b4a3ef0246a4341869bf5163487cac6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.336052 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc88653cdfec4120b37097b361f441d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.336108 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 5006eb8055b04bf79773830de5974ef6 (DEFAULT_TABLE table=default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.336321 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5006eb8055b04bf79773830de5974ef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.336426 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 44585e70c3434fa1915d31c46b1c0d50 (DEFAULT_TABLE table=default.Table_0d1a416fff2a4f07bc52910954d44e6d [id=d9fda51f2bed4c1ca720fe30ec97f440]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.336627 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44585e70c3434fa1915d31c46b1c0d50. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.338384 14401 tablet_service.cc:1467] Processing CreateTablet for tablet cc88653cdfec4120b37097b361f441d4 (DEFAULT_TABLE table=default.Table_ff7c7c1250d74aec9ace68037a6d6a86 [id=96a7774dedae4e7fb54cb70b3e8a85fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.338599 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc88653cdfec4120b37097b361f441d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.338789 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 94c861b4ff2244deb37a850340f71a3d (DEFAULT_TABLE table=default.Table_5dec8aa2c23146558b4c71b9ab0a48b5 [id=1b4a3ef0246a4341869bf5163487cac6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.338862 14534 tablet_service.cc:1467] Processing CreateTablet for tablet cc88653cdfec4120b37097b361f441d4 (DEFAULT_TABLE table=default.Table_ff7c7c1250d74aec9ace68037a6d6a86 [id=96a7774dedae4e7fb54cb70b3e8a85fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.339183 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 5006eb8055b04bf79773830de5974ef6 (DEFAULT_TABLE table=default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.341032 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94c861b4ff2244deb37a850340f71a3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.341424 14809 tablet_bootstrap.cc:654] T fa0228294e0c42f3bd637838cf58c692 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.342628 14809 log.cc:826] T fa0228294e0c42f3bd637838cf58c692 P e8df7247ee9c43e0aa18216da2661f75: Log is configured to *not* fsync() on all Append() calls
I20250115 03:06:11.343434 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 5006eb8055b04bf79773830de5974ef6 (DEFAULT_TABLE table=default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.343659 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5006eb8055b04bf79773830de5974ef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.344065 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa0228294e0c42f3bd637838cf58c692. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.344311 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94c861b4ff2244deb37a850340f71a3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.344547 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc88653cdfec4120b37097b361f441d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.345096 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5006eb8055b04bf79773830de5974ef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.346580 14809 tablet_bootstrap.cc:492] T fa0228294e0c42f3bd637838cf58c692 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.346796 14809 ts_tablet_manager.cc:1397] T fa0228294e0c42f3bd637838cf58c692 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20250115 03:06:11.347141 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 94c861b4ff2244deb37a850340f71a3d (DEFAULT_TABLE table=default.Table_5dec8aa2c23146558b4c71b9ab0a48b5 [id=1b4a3ef0246a4341869bf5163487cac6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.347422 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94c861b4ff2244deb37a850340f71a3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.348848 14811 tablet_bootstrap.cc:492] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
W20250115 03:06:11.349323 14158 catalog_manager.cc:4670] Async tablet task AlterTable RPC for tablet 5006eb8055b04bf79773830de5974ef6 (table default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5006eb8055b04bf79773830de5974ef6 consensus state has no leader
I20250115 03:06:11.349457 14812 tablet_bootstrap.cc:492] T fa0228294e0c42f3bd637838cf58c692 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.350445 14809 raft_consensus.cc:357] T fa0228294e0c42f3bd637838cf58c692 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.350718 14809 raft_consensus.cc:383] T fa0228294e0c42f3bd637838cf58c692 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.350813 14809 raft_consensus.cc:738] T fa0228294e0c42f3bd637838cf58c692 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.350943 14811 tablet_bootstrap.cc:654] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.351459 14812 tablet_bootstrap.cc:654] T fa0228294e0c42f3bd637838cf58c692 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.351486 14809 consensus_queue.cc:260] T fa0228294e0c42f3bd637838cf58c692 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.351744 14811 log.cc:826] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1: Log is configured to *not* fsync() on all Append() calls
I20250115 03:06:11.352217 14812 log.cc:826] T fa0228294e0c42f3bd637838cf58c692 P 5c0dae7aff4b44c699275b95ba639770: Log is configured to *not* fsync() on all Append() calls
I20250115 03:06:11.352437 14809 ts_tablet_manager.cc:1428] T fa0228294e0c42f3bd637838cf58c692 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250115 03:06:11.352777 14809 tablet_bootstrap.cc:492] T 44585e70c3434fa1915d31c46b1c0d50 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.354969 14171 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet fa0228294e0c42f3bd637838cf58c692 (table default.Table_9928f846023c45a0bc9c92d5447958cb [id=39abf3d98dba4d63979ccc4c27af3a97]). Expected version 1 got 0
I20250115 03:06:11.355991 14809 tablet_bootstrap.cc:654] T 44585e70c3434fa1915d31c46b1c0d50 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.356796 14470 heartbeater.cc:501] Master 127.13.186.253:35411 was elected leader, sending a full tablet report...
I20250115 03:06:11.357021 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet 5006eb8055b04bf79773830de5974ef6 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:11.358834 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet fa0228294e0c42f3bd637838cf58c692 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:11.358913 14170 master_service.cc:946] ReplaceTablet: received request to replace tablet fa0228294e0c42f3bd637838cf58c692 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:11.360239 14811 tablet_bootstrap.cc:492] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.360442 14812 tablet_bootstrap.cc:492] T fa0228294e0c42f3bd637838cf58c692 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.360447 14811 ts_tablet_manager.cc:1397] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20250115 03:06:11.360603 14812 ts_tablet_manager.cc:1397] T fa0228294e0c42f3bd637838cf58c692 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20250115 03:06:11.362915 14811 raft_consensus.cc:357] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.363097 14811 raft_consensus.cc:383] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.363181 14811 raft_consensus.cc:738] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.363545 14811 consensus_queue.cc:260] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.363834 14812 raft_consensus.cc:357] T fa0228294e0c42f3bd637838cf58c692 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.364094 14162 master_service.cc:946] ReplaceTablet: received request to replace tablet 94c861b4ff2244deb37a850340f71a3d from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:11.364281 14811 ts_tablet_manager.cc:1428] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20250115 03:06:11.364760 14166 master_service.cc:946] ReplaceTablet: received request to replace tablet 94c861b4ff2244deb37a850340f71a3d from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:11.364782 14809 tablet_bootstrap.cc:492] T 44585e70c3434fa1915d31c46b1c0d50 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.364851 14809 ts_tablet_manager.cc:1397] T 44585e70c3434fa1915d31c46b1c0d50 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20250115 03:06:11.365360 14809 raft_consensus.cc:357] T 44585e70c3434fa1915d31c46b1c0d50 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.365475 14809 raft_consensus.cc:383] T 44585e70c3434fa1915d31c46b1c0d50 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.365509 14809 raft_consensus.cc:738] T 44585e70c3434fa1915d31c46b1c0d50 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.365617 14809 consensus_queue.cc:260] T 44585e70c3434fa1915d31c46b1c0d50 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.365867 14809 ts_tablet_manager.cc:1428] T 44585e70c3434fa1915d31c46b1c0d50 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:11.366024 14809 tablet_bootstrap.cc:492] T 5006eb8055b04bf79773830de5974ef6 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.367024 14809 tablet_bootstrap.cc:654] T 5006eb8055b04bf79773830de5974ef6 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.367937 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 5006eb8055b04bf79773830de5974ef6
I20250115 03:06:11.368275 14171 catalog_manager.cc:6596] ReplaceTablet: tablet 5006eb8055b04bf79773830de5974ef6 deleted and replaced by tablet 180df492348a4cc09afa03c8a62a79c1
I20250115 03:06:11.368505 14402 tablet_service.cc:1514] Processing DeleteTablet for tablet 5006eb8055b04bf79773830de5974ef6 with delete_type TABLET_DATA_DELETED (replaced by tablet 180df492348a4cc09afa03c8a62a79c1) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.369269 14809 tablet_bootstrap.cc:492] T 5006eb8055b04bf79773830de5974ef6 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.369345 14809 ts_tablet_manager.cc:1397] T 5006eb8055b04bf79773830de5974ef6 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20250115 03:06:11.369478 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet 5006eb8055b04bf79773830de5974ef6 with delete_type TABLET_DATA_DELETED (replaced by tablet 180df492348a4cc09afa03c8a62a79c1) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.369728 14809 raft_consensus.cc:357] T 5006eb8055b04bf79773830de5974ef6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.369807 14809 raft_consensus.cc:383] T 5006eb8055b04bf79773830de5974ef6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.369835 14809 raft_consensus.cc:738] T 5006eb8055b04bf79773830de5974ef6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.369938 14809 consensus_queue.cc:260] T 5006eb8055b04bf79773830de5974ef6 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.370244 14809 ts_tablet_manager.cc:1428] T 5006eb8055b04bf79773830de5974ef6 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:11.370390 14809 tablet_bootstrap.cc:492] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.371250 14809 tablet_bootstrap.cc:654] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.371698 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet fa0228294e0c42f3bd637838cf58c692
I20250115 03:06:11.371928 14811 tablet_bootstrap.cc:492] T fa0228294e0c42f3bd637838cf58c692 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.371944 14164 catalog_manager.cc:6596] ReplaceTablet: tablet fa0228294e0c42f3bd637838cf58c692 deleted and replaced by tablet 0147f9743e924c9fb82a5bcd872e79fd
I20250115 03:06:11.373035 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 94c861b4ff2244deb37a850340f71a3d
I20250115 03:06:11.373158 14811 tablet_bootstrap.cc:654] T fa0228294e0c42f3bd637838cf58c692 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.373265 14162 catalog_manager.cc:6596] ReplaceTablet: tablet 94c861b4ff2244deb37a850340f71a3d deleted and replaced by tablet 063999d856bf4ac28bb7ec01509d04a4
I20250115 03:06:11.373586 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 44585e70c3434fa1915d31c46b1c0d50 (table default.Table_d46d2affc6e74c7a8d3cc2a8ec0233e6 [id=d9fda51f2bed4c1ca720fe30ec97f440]). Expected version 1 got 0
I20250115 03:06:11.373742 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet cc88653cdfec4120b37097b361f441d4 (table default.Table_ff7c7c1250d74aec9ace68037a6d6a86 [id=96a7774dedae4e7fb54cb70b3e8a85fd]). Expected version 1 got 0
I20250115 03:06:11.374071 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 94c861b4ff2244deb37a850340f71a3d with delete_type TABLET_DATA_DELETED (replaced by tablet 063999d856bf4ac28bb7ec01509d04a4) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.374333 14402 tablet_service.cc:1514] Processing DeleteTablet for tablet fa0228294e0c42f3bd637838cf58c692 with delete_type TABLET_DATA_DELETED (replaced by tablet 0147f9743e924c9fb82a5bcd872e79fd) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.374958 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 94c861b4ff2244deb37a850340f71a3d with delete_type TABLET_DATA_DELETED (replaced by tablet 063999d856bf4ac28bb7ec01509d04a4) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.375198 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet fa0228294e0c42f3bd637838cf58c692 with delete_type TABLET_DATA_DELETED (replaced by tablet 0147f9743e924c9fb82a5bcd872e79fd) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.376596 14812 raft_consensus.cc:383] T fa0228294e0c42f3bd637838cf58c692 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.376672 14812 raft_consensus.cc:738] T fa0228294e0c42f3bd637838cf58c692 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.368901 14671 tablet_service.cc:1514] Processing DeleteTablet for tablet 5006eb8055b04bf79773830de5974ef6 with delete_type TABLET_DATA_DELETED (replaced by tablet 180df492348a4cc09afa03c8a62a79c1) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:11.373649 14670 tablet_service.cc:1514] Processing DeleteTablet for tablet fa0228294e0c42f3bd637838cf58c692 with delete_type TABLET_DATA_DELETED (replaced by tablet 0147f9743e924c9fb82a5bcd872e79fd) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:11.373842 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 94c861b4ff2244deb37a850340f71a3d with delete_type TABLET_DATA_DELETED (replaced by tablet 063999d856bf4ac28bb7ec01509d04a4) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:11.377166 14812 consensus_queue.cc:260] T fa0228294e0c42f3bd637838cf58c692 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.377490 14809 tablet_bootstrap.cc:492] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.377562 14809 ts_tablet_manager.cc:1397] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20250115 03:06:11.377939 14809 raft_consensus.cc:357] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.377992 14812 ts_tablet_manager.cc:1428] T fa0228294e0c42f3bd637838cf58c692 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.017s user 0.004s sys 0.000s
I20250115 03:06:11.378033 14809 raft_consensus.cc:383] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.378064 14809 raft_consensus.cc:738] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.378161 14809 consensus_queue.cc:260] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:11.378234 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 5006eb8055b04bf79773830de5974ef6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5006eb8055b04bf79773830de5974ef6 already in progress: creating tablet
I20250115 03:06:11.378387 14809 ts_tablet_manager.cc:1428] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.378537 14809 tablet_bootstrap.cc:492] T 94c861b4ff2244deb37a850340f71a3d P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.379369 14809 tablet_bootstrap.cc:654] T 94c861b4ff2244deb37a850340f71a3d P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.379437 14166 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 44585e70c3434fa1915d31c46b1c0d50 (table default.Table_d46d2affc6e74c7a8d3cc2a8ec0233e6 [id=d9fda51f2bed4c1ca720fe30ec97f440]). Expected version 1 got 0
I20250115 03:06:11.380656 14739 heartbeater.cc:501] Master 127.13.186.254:39901 was elected leader, sending a full tablet report...
I20250115 03:06:11.380646 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 180df492348a4cc09afa03c8a62a79c1 (DEFAULT_TABLE table=default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.380867 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 180df492348a4cc09afa03c8a62a79c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.383141 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 180df492348a4cc09afa03c8a62a79c1 (DEFAULT_TABLE table=default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.383359 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 180df492348a4cc09afa03c8a62a79c1. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:11.385049 14739 heartbeater.cc:646] Failed to heartbeat to 127.13.186.254:39901 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.186.254:39901: connect: Connection refused (error 111)
I20250115 03:06:11.385102 14819 tablet_replica.cc:331] T 5006eb8055b04bf79773830de5974ef6 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:11.385361 14819 raft_consensus.cc:2238] T 5006eb8055b04bf79773830de5974ef6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.385615 14819 raft_consensus.cc:2267] T 5006eb8055b04bf79773830de5974ef6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.385854 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 063999d856bf4ac28bb7ec01509d04a4 (DEFAULT_TABLE table=default.Table_5dec8aa2c23146558b4c71b9ab0a48b5 [id=1b4a3ef0246a4341869bf5163487cac6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.386076 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 063999d856bf4ac28bb7ec01509d04a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.386253 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 0147f9743e924c9fb82a5bcd872e79fd (DEFAULT_TABLE table=default.Table_9928f846023c45a0bc9c92d5447958cb [id=39abf3d98dba4d63979ccc4c27af3a97]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.386472 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0147f9743e924c9fb82a5bcd872e79fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.386488 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 0147f9743e924c9fb82a5bcd872e79fd (DEFAULT_TABLE table=default.Table_9928f846023c45a0bc9c92d5447958cb [id=39abf3d98dba4d63979ccc4c27af3a97]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.386682 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0147f9743e924c9fb82a5bcd872e79fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.388461 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 180df492348a4cc09afa03c8a62a79c1 (DEFAULT_TABLE table=default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.388523 14812 tablet_bootstrap.cc:492] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.388671 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 180df492348a4cc09afa03c8a62a79c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.389523 14812 tablet_bootstrap.cc:654] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:11.390552 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 5006eb8055b04bf79773830de5974ef6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5006eb8055b04bf79773830de5974ef6 already in progress: creating tablet
W20250115 03:06:11.390736 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 94c861b4ff2244deb37a850340f71a3d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 94c861b4ff2244deb37a850340f71a3d already in progress: creating tablet
I20250115 03:06:11.390818 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 0147f9743e924c9fb82a5bcd872e79fd (DEFAULT_TABLE table=default.Table_9928f846023c45a0bc9c92d5447958cb [id=39abf3d98dba4d63979ccc4c27af3a97]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:11.390921 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet fa0228294e0c42f3bd637838cf58c692 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa0228294e0c42f3bd637838cf58c692 already in progress: creating tablet
I20250115 03:06:11.391037 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0147f9743e924c9fb82a5bcd872e79fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.391441 14809 tablet_bootstrap.cc:492] T 94c861b4ff2244deb37a850340f71a3d P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.391508 14809 ts_tablet_manager.cc:1397] T 94c861b4ff2244deb37a850340f71a3d P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20250115 03:06:11.391764 14819 ts_tablet_manager.cc:1905] T 5006eb8055b04bf79773830de5974ef6 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.392024 14809 raft_consensus.cc:357] T 94c861b4ff2244deb37a850340f71a3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.392113 14809 raft_consensus.cc:383] T 94c861b4ff2244deb37a850340f71a3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.392143 14809 raft_consensus.cc:738] T 94c861b4ff2244deb37a850340f71a3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.392243 14809 consensus_queue.cc:260] T 94c861b4ff2244deb37a850340f71a3d P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.392469 14809 ts_tablet_manager.cc:1428] T 94c861b4ff2244deb37a850340f71a3d P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.392611 14809 tablet_bootstrap.cc:492] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.393512 14809 tablet_bootstrap.cc:654] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.394620 14812 tablet_bootstrap.cc:492] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.394626 14166 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet cc88653cdfec4120b37097b361f441d4 (table default.Table_ff7c7c1250d74aec9ace68037a6d6a86 [id=96a7774dedae4e7fb54cb70b3e8a85fd]). Expected version 1 got 0
I20250115 03:06:11.394711 14812 ts_tablet_manager.cc:1397] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250115 03:06:11.394953 14819 ts_tablet_manager.cc:1918] T 5006eb8055b04bf79773830de5974ef6 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.395026 14819 log.cc:1198] T 5006eb8055b04bf79773830de5974ef6 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/5006eb8055b04bf79773830de5974ef6
I20250115 03:06:11.395257 14819 ts_tablet_manager.cc:1939] T 5006eb8055b04bf79773830de5974ef6 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:11.395238 14812 raft_consensus.cc:357] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.395357 14812 raft_consensus.cc:383] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.395385 14812 raft_consensus.cc:738] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.395494 14812 consensus_queue.cc:260] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.395823 14812 ts_tablet_manager.cc:1428] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.396209 14819 tablet_replica.cc:331] T 94c861b4ff2244deb37a850340f71a3d P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:11.396301 14819 raft_consensus.cc:2238] T 94c861b4ff2244deb37a850340f71a3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.396338 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 5006eb8055b04bf79773830de5974ef6 (table default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]) successfully deleted
I20250115 03:06:11.396384 14819 raft_consensus.cc:2267] T 94c861b4ff2244deb37a850340f71a3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.396559 14740 heartbeater.cc:501] Master 127.13.186.253:35411 was elected leader, sending a full tablet report...
I20250115 03:06:11.396788 14819 ts_tablet_manager.cc:1905] T 94c861b4ff2244deb37a850340f71a3d P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.397020 14812 tablet_bootstrap.cc:492] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.397922 14812 tablet_bootstrap.cc:654] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.397944 14169 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 44585e70c3434fa1915d31c46b1c0d50 (table default.Table_d46d2affc6e74c7a8d3cc2a8ec0233e6 [id=d9fda51f2bed4c1ca720fe30ec97f440]). Expected version 1 got 0
I20250115 03:06:11.398523 14819 ts_tablet_manager.cc:1918] T 94c861b4ff2244deb37a850340f71a3d P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.398552 14605 heartbeater.cc:501] Master 127.13.186.253:35411 was elected leader, sending a full tablet report...
I20250115 03:06:11.398589 14819 log.cc:1198] T 94c861b4ff2244deb37a850340f71a3d P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/94c861b4ff2244deb37a850340f71a3d
I20250115 03:06:11.398773 14819 ts_tablet_manager.cc:1939] T 94c861b4ff2244deb37a850340f71a3d P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:11.399199 14819 tablet_replica.cc:331] T fa0228294e0c42f3bd637838cf58c692 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:11.399289 14819 raft_consensus.cc:2238] T fa0228294e0c42f3bd637838cf58c692 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.399369 14819 raft_consensus.cc:2267] T fa0228294e0c42f3bd637838cf58c692 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.399434 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 94c861b4ff2244deb37a850340f71a3d (table default.Table_5dec8aa2c23146558b4c71b9ab0a48b5 [id=1b4a3ef0246a4341869bf5163487cac6]) successfully deleted
I20250115 03:06:11.399773 14809 tablet_bootstrap.cc:492] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.399816 14811 tablet_bootstrap.cc:492] T fa0228294e0c42f3bd637838cf58c692 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.399847 14809 ts_tablet_manager.cc:1397] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:11.399888 14811 ts_tablet_manager.cc:1397] T fa0228294e0c42f3bd637838cf58c692 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.000s
I20250115 03:06:11.400209 14809 raft_consensus.cc:357] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.400290 14809 raft_consensus.cc:383] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.400317 14809 raft_consensus.cc:738] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.400338 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 0147f9743e924c9fb82a5bcd872e79fd (table default.Table_9928f846023c45a0bc9c92d5447958cb [id=39abf3d98dba4d63979ccc4c27af3a97]). Expected version 1 got 0
I20250115 03:06:11.400424 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet cc88653cdfec4120b37097b361f441d4 (table default.Table_ff7c7c1250d74aec9ace68037a6d6a86 [id=96a7774dedae4e7fb54cb70b3e8a85fd]). Expected version 1 got 0
I20250115 03:06:11.400413 14809 consensus_queue.cc:260] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.400425 14811 raft_consensus.cc:357] T fa0228294e0c42f3bd637838cf58c692 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.400498 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 180df492348a4cc09afa03c8a62a79c1 (table default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]). Expected version 1 got 0
I20250115 03:06:11.400518 14811 raft_consensus.cc:383] T fa0228294e0c42f3bd637838cf58c692 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.400549 14811 raft_consensus.cc:738] T fa0228294e0c42f3bd637838cf58c692 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.400561 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 44585e70c3434fa1915d31c46b1c0d50 (table default.Table_d46d2affc6e74c7a8d3cc2a8ec0233e6 [id=d9fda51f2bed4c1ca720fe30ec97f440]). Expected version 1 got 0
I20250115 03:06:11.400622 14809 ts_tablet_manager.cc:1428] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.400660 14811 consensus_queue.cc:260] T fa0228294e0c42f3bd637838cf58c692 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.400777 14809 tablet_bootstrap.cc:492] T 0147f9743e924c9fb82a5bcd872e79fd P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.400913 14811 ts_tablet_manager.cc:1428] T fa0228294e0c42f3bd637838cf58c692 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.401095 14811 tablet_bootstrap.cc:492] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.401618 14809 tablet_bootstrap.cc:654] T 0147f9743e924c9fb82a5bcd872e79fd P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.401952 14811 tablet_bootstrap.cc:654] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.403079 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 063999d856bf4ac28bb7ec01509d04a4 (DEFAULT_TABLE table=default.Table_5dec8aa2c23146558b4c71b9ab0a48b5 [id=1b4a3ef0246a4341869bf5163487cac6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.403291 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 063999d856bf4ac28bb7ec01509d04a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.403697 14812 tablet_bootstrap.cc:492] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.403761 14812 ts_tablet_manager.cc:1397] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:11.404062 14819 ts_tablet_manager.cc:1905] T fa0228294e0c42f3bd637838cf58c692 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.404139 14812 raft_consensus.cc:357] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.404218 14812 raft_consensus.cc:383] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.404247 14812 raft_consensus.cc:738] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.404356 14812 consensus_queue.cc:260] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.404668 14812 ts_tablet_manager.cc:1428] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.404829 14812 tablet_bootstrap.cc:492] T 94c861b4ff2244deb37a850340f71a3d P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.405701 14819 ts_tablet_manager.cc:1918] T fa0228294e0c42f3bd637838cf58c692 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.405774 14819 log.cc:1198] T fa0228294e0c42f3bd637838cf58c692 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/fa0228294e0c42f3bd637838cf58c692
I20250115 03:06:11.405951 14819 ts_tablet_manager.cc:1939] T fa0228294e0c42f3bd637838cf58c692 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:11.406014 14812 tablet_bootstrap.cc:654] T 94c861b4ff2244deb37a850340f71a3d P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.406605 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 180df492348a4cc09afa03c8a62a79c1 (table default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]). Expected version 1 got 0
I20250115 03:06:11.406859 14809 tablet_bootstrap.cc:492] T 0147f9743e924c9fb82a5bcd872e79fd P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
W20250115 03:06:11.406877 14169 catalog_manager.cc:5718] Failed to send AlterTable RPC for tablet 180df492348a4cc09afa03c8a62a79c1 (table default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 180df492348a4cc09afa03c8a62a79c1 consensus state has no leader
I20250115 03:06:11.406924 14809 ts_tablet_manager.cc:1397] T 0147f9743e924c9fb82a5bcd872e79fd P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:11.407227 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 063999d856bf4ac28bb7ec01509d04a4 (DEFAULT_TABLE table=default.Table_5dec8aa2c23146558b4c71b9ab0a48b5 [id=1b4a3ef0246a4341869bf5163487cac6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.407335 14809 raft_consensus.cc:357] T 0147f9743e924c9fb82a5bcd872e79fd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
W20250115 03:06:11.407371 14741 tablet.cc:2367] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250115 03:06:11.407418 14809 raft_consensus.cc:383] T 0147f9743e924c9fb82a5bcd872e79fd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.407457 14809 raft_consensus.cc:738] T 0147f9743e924c9fb82a5bcd872e79fd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.407553 14809 consensus_queue.cc:260] T 0147f9743e924c9fb82a5bcd872e79fd P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:11.407790 14809 ts_tablet_manager.cc:1428] T 0147f9743e924c9fb82a5bcd872e79fd P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.407420 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 063999d856bf4ac28bb7ec01509d04a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.410815 14809 tablet_bootstrap.cc:492] T 063999d856bf4ac28bb7ec01509d04a4 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
W20250115 03:06:11.411324 14162 catalog_manager.cc:5718] Failed to send AlterTable RPC for tablet 0147f9743e924c9fb82a5bcd872e79fd (table default.Table_9928f846023c45a0bc9c92d5447958cb [id=39abf3d98dba4d63979ccc4c27af3a97], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 0147f9743e924c9fb82a5bcd872e79fd consensus state has no leader
I20250115 03:06:11.411468 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 063999d856bf4ac28bb7ec01509d04a4 (table default.Table_5dec8aa2c23146558b4c71b9ab0a48b5 [id=1b4a3ef0246a4341869bf5163487cac6]). Expected version 1 got 0
I20250115 03:06:11.411582 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 0147f9743e924c9fb82a5bcd872e79fd (table default.Table_9928f846023c45a0bc9c92d5447958cb [id=39abf3d98dba4d63979ccc4c27af3a97]). Expected version 1 got 0
I20250115 03:06:11.411641 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 180df492348a4cc09afa03c8a62a79c1 (table default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]). Expected version 1 got 0
I20250115 03:06:11.411698 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 44585e70c3434fa1915d31c46b1c0d50 (table default.Table_d46d2affc6e74c7a8d3cc2a8ec0233e6 [id=d9fda51f2bed4c1ca720fe30ec97f440]). Expected version 1 got 0
I20250115 03:06:11.411746 14823 tablet_replica.cc:331] T fa0228294e0c42f3bd637838cf58c692 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:11.411780 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet cc88653cdfec4120b37097b361f441d4 (table default.Table_ff7c7c1250d74aec9ace68037a6d6a86 [id=96a7774dedae4e7fb54cb70b3e8a85fd]). Expected version 1 got 0
I20250115 03:06:11.411850 14809 tablet_bootstrap.cc:654] T 063999d856bf4ac28bb7ec01509d04a4 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.412003 14823 raft_consensus.cc:2238] T fa0228294e0c42f3bd637838cf58c692 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.412225 14823 raft_consensus.cc:2267] T fa0228294e0c42f3bd637838cf58c692 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:11.412493 14166 catalog_manager.cc:5718] Failed to send AlterTable RPC for tablet 063999d856bf4ac28bb7ec01509d04a4 (table default.Table_5dec8aa2c23146558b4c71b9ab0a48b5 [id=1b4a3ef0246a4341869bf5163487cac6], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 063999d856bf4ac28bb7ec01509d04a4 consensus state has no leader
I20250115 03:06:11.413007 14812 tablet_bootstrap.cc:492] T 94c861b4ff2244deb37a850340f71a3d P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.413082 14812 ts_tablet_manager.cc:1397] T 94c861b4ff2244deb37a850340f71a3d P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20250115 03:06:11.413210 14823 ts_tablet_manager.cc:1905] T fa0228294e0c42f3bd637838cf58c692 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.413308 14811 tablet_bootstrap.cc:492] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.413372 14811 ts_tablet_manager.cc:1397] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.001s
I20250115 03:06:11.413484 14812 raft_consensus.cc:357] T 94c861b4ff2244deb37a850340f71a3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.413576 14812 raft_consensus.cc:383] T 94c861b4ff2244deb37a850340f71a3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.413606 14812 raft_consensus.cc:738] T 94c861b4ff2244deb37a850340f71a3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.413712 14812 consensus_queue.cc:260] T 94c861b4ff2244deb37a850340f71a3d P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.413834 14811 raft_consensus.cc:357] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.413924 14811 raft_consensus.cc:383] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.413937 14812 ts_tablet_manager.cc:1428] T 94c861b4ff2244deb37a850340f71a3d P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.413954 14811 raft_consensus.cc:738] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.414094 14812 tablet_bootstrap.cc:492] T 5006eb8055b04bf79773830de5974ef6 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.414165 14811 consensus_queue.cc:260] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.414266 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 44585e70c3434fa1915d31c46b1c0d50 (table default.Table_d46d2affc6e74c7a8d3cc2a8ec0233e6 [id=d9fda51f2bed4c1ca720fe30ec97f440]). Expected version 1 got 0
I20250115 03:06:11.414343 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet cc88653cdfec4120b37097b361f441d4 (table default.Table_ff7c7c1250d74aec9ace68037a6d6a86 [id=96a7774dedae4e7fb54cb70b3e8a85fd]). Expected version 1 got 0
I20250115 03:06:11.414438 14811 ts_tablet_manager.cc:1428] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:11.414960 14812 tablet_bootstrap.cc:654] T 5006eb8055b04bf79773830de5974ef6 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.415102 14823 ts_tablet_manager.cc:1918] T fa0228294e0c42f3bd637838cf58c692 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.415174 14823 log.cc:1198] T fa0228294e0c42f3bd637838cf58c692 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/fa0228294e0c42f3bd637838cf58c692
I20250115 03:06:11.415197 14811 tablet_bootstrap.cc:492] T 5006eb8055b04bf79773830de5974ef6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.415385 14823 ts_tablet_manager.cc:1939] T fa0228294e0c42f3bd637838cf58c692 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:11.416121 14811 tablet_bootstrap.cc:654] T 5006eb8055b04bf79773830de5974ef6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.417340 14812 tablet_bootstrap.cc:492] T 5006eb8055b04bf79773830de5974ef6 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.417418 14812 ts_tablet_manager.cc:1397] T 5006eb8055b04bf79773830de5974ef6 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:11.417807 14812 raft_consensus.cc:357] T 5006eb8055b04bf79773830de5974ef6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.417901 14812 raft_consensus.cc:383] T 5006eb8055b04bf79773830de5974ef6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.417929 14812 raft_consensus.cc:738] T 5006eb8055b04bf79773830de5974ef6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.418037 14812 consensus_queue.cc:260] T 5006eb8055b04bf79773830de5974ef6 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.418344 14812 ts_tablet_manager.cc:1428] T 5006eb8055b04bf79773830de5974ef6 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.418495 14812 tablet_bootstrap.cc:492] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.419340 14812 tablet_bootstrap.cc:654] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.420244 14812 tablet_bootstrap.cc:492] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.420365 14812 ts_tablet_manager.cc:1397] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:11.420431 14823 tablet_replica.cc:331] T 94c861b4ff2244deb37a850340f71a3d P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:11.420534 14823 raft_consensus.cc:2238] T 94c861b4ff2244deb37a850340f71a3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.420619 14823 raft_consensus.cc:2267] T 94c861b4ff2244deb37a850340f71a3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.407045 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet fa0228294e0c42f3bd637838cf58c692 (table default.Table_9928f846023c45a0bc9c92d5447958cb [id=39abf3d98dba4d63979ccc4c27af3a97]) successfully deleted
I20250115 03:06:11.421030 14812 raft_consensus.cc:357] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.421177 14812 raft_consensus.cc:383] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.421170 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet fa0228294e0c42f3bd637838cf58c692 (table default.Table_9928f846023c45a0bc9c92d5447958cb [id=39abf3d98dba4d63979ccc4c27af3a97]) successfully deleted
I20250115 03:06:11.421238 14811 tablet_bootstrap.cc:492] T 5006eb8055b04bf79773830de5974ef6 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.421255 14812 raft_consensus.cc:738] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.421322 14811 ts_tablet_manager.cc:1397] T 5006eb8055b04bf79773830de5974ef6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20250115 03:06:11.421484 14812 consensus_queue.cc:260] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.421710 14811 raft_consensus.cc:357] T 5006eb8055b04bf79773830de5974ef6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.421808 14811 raft_consensus.cc:383] T 5006eb8055b04bf79773830de5974ef6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.421844 14811 raft_consensus.cc:738] T 5006eb8055b04bf79773830de5974ef6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.421769 14812 ts_tablet_manager.cc:1428] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.421970 14811 consensus_queue.cc:260] T 5006eb8055b04bf79773830de5974ef6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.422151 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 5006eb8055b04bf79773830de5974ef6 with delete_type TABLET_DATA_DELETED (replaced by tablet 180df492348a4cc09afa03c8a62a79c1) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:11.422191 14811 ts_tablet_manager.cc:1428] T 5006eb8055b04bf79773830de5974ef6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:11.422338 14811 tablet_bootstrap.cc:492] T 94c861b4ff2244deb37a850340f71a3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.422430 14812 tablet_bootstrap.cc:492] T 0147f9743e924c9fb82a5bcd872e79fd P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.423210 14811 tablet_bootstrap.cc:654] T 94c861b4ff2244deb37a850340f71a3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.423498 14812 tablet_bootstrap.cc:654] T 0147f9743e924c9fb82a5bcd872e79fd P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.424305 14823 ts_tablet_manager.cc:1905] T 94c861b4ff2244deb37a850340f71a3d P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.425863 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 180df492348a4cc09afa03c8a62a79c1 (table default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]). Expected version 1 got 0
I20250115 03:06:11.426039 14823 ts_tablet_manager.cc:1918] T 94c861b4ff2244deb37a850340f71a3d P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.426110 14823 log.cc:1198] T 94c861b4ff2244deb37a850340f71a3d P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/94c861b4ff2244deb37a850340f71a3d
I20250115 03:06:11.426414 14823 ts_tablet_manager.cc:1939] T 94c861b4ff2244deb37a850340f71a3d P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:11.426549 14811 tablet_bootstrap.cc:492] T 94c861b4ff2244deb37a850340f71a3d P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.426635 14811 ts_tablet_manager.cc:1397] T 94c861b4ff2244deb37a850340f71a3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20250115 03:06:11.427011 14811 raft_consensus.cc:357] T 94c861b4ff2244deb37a850340f71a3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.427104 14811 raft_consensus.cc:383] T 94c861b4ff2244deb37a850340f71a3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.427135 14811 raft_consensus.cc:738] T 94c861b4ff2244deb37a850340f71a3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.427237 14811 consensus_queue.cc:260] T 94c861b4ff2244deb37a850340f71a3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.427491 14811 ts_tablet_manager.cc:1428] T 94c861b4ff2244deb37a850340f71a3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.427654 14811 tablet_bootstrap.cc:492] T 063999d856bf4ac28bb7ec01509d04a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.427841 14823 tablet_replica.cc:331] T 5006eb8055b04bf79773830de5974ef6 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:11.427971 14823 raft_consensus.cc:2238] T 5006eb8055b04bf79773830de5974ef6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.428087 14823 raft_consensus.cc:2267] T 5006eb8055b04bf79773830de5974ef6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.428479 14811 tablet_bootstrap.cc:654] T 063999d856bf4ac28bb7ec01509d04a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.428476 14823 ts_tablet_manager.cc:1905] T 5006eb8055b04bf79773830de5974ef6 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.429684 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 94c861b4ff2244deb37a850340f71a3d (table default.Table_5dec8aa2c23146558b4c71b9ab0a48b5 [id=1b4a3ef0246a4341869bf5163487cac6]) successfully deleted
I20250115 03:06:11.429730 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 0147f9743e924c9fb82a5bcd872e79fd (table default.Table_9928f846023c45a0bc9c92d5447958cb [id=39abf3d98dba4d63979ccc4c27af3a97]). Expected version 1 got 0
I20250115 03:06:11.429864 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet 94c861b4ff2244deb37a850340f71a3d with delete_type TABLET_DATA_DELETED (replaced by tablet 063999d856bf4ac28bb7ec01509d04a4) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.430037 14820 tablet_replica.cc:331] T 94c861b4ff2244deb37a850340f71a3d P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:11.430301 14820 raft_consensus.cc:2238] T 94c861b4ff2244deb37a850340f71a3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.430518 14820 raft_consensus.cc:2267] T 94c861b4ff2244deb37a850340f71a3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.430799 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 180df492348a4cc09afa03c8a62a79c1 (table default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]). Expected version 1 got 0
I20250115 03:06:11.431478 14820 ts_tablet_manager.cc:1905] T 94c861b4ff2244deb37a850340f71a3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.431769 14809 tablet_bootstrap.cc:492] T 063999d856bf4ac28bb7ec01509d04a4 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.431844 14809 ts_tablet_manager.cc:1397] T 063999d856bf4ac28bb7ec01509d04a4 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20250115 03:06:11.432183 14809 raft_consensus.cc:357] T 063999d856bf4ac28bb7ec01509d04a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.432264 14809 raft_consensus.cc:383] T 063999d856bf4ac28bb7ec01509d04a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.432293 14809 raft_consensus.cc:738] T 063999d856bf4ac28bb7ec01509d04a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.432399 14809 consensus_queue.cc:260] T 063999d856bf4ac28bb7ec01509d04a4 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.432612 14809 ts_tablet_manager.cc:1428] T 063999d856bf4ac28bb7ec01509d04a4 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.433563 14820 ts_tablet_manager.cc:1918] T 94c861b4ff2244deb37a850340f71a3d P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.433938 14166 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 063999d856bf4ac28bb7ec01509d04a4 (table default.Table_5dec8aa2c23146558b4c71b9ab0a48b5 [id=1b4a3ef0246a4341869bf5163487cac6]). Expected version 1 got 0
I20250115 03:06:11.434253 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 5006eb8055b04bf79773830de5974ef6 with delete_type TABLET_DATA_DELETED (replaced by tablet 180df492348a4cc09afa03c8a62a79c1) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.434334 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet fa0228294e0c42f3bd637838cf58c692 with delete_type TABLET_DATA_DELETED (replaced by tablet 0147f9743e924c9fb82a5bcd872e79fd) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.435779 14820 log.cc:1198] T 94c861b4ff2244deb37a850340f71a3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/94c861b4ff2244deb37a850340f71a3d
I20250115 03:06:11.435998 14166 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 063999d856bf4ac28bb7ec01509d04a4 (table default.Table_5dec8aa2c23146558b4c71b9ab0a48b5 [id=1b4a3ef0246a4341869bf5163487cac6]). Expected version 1 got 0
I20250115 03:06:11.436131 14820 ts_tablet_manager.cc:1939] T 94c861b4ff2244deb37a850340f71a3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:11.436249 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet cc88653cdfec4120b37097b361f441d4 (table default.Table_ff7c7c1250d74aec9ace68037a6d6a86 [id=96a7774dedae4e7fb54cb70b3e8a85fd]). Expected version 1 got 0
I20250115 03:06:11.437301 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 94c861b4ff2244deb37a850340f71a3d (table default.Table_5dec8aa2c23146558b4c71b9ab0a48b5 [id=1b4a3ef0246a4341869bf5163487cac6]) successfully deleted
I20250115 03:06:11.437417 14820 tablet_replica.cc:331] T 5006eb8055b04bf79773830de5974ef6 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:11.437516 14820 raft_consensus.cc:2238] T 5006eb8055b04bf79773830de5974ef6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.437602 14820 raft_consensus.cc:2267] T 5006eb8055b04bf79773830de5974ef6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.437626 14823 ts_tablet_manager.cc:1918] T 5006eb8055b04bf79773830de5974ef6 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.437690 14823 log.cc:1198] T 5006eb8055b04bf79773830de5974ef6 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/5006eb8055b04bf79773830de5974ef6
I20250115 03:06:11.437863 14823 ts_tablet_manager.cc:1939] T 5006eb8055b04bf79773830de5974ef6 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:11.438030 14820 ts_tablet_manager.cc:1905] T 5006eb8055b04bf79773830de5974ef6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.438308 14812 tablet_bootstrap.cc:492] T 0147f9743e924c9fb82a5bcd872e79fd P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.438421 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 5006eb8055b04bf79773830de5974ef6 (table default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]) successfully deleted
I20250115 03:06:11.438484 14812 ts_tablet_manager.cc:1397] T 0147f9743e924c9fb82a5bcd872e79fd P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20250115 03:06:11.438835 14812 raft_consensus.cc:357] T 0147f9743e924c9fb82a5bcd872e79fd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:11.438915 14812 raft_consensus.cc:383] T 0147f9743e924c9fb82a5bcd872e79fd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.438943 14812 raft_consensus.cc:738] T 0147f9743e924c9fb82a5bcd872e79fd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.439061 14812 consensus_queue.cc:260] T 0147f9743e924c9fb82a5bcd872e79fd P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:11.439302 14812 ts_tablet_manager.cc:1428] T 0147f9743e924c9fb82a5bcd872e79fd P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.439450 14812 tablet_bootstrap.cc:492] T 063999d856bf4ac28bb7ec01509d04a4 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.440245 14812 tablet_bootstrap.cc:654] T 063999d856bf4ac28bb7ec01509d04a4 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.440734 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 0147f9743e924c9fb82a5bcd872e79fd (table default.Table_9928f846023c45a0bc9c92d5447958cb [id=39abf3d98dba4d63979ccc4c27af3a97]). Expected version 1 got 0
I20250115 03:06:11.441392 14811 tablet_bootstrap.cc:492] T 063999d856bf4ac28bb7ec01509d04a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.441476 14811 ts_tablet_manager.cc:1397] T 063999d856bf4ac28bb7ec01509d04a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20250115 03:06:11.441844 14811 raft_consensus.cc:357] T 063999d856bf4ac28bb7ec01509d04a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.441931 14811 raft_consensus.cc:383] T 063999d856bf4ac28bb7ec01509d04a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.441952 14811 raft_consensus.cc:738] T 063999d856bf4ac28bb7ec01509d04a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.442041 14811 consensus_queue.cc:260] T 063999d856bf4ac28bb7ec01509d04a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.442270 14811 ts_tablet_manager.cc:1428] T 063999d856bf4ac28bb7ec01509d04a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.442416 14811 tablet_bootstrap.cc:492] T 0147f9743e924c9fb82a5bcd872e79fd P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.443356 14811 tablet_bootstrap.cc:654] T 0147f9743e924c9fb82a5bcd872e79fd P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.443851 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 063999d856bf4ac28bb7ec01509d04a4 (table default.Table_5dec8aa2c23146558b4c71b9ab0a48b5 [id=1b4a3ef0246a4341869bf5163487cac6]). Expected version 1 got 0
I20250115 03:06:11.444195 14811 tablet_bootstrap.cc:492] T 0147f9743e924c9fb82a5bcd872e79fd P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.444269 14811 ts_tablet_manager.cc:1397] T 0147f9743e924c9fb82a5bcd872e79fd P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:11.444657 14811 raft_consensus.cc:357] T 0147f9743e924c9fb82a5bcd872e79fd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:11.444761 14811 raft_consensus.cc:383] T 0147f9743e924c9fb82a5bcd872e79fd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.444800 14811 raft_consensus.cc:738] T 0147f9743e924c9fb82a5bcd872e79fd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.444953 14811 consensus_queue.cc:260] T 0147f9743e924c9fb82a5bcd872e79fd P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:11.445196 14811 ts_tablet_manager.cc:1428] T 0147f9743e924c9fb82a5bcd872e79fd P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.445348 14811 tablet_bootstrap.cc:492] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.446275 14811 tablet_bootstrap.cc:654] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.446990 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 0147f9743e924c9fb82a5bcd872e79fd (table default.Table_9928f846023c45a0bc9c92d5447958cb [id=39abf3d98dba4d63979ccc4c27af3a97]). Expected version 1 got 0
I20250115 03:06:11.447255 14811 tablet_bootstrap.cc:492] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.447322 14811 ts_tablet_manager.cc:1397] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:11.447633 14820 ts_tablet_manager.cc:1918] T 5006eb8055b04bf79773830de5974ef6 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.447722 14820 log.cc:1198] T 5006eb8055b04bf79773830de5974ef6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/5006eb8055b04bf79773830de5974ef6
I20250115 03:06:11.447682 14811 raft_consensus.cc:357] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.447831 14811 raft_consensus.cc:383] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.447876 14811 raft_consensus.cc:738] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.447970 14820 ts_tablet_manager.cc:1939] T 5006eb8055b04bf79773830de5974ef6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:11.447985 14811 consensus_queue.cc:260] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.448212 14811 ts_tablet_manager.cc:1428] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.449416 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 180df492348a4cc09afa03c8a62a79c1 (table default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]). Expected version 1 got 0
I20250115 03:06:11.450023 14820 tablet_replica.cc:331] T fa0228294e0c42f3bd637838cf58c692 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:11.450122 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 5006eb8055b04bf79773830de5974ef6 (table default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]) successfully deleted
I20250115 03:06:11.450196 14820 raft_consensus.cc:2238] T fa0228294e0c42f3bd637838cf58c692 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.450280 14820 raft_consensus.cc:2267] T fa0228294e0c42f3bd637838cf58c692 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.450614 14820 ts_tablet_manager.cc:1905] T fa0228294e0c42f3bd637838cf58c692 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.450872 14812 tablet_bootstrap.cc:492] T 063999d856bf4ac28bb7ec01509d04a4 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.450953 14812 ts_tablet_manager.cc:1397] T 063999d856bf4ac28bb7ec01509d04a4 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20250115 03:06:11.451355 14812 raft_consensus.cc:357] T 063999d856bf4ac28bb7ec01509d04a4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.451459 14812 raft_consensus.cc:383] T 063999d856bf4ac28bb7ec01509d04a4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.451501 14812 raft_consensus.cc:738] T 063999d856bf4ac28bb7ec01509d04a4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.451643 14812 consensus_queue.cc:260] T 063999d856bf4ac28bb7ec01509d04a4 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.451889 14812 ts_tablet_manager.cc:1428] T 063999d856bf4ac28bb7ec01509d04a4 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:11.452522 14820 ts_tablet_manager.cc:1918] T fa0228294e0c42f3bd637838cf58c692 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.452610 14820 log.cc:1198] T fa0228294e0c42f3bd637838cf58c692 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/fa0228294e0c42f3bd637838cf58c692
I20250115 03:06:11.452811 14820 ts_tablet_manager.cc:1939] T fa0228294e0c42f3bd637838cf58c692 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:11.453557 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet fa0228294e0c42f3bd637838cf58c692 (table default.Table_9928f846023c45a0bc9c92d5447958cb [id=39abf3d98dba4d63979ccc4c27af3a97]) successfully deleted
I20250115 03:06:11.453924 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 063999d856bf4ac28bb7ec01509d04a4 (table default.Table_5dec8aa2c23146558b4c71b9ab0a48b5 [id=1b4a3ef0246a4341869bf5163487cac6]). Expected version 1 got 0
W20250115 03:06:11.494355 14606 tablet.cc:2367] T 0147f9743e924c9fb82a5bcd872e79fd P 84b1408700ef479cbb5a7b4d2f0bebf1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250115 03:06:11.508369 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_9a7ec6b5bb5b4a9d962fc4748c2a6333"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:11.508841 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9a7ec6b5bb5b4a9d962fc4748c2a6333 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.509550 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_4d9c9103a56043a998dc4228d30f4074"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:11.509860 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d9c9103a56043a998dc4228d30f4074 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.511068 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2c468a7cc67a4a8ab2d27b869bddfd49"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:11.511385 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2c468a7cc67a4a8ab2d27b869bddfd49 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.512748 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d2b6a3fb0daf4d4dba0ef0007a0996c9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.512800 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_3ee99e9b06794a7d9521db355534da2f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.512810 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ca3508a1c2694a108d5eb34470d19ade"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:11.513060 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d2b6a3fb0daf4d4dba0ef0007a0996c9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:11.513123 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3ee99e9b06794a7d9521db355534da2f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:11.513150 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ca3508a1c2694a108d5eb34470d19ade in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.512888 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_27621a9d3c294854ac7115d94f8952f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:11.513444 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_27621a9d3c294854ac7115d94f8952f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.515894 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_1132d16f9e04448689aab85d1952013b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:11.516171 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1132d16f9e04448689aab85d1952013b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.518185 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_dee96a49a9754f88b4298601381f34c5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.518477 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_03a772b18f874e3fb42b5dc7327e4543"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:11.518522 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dee96a49a9754f88b4298601381f34c5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:11.518770 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_03a772b18f874e3fb42b5dc7327e4543 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.520634 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4d9c9103a56043a998dc4228d30f4074" } new_table_name: "default.Table_c1020680eef344a49998eddac921fea5" modify_external_catalogs: true
I20250115 03:06:11.521366 14171 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ff7c7c1250d74aec9ace68037a6d6a86" } new_table_name: "default.Table_74d1acd4f65e41bdb046d79c090a111b" modify_external_catalogs: true
I20250115 03:06:11.522882 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5dec8aa2c23146558b4c71b9ab0a48b5" } new_table_name: "default.Table_db8eb767c1214ff89ccfb64f0ad123ae" modify_external_catalogs: true
I20250115 03:06:11.523545 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d2b6a3fb0daf4d4dba0ef0007a0996c9" } new_table_name: "default.Table_b12aff7a3c8d41f79392773a151c463a" modify_external_catalogs: true
I20250115 03:06:11.524402 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2c468a7cc67a4a8ab2d27b869bddfd49" } modify_external_catalogs: true
I20250115 03:06:11.524492 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2c468a7cc67a4a8ab2d27b869bddfd49" } modify_external_catalogs: true
I20250115 03:06:11.524634 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3ee99e9b06794a7d9521db355534da2f" } new_table_name: "default.Table_4009ecd0a7ed495c99e5e4fd76c4774f" modify_external_catalogs: true
I20250115 03:06:11.525238 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_9a7ec6b5bb5b4a9d962fc4748c2a6333" } modify_external_catalogs: true
I20250115 03:06:11.525329 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_9a7ec6b5bb5b4a9d962fc4748c2a6333" } modify_external_catalogs: true
I20250115 03:06:11.528256 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ca3508a1c2694a108d5eb34470d19ade" } modify_external_catalogs: true
I20250115 03:06:11.528352 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ca3508a1c2694a108d5eb34470d19ade" } modify_external_catalogs: true
I20250115 03:06:11.529541 14535 tablet_service.cc:1467] Processing CreateTablet for tablet b061fea3b190420682a91662d0effd20 (DEFAULT_TABLE table=default.Table_9a7ec6b5bb5b4a9d962fc4748c2a6333 [id=5b89c9008ffd4a7d8149c8420dfc3d0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.529613 14402 tablet_service.cc:1467] Processing CreateTablet for tablet b061fea3b190420682a91662d0effd20 (DEFAULT_TABLE table=default.Table_9a7ec6b5bb5b4a9d962fc4748c2a6333 [id=5b89c9008ffd4a7d8149c8420dfc3d0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.529816 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b061fea3b190420682a91662d0effd20. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.529899 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b061fea3b190420682a91662d0effd20. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.531670 14809 tablet_bootstrap.cc:492] T b061fea3b190420682a91662d0effd20 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.532044 14811 tablet_bootstrap.cc:492] T b061fea3b190420682a91662d0effd20 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.532635 14809 tablet_bootstrap.cc:654] T b061fea3b190420682a91662d0effd20 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.532577 14672 tablet_service.cc:1467] Processing CreateTablet for tablet b061fea3b190420682a91662d0effd20 (DEFAULT_TABLE table=default.Table_9a7ec6b5bb5b4a9d962fc4748c2a6333 [id=5b89c9008ffd4a7d8149c8420dfc3d0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.532832 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b061fea3b190420682a91662d0effd20. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.533030 14811 tablet_bootstrap.cc:654] T b061fea3b190420682a91662d0effd20 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.533618 14809 tablet_bootstrap.cc:492] T b061fea3b190420682a91662d0effd20 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.533689 14809 ts_tablet_manager.cc:1397] T b061fea3b190420682a91662d0effd20 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20250115 03:06:11.534088 14809 raft_consensus.cc:357] T b061fea3b190420682a91662d0effd20 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:11.534179 14809 raft_consensus.cc:383] T b061fea3b190420682a91662d0effd20 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.534209 14809 raft_consensus.cc:738] T b061fea3b190420682a91662d0effd20 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.534328 14809 consensus_queue.cc:260] T b061fea3b190420682a91662d0effd20 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:11.534603 14809 ts_tablet_manager.cc:1428] T b061fea3b190420682a91662d0effd20 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.537061 14812 tablet_bootstrap.cc:492] T b061fea3b190420682a91662d0effd20 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.537978 14812 tablet_bootstrap.cc:654] T b061fea3b190420682a91662d0effd20 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.538951 14811 tablet_bootstrap.cc:492] T b061fea3b190420682a91662d0effd20 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.539032 14811 ts_tablet_manager.cc:1397] T b061fea3b190420682a91662d0effd20 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20250115 03:06:11.539613 14811 raft_consensus.cc:357] T b061fea3b190420682a91662d0effd20 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:11.539722 14811 raft_consensus.cc:383] T b061fea3b190420682a91662d0effd20 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.539757 14811 raft_consensus.cc:738] T b061fea3b190420682a91662d0effd20 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.539852 14812 tablet_bootstrap.cc:492] T b061fea3b190420682a91662d0effd20 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.539918 14812 ts_tablet_manager.cc:1397] T b061fea3b190420682a91662d0effd20 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20250115 03:06:11.539876 14811 consensus_queue.cc:260] T b061fea3b190420682a91662d0effd20 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:11.540318 14812 raft_consensus.cc:357] T b061fea3b190420682a91662d0effd20 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:11.540408 14812 raft_consensus.cc:383] T b061fea3b190420682a91662d0effd20 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.540438 14812 raft_consensus.cc:738] T b061fea3b190420682a91662d0effd20 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.540544 14812 consensus_queue.cc:260] T b061fea3b190420682a91662d0effd20 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:11.540810 14812 ts_tablet_manager.cc:1428] T b061fea3b190420682a91662d0effd20 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:11.540979 14811 ts_tablet_manager.cc:1428] T b061fea3b190420682a91662d0effd20 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
W20250115 03:06:11.545028 14163 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 063999d856bf4ac28bb7ec01509d04a4 consensus state has no leader
W20250115 03:06:11.545522 14167 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4f32d3e059e44722af83c55b71f14c26 has no consensus state
W20250115 03:06:11.546042 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fd5703df37db41daafc6697793355f31 has no consensus state
W20250115 03:06:11.546257 14171 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cc88653cdfec4120b37097b361f441d4 consensus state has no leader
I20250115 03:06:11.546468 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_9928f846023c45a0bc9c92d5447958cb" } modify_external_catalogs: true
I20250115 03:06:11.546497 14817 raft_consensus.cc:491] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.546552 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_9928f846023c45a0bc9c92d5447958cb" } modify_external_catalogs: true
I20250115 03:06:11.546627 14817 raft_consensus.cc:513] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.547811 14817 leader_election.cc:290] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.547835 14166 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_27621a9d3c294854ac7115d94f8952f1" } modify_external_catalogs: true
I20250115 03:06:11.547919 14166 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_27621a9d3c294854ac7115d94f8952f1" } modify_external_catalogs: true
I20250115 03:06:11.551467 14164 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet edbbebc96c5345f6b14780c31c43a531
W20250115 03:06:11.551741 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4379cf2eb12844a29c5f2e36d1e1dffc has no consensus state
I20250115 03:06:11.551942 14165 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet 686b079569ff4e0f84cc014899cf9a75
I20250115 03:06:11.553165 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "180df492348a4cc09afa03c8a62a79c1" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:11.553391 14422 raft_consensus.cc:2463] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:11.553874 14493 leader_election.cc:304] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.554620 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet b061fea3b190420682a91662d0effd20
I20250115 03:06:11.555018 14166 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet 7a5ef6fe0433481cbe77d58879996738
I20250115 03:06:11.555271 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet b061fea3b190420682a91662d0effd20 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.555415 14820 tablet_replica.cc:331] T b061fea3b190420682a91662d0effd20 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:11.555534 14820 raft_consensus.cc:2238] T b061fea3b190420682a91662d0effd20 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.555550 14402 tablet_service.cc:1514] Processing DeleteTablet for tablet b061fea3b190420682a91662d0effd20 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.555626 14820 raft_consensus.cc:2267] T b061fea3b190420682a91662d0effd20 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.555666 14813 raft_consensus.cc:491] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.555761 14813 raft_consensus.cc:513] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.556762 14813 leader_election.cc:290] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:11.557135 14471 tablet.cc:2367] T 0147f9743e924c9fb82a5bcd872e79fd P e8df7247ee9c43e0aa18216da2661f75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250115 03:06:11.555681 14819 tablet_replica.cc:331] T b061fea3b190420682a91662d0effd20 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:11.558331 14819 raft_consensus.cc:2238] T b061fea3b190420682a91662d0effd20 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.558444 14819 raft_consensus.cc:2267] T b061fea3b190420682a91662d0effd20 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.558473 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet b061fea3b190420682a91662d0effd20 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:11.558627 14823 tablet_replica.cc:331] T b061fea3b190420682a91662d0effd20 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:11.558733 14823 raft_consensus.cc:2238] T b061fea3b190420682a91662d0effd20 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.558822 14823 raft_consensus.cc:2267] T b061fea3b190420682a91662d0effd20 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.558970 14819 ts_tablet_manager.cc:1905] T b061fea3b190420682a91662d0effd20 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.559268 14823 ts_tablet_manager.cc:1905] T b061fea3b190420682a91662d0effd20 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.560993 14819 ts_tablet_manager.cc:1918] T b061fea3b190420682a91662d0effd20 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.560993 14823 ts_tablet_manager.cc:1918] T b061fea3b190420682a91662d0effd20 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.561064 14819 log.cc:1198] T b061fea3b190420682a91662d0effd20 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/b061fea3b190420682a91662d0effd20
I20250115 03:06:11.561064 14823 log.cc:1198] T b061fea3b190420682a91662d0effd20 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/b061fea3b190420682a91662d0effd20
I20250115 03:06:11.561254 14819 ts_tablet_manager.cc:1939] T b061fea3b190420682a91662d0effd20 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:11.561254 14823 ts_tablet_manager.cc:1939] T b061fea3b190420682a91662d0effd20 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:11.561812 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet b061fea3b190420682a91662d0effd20 (table default.Table_9a7ec6b5bb5b4a9d962fc4748c2a6333 [id=5b89c9008ffd4a7d8149c8420dfc3d0e]) successfully deleted
I20250115 03:06:11.564412 14817 raft_consensus.cc:2798] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:11.564488 14817 raft_consensus.cc:491] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.564528 14817 raft_consensus.cc:3054] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.565632 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 4b21178f74d1465f9d6b64ac1b3a854b (DEFAULT_TABLE table=default.Table_1132d16f9e04448689aab85d1952013b [id=9f650455cb3440a49a11010f541084db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.565866 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b21178f74d1465f9d6b64ac1b3a854b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.565900 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 4b21178f74d1465f9d6b64ac1b3a854b (DEFAULT_TABLE table=default.Table_1132d16f9e04448689aab85d1952013b [id=9f650455cb3440a49a11010f541084db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.566085 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b21178f74d1465f9d6b64ac1b3a854b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.566547 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 4b21178f74d1465f9d6b64ac1b3a854b (DEFAULT_TABLE table=default.Table_1132d16f9e04448689aab85d1952013b [id=9f650455cb3440a49a11010f541084db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.566761 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b21178f74d1465f9d6b64ac1b3a854b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.566726 14534 tablet_service.cc:1467] Processing CreateTablet for tablet fd5703df37db41daafc6697793355f31 (DEFAULT_TABLE table=default.Table_b12aff7a3c8d41f79392773a151c463a [id=2c420d8e08fd407386e8566fa323a55f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.566900 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd5703df37db41daafc6697793355f31. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.567721 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 4379cf2eb12844a29c5f2e36d1e1dffc (DEFAULT_TABLE table=default.Table_4009ecd0a7ed495c99e5e4fd76c4774f [id=05f9fd2ebec34d2299ea59c47a74c828]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.567770 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 4379cf2eb12844a29c5f2e36d1e1dffc (DEFAULT_TABLE table=default.Table_4009ecd0a7ed495c99e5e4fd76c4774f [id=05f9fd2ebec34d2299ea59c47a74c828]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.567919 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4379cf2eb12844a29c5f2e36d1e1dffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.567932 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4379cf2eb12844a29c5f2e36d1e1dffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.569586 14809 tablet_bootstrap.cc:492] T 4b21178f74d1465f9d6b64ac1b3a854b P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.569698 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 4f32d3e059e44722af83c55b71f14c26 (DEFAULT_TABLE table=default.Table_c1020680eef344a49998eddac921fea5 [id=649880f1ff0d4cfda2358b408110d8dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.569875 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f32d3e059e44722af83c55b71f14c26. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.570545 14809 tablet_bootstrap.cc:654] T 4b21178f74d1465f9d6b64ac1b3a854b P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.570860 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 4f32d3e059e44722af83c55b71f14c26 (DEFAULT_TABLE table=default.Table_c1020680eef344a49998eddac921fea5 [id=649880f1ff0d4cfda2358b408110d8dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.571035 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f32d3e059e44722af83c55b71f14c26. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.571841 14811 tablet_bootstrap.cc:492] T 4b21178f74d1465f9d6b64ac1b3a854b P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.572083 14851 raft_consensus.cc:491] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.572095 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "180df492348a4cc09afa03c8a62a79c1" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:11.572158 14851 raft_consensus.cc:513] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.572316 14692 raft_consensus.cc:2463] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:11.572724 14811 tablet_bootstrap.cc:654] T 4b21178f74d1465f9d6b64ac1b3a854b P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.572742 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44585e70c3434fa1915d31c46b1c0d50" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:11.572715 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 4379cf2eb12844a29c5f2e36d1e1dffc (DEFAULT_TABLE table=default.Table_4009ecd0a7ed495c99e5e4fd76c4774f [id=05f9fd2ebec34d2299ea59c47a74c828]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.572837 14422 raft_consensus.cc:2463] T 44585e70c3434fa1915d31c46b1c0d50 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:11.572947 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4379cf2eb12844a29c5f2e36d1e1dffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.573357 14493 leader_election.cc:304] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.573746 14809 tablet_bootstrap.cc:492] T 4b21178f74d1465f9d6b64ac1b3a854b P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.573814 14809 ts_tablet_manager.cc:1397] T 4b21178f74d1465f9d6b64ac1b3a854b P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20250115 03:06:11.574074 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44585e70c3434fa1915d31c46b1c0d50" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:11.574169 14692 raft_consensus.cc:2463] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:11.574218 14809 raft_consensus.cc:357] T 4b21178f74d1465f9d6b64ac1b3a854b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.574294 14809 raft_consensus.cc:383] T 4b21178f74d1465f9d6b64ac1b3a854b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.574323 14809 raft_consensus.cc:738] T 4b21178f74d1465f9d6b64ac1b3a854b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.574472 14851 leader_election.cc:290] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.574447 14809 consensus_queue.cc:260] T 4b21178f74d1465f9d6b64ac1b3a854b P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.574692 14809 ts_tablet_manager.cc:1428] T 4b21178f74d1465f9d6b64ac1b3a854b P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.574832 14809 tablet_bootstrap.cc:492] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.575661 14809 tablet_bootstrap.cc:654] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.575667 14812 tablet_bootstrap.cc:492] T 4b21178f74d1465f9d6b64ac1b3a854b P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.575855 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 4f32d3e059e44722af83c55b71f14c26 (DEFAULT_TABLE table=default.Table_c1020680eef344a49998eddac921fea5 [id=649880f1ff0d4cfda2358b408110d8dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.576046 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f32d3e059e44722af83c55b71f14c26. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.576359 14397 tablet_service.cc:1467] Processing CreateTablet for tablet fd5703df37db41daafc6697793355f31 (DEFAULT_TABLE table=default.Table_b12aff7a3c8d41f79392773a151c463a [id=2c420d8e08fd407386e8566fa323a55f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.576558 14812 tablet_bootstrap.cc:654] T 4b21178f74d1465f9d6b64ac1b3a854b P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.576563 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd5703df37db41daafc6697793355f31. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.577832 14672 tablet_service.cc:1467] Processing CreateTablet for tablet fd5703df37db41daafc6697793355f31 (DEFAULT_TABLE table=default.Table_b12aff7a3c8d41f79392773a151c463a [id=2c420d8e08fd407386e8566fa323a55f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.577883 14817 raft_consensus.cc:513] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.578027 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd5703df37db41daafc6697793355f31. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.578193 14817 leader_election.cc:290] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.578297 14817 raft_consensus.cc:2798] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:11.578334 14817 raft_consensus.cc:491] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.578367 14817 raft_consensus.cc:3054] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.578622 14809 tablet_bootstrap.cc:492] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.578691 14809 ts_tablet_manager.cc:1397] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:11.579047 14817 raft_consensus.cc:513] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.579062 14809 raft_consensus.cc:357] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.579142 14809 raft_consensus.cc:383] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.579167 14809 raft_consensus.cc:738] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.579263 14809 consensus_queue.cc:260] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.579340 14817 leader_election.cc:290] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.579495 14809 ts_tablet_manager.cc:1428] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.579649 14809 tablet_bootstrap.cc:492] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.579660 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 0147f9743e924c9fb82a5bcd872e79fd
I20250115 03:06:11.580273 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet b061fea3b190420682a91662d0effd20 (table default.Table_9a7ec6b5bb5b4a9d962fc4748c2a6333 [id=5b89c9008ffd4a7d8149c8420dfc3d0e]) successfully deleted
I20250115 03:06:11.580451 14809 tablet_bootstrap.cc:654] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.580531 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "180df492348a4cc09afa03c8a62a79c1" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:11.580622 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 0147f9743e924c9fb82a5bcd872e79fd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.580787 14819 tablet_replica.cc:331] T 0147f9743e924c9fb82a5bcd872e79fd P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:11.580881 14819 raft_consensus.cc:2238] T 0147f9743e924c9fb82a5bcd872e79fd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.580991 14819 raft_consensus.cc:2267] T 0147f9743e924c9fb82a5bcd872e79fd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.581032 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44585e70c3434fa1915d31c46b1c0d50" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:11.581032 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44585e70c3434fa1915d31c46b1c0d50" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:11.580623 14422 raft_consensus.cc:3054] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.581123 14421 raft_consensus.cc:3054] T 44585e70c3434fa1915d31c46b1c0d50 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.581127 14691 raft_consensus.cc:3054] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.581959 14691 raft_consensus.cc:2463] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:11.582012 14421 raft_consensus.cc:2463] T 44585e70c3434fa1915d31c46b1c0d50 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:11.582093 14422 raft_consensus.cc:2463] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:11.582504 14819 ts_tablet_manager.cc:1905] T 0147f9743e924c9fb82a5bcd872e79fd P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.582499 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "180df492348a4cc09afa03c8a62a79c1" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:11.582595 14692 raft_consensus.cc:3054] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.583326 14692 raft_consensus.cc:2463] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:11.584146 14819 ts_tablet_manager.cc:1918] T 0147f9743e924c9fb82a5bcd872e79fd P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.584201 14819 log.cc:1198] T 0147f9743e924c9fb82a5bcd872e79fd P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/0147f9743e924c9fb82a5bcd872e79fd
I20250115 03:06:11.584367 14819 ts_tablet_manager.cc:1939] T 0147f9743e924c9fb82a5bcd872e79fd P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:11.585786 14493 leader_election.cc:304] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.585933 14493 leader_election.cc:304] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.586786 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 0147f9743e924c9fb82a5bcd872e79fd (table default.Table_9928f846023c45a0bc9c92d5447958cb [id=39abf3d98dba4d63979ccc4c27af3a97]) successfully deleted
I20250115 03:06:11.586814 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 0147f9743e924c9fb82a5bcd872e79fd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:11.586997 14823 tablet_replica.cc:331] T 0147f9743e924c9fb82a5bcd872e79fd P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:11.587101 14823 raft_consensus.cc:2238] T 0147f9743e924c9fb82a5bcd872e79fd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.587188 14823 raft_consensus.cc:2267] T 0147f9743e924c9fb82a5bcd872e79fd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.587349 14817 raft_consensus.cc:2798] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:11.587508 14811 tablet_bootstrap.cc:492] T 4b21178f74d1465f9d6b64ac1b3a854b P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.587579 14811 ts_tablet_manager.cc:1397] T 4b21178f74d1465f9d6b64ac1b3a854b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.001s
I20250115 03:06:11.587716 14817 raft_consensus.cc:695] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:11.588016 14851 raft_consensus.cc:2798] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:11.587965 14811 raft_consensus.cc:357] T 4b21178f74d1465f9d6b64ac1b3a854b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.588122 14811 raft_consensus.cc:383] T 4b21178f74d1465f9d6b64ac1b3a854b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.588153 14811 raft_consensus.cc:738] T 4b21178f74d1465f9d6b64ac1b3a854b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.588254 14811 consensus_queue.cc:260] T 4b21178f74d1465f9d6b64ac1b3a854b P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.588450 14811 ts_tablet_manager.cc:1428] T 4b21178f74d1465f9d6b64ac1b3a854b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:11.588594 14811 tablet_bootstrap.cc:492] T fd5703df37db41daafc6697793355f31 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.588878 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc88653cdfec4120b37097b361f441d4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:11.588994 14692 raft_consensus.cc:2463] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:11.589296 14358 leader_election.cc:304] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.589453 14811 tablet_bootstrap.cc:654] T fd5703df37db41daafc6697793355f31 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.589507 14813 raft_consensus.cc:2798] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:11.589550 14813 raft_consensus.cc:491] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.589581 14813 raft_consensus.cc:3054] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.587975 14817 consensus_queue.cc:237] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.590327 14813 raft_consensus.cc:513] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.590646 14813 leader_election.cc:290] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.591058 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc88653cdfec4120b37097b361f441d4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:11.591142 14692 raft_consensus.cc:3054] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.591847 14692 raft_consensus.cc:2463] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:11.592087 14358 leader_election.cc:304] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.592288 14813 raft_consensus.cc:2798] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:11.592410 14813 raft_consensus.cc:695] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:11.592586 14813 consensus_queue.cc:237] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.593217 14820 ts_tablet_manager.cc:1905] T b061fea3b190420682a91662d0effd20 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.593654 14851 raft_consensus.cc:695] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:11.593801 14851 consensus_queue.cc:237] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.594849 14823 ts_tablet_manager.cc:1905] T 0147f9743e924c9fb82a5bcd872e79fd P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.594955 14820 ts_tablet_manager.cc:1918] T b061fea3b190420682a91662d0effd20 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.595021 14820 log.cc:1198] T b061fea3b190420682a91662d0effd20 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/b061fea3b190420682a91662d0effd20
I20250115 03:06:11.595208 14820 ts_tablet_manager.cc:1939] T b061fea3b190420682a91662d0effd20 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:11.595947 14809 tablet_bootstrap.cc:492] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.596028 14809 ts_tablet_manager.cc:1397] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.001s
I20250115 03:06:11.596241 14812 tablet_bootstrap.cc:492] T 4b21178f74d1465f9d6b64ac1b3a854b P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.596326 14812 ts_tablet_manager.cc:1397] T 4b21178f74d1465f9d6b64ac1b3a854b P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.000s
I20250115 03:06:11.596383 14809 raft_consensus.cc:357] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.596486 14809 raft_consensus.cc:383] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.596524 14809 raft_consensus.cc:738] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.596630 14809 consensus_queue.cc:260] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.596767 14812 raft_consensus.cc:357] T 4b21178f74d1465f9d6b64ac1b3a854b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.596861 14812 raft_consensus.cc:383] T 4b21178f74d1465f9d6b64ac1b3a854b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.596875 14809 ts_tablet_manager.cc:1428] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:11.596901 14812 raft_consensus.cc:738] T 4b21178f74d1465f9d6b64ac1b3a854b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.597047 14809 tablet_bootstrap.cc:492] T fd5703df37db41daafc6697793355f31 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.597038 14812 consensus_queue.cc:260] T 4b21178f74d1465f9d6b64ac1b3a854b P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.597882 14809 tablet_bootstrap.cc:654] T fd5703df37db41daafc6697793355f31 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.597965 14812 ts_tablet_manager.cc:1428] T 4b21178f74d1465f9d6b64ac1b3a854b P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:11.598673 14812 tablet_bootstrap.cc:492] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.599143 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc88653cdfec4120b37097b361f441d4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:11.599289 14557 raft_consensus.cc:2463] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:11.599532 14812 tablet_bootstrap.cc:654] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.599537 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc88653cdfec4120b37097b361f441d4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:11.599617 14556 raft_consensus.cc:3054] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.600006 14823 ts_tablet_manager.cc:1918] T 0147f9743e924c9fb82a5bcd872e79fd P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.600106 14823 log.cc:1198] T 0147f9743e924c9fb82a5bcd872e79fd P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/0147f9743e924c9fb82a5bcd872e79fd
I20250115 03:06:11.600296 14811 tablet_bootstrap.cc:492] T fd5703df37db41daafc6697793355f31 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.600322 14823 ts_tablet_manager.cc:1939] T 0147f9743e924c9fb82a5bcd872e79fd P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:11.600380 14811 ts_tablet_manager.cc:1397] T fd5703df37db41daafc6697793355f31 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20250115 03:06:11.600407 14556 raft_consensus.cc:2463] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:11.600806 14811 raft_consensus.cc:357] T fd5703df37db41daafc6697793355f31 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.600891 14811 raft_consensus.cc:383] T fd5703df37db41daafc6697793355f31 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.600942 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 0147f9743e924c9fb82a5bcd872e79fd (table default.Table_9928f846023c45a0bc9c92d5447958cb [id=39abf3d98dba4d63979ccc4c27af3a97]) successfully deleted
I20250115 03:06:11.600976 14811 raft_consensus.cc:738] T fd5703df37db41daafc6697793355f31 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.601084 14811 consensus_queue.cc:260] T fd5703df37db41daafc6697793355f31 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.601315 14811 ts_tablet_manager.cc:1428] T fd5703df37db41daafc6697793355f31 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.601459 14811 tablet_bootstrap.cc:492] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.602272 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 0147f9743e924c9fb82a5bcd872e79fd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.602411 14820 tablet_replica.cc:331] T 0147f9743e924c9fb82a5bcd872e79fd P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:11.602511 14820 raft_consensus.cc:2238] T 0147f9743e924c9fb82a5bcd872e79fd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.602594 14820 raft_consensus.cc:2267] T 0147f9743e924c9fb82a5bcd872e79fd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.602286 14811 tablet_bootstrap.cc:654] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.603271 14820 ts_tablet_manager.cc:1905] T 0147f9743e924c9fb82a5bcd872e79fd P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.604789 14820 ts_tablet_manager.cc:1918] T 0147f9743e924c9fb82a5bcd872e79fd P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.604853 14820 log.cc:1198] T 0147f9743e924c9fb82a5bcd872e79fd P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/0147f9743e924c9fb82a5bcd872e79fd
I20250115 03:06:11.604918 14166 master_service.cc:946] ReplaceTablet: received request to replace tablet 4b21178f74d1465f9d6b64ac1b3a854b from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:11.605019 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet 0147f9743e924c9fb82a5bcd872e79fd from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:11.605052 14820 ts_tablet_manager.cc:1939] T 0147f9743e924c9fb82a5bcd872e79fd P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:11.605103 14170 master_service.cc:946] ReplaceTablet: received request to replace tablet 44585e70c3434fa1915d31c46b1c0d50 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:11.605221 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet b061fea3b190420682a91662d0effd20 (table default.Table_9a7ec6b5bb5b4a9d962fc4748c2a6333 [id=5b89c9008ffd4a7d8149c8420dfc3d0e]) successfully deleted
I20250115 03:06:11.605538 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet fd5703df37db41daafc6697793355f31 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:11.605671 14165 master_service.cc:946] ReplaceTablet: received request to replace tablet 063999d856bf4ac28bb7ec01509d04a4 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:11.605765 14812 tablet_bootstrap.cc:492] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.605831 14812 ts_tablet_manager.cc:1397] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:11.606212 14812 raft_consensus.cc:357] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.606309 14812 raft_consensus.cc:383] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.606323 14809 tablet_bootstrap.cc:492] T fd5703df37db41daafc6697793355f31 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.606343 14812 raft_consensus.cc:738] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.606385 14809 ts_tablet_manager.cc:1397] T fd5703df37db41daafc6697793355f31 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20250115 03:06:11.606457 14812 consensus_queue.cc:260] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.606696 14812 ts_tablet_manager.cc:1428] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.606842 14812 tablet_bootstrap.cc:492] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.606948 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 0147f9743e924c9fb82a5bcd872e79fd (table default.Table_9928f846023c45a0bc9c92d5447958cb [id=39abf3d98dba4d63979ccc4c27af3a97]) successfully deleted
I20250115 03:06:11.607635 14809 raft_consensus.cc:357] T fd5703df37db41daafc6697793355f31 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.607743 14809 raft_consensus.cc:383] T fd5703df37db41daafc6697793355f31 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.607774 14809 raft_consensus.cc:738] T fd5703df37db41daafc6697793355f31 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.607820 14812 tablet_bootstrap.cc:654] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.607875 14809 consensus_queue.cc:260] T fd5703df37db41daafc6697793355f31 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.608103 14809 ts_tablet_manager.cc:1428] T fd5703df37db41daafc6697793355f31 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:11.608196 14811 tablet_bootstrap.cc:492] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.608261 14811 ts_tablet_manager.cc:1397] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:11.608665 14811 raft_consensus.cc:357] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.608752 14811 raft_consensus.cc:383] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.608781 14811 raft_consensus.cc:738] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.608886 14811 consensus_queue.cc:260] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.609141 14811 ts_tablet_manager.cc:1428] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.609302 14811 tablet_bootstrap.cc:492] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.610389 14811 tablet_bootstrap.cc:654] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.611033 14171 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 4379cf2eb12844a29c5f2e36d1e1dffc (table default.Table_4009ecd0a7ed495c99e5e4fd76c4774f [id=05f9fd2ebec34d2299ea59c47a74c828]). Expected version 1 got 0
I20250115 03:06:11.612408 14811 tablet_bootstrap.cc:492] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.612494 14811 ts_tablet_manager.cc:1397] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:11.612895 14811 raft_consensus.cc:357] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.613013 14811 raft_consensus.cc:383] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.613044 14811 raft_consensus.cc:738] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.613139 14811 consensus_queue.cc:260] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.613375 14811 ts_tablet_manager.cc:1428] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.615391 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 063999d856bf4ac28bb7ec01509d04a4
I20250115 03:06:11.615657 14165 catalog_manager.cc:6596] ReplaceTablet: tablet 063999d856bf4ac28bb7ec01509d04a4 deleted and replaced by tablet 35b90c4b0ef341c8a7c1bc517a1231ed
I20250115 03:06:11.616513 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 3ae1c21e6b564e08b6e7eec75290b117 (DEFAULT_TABLE table=default.Table_dee96a49a9754f88b4298601381f34c5 [id=9dcc0154ed914304a15cf96049afaf52]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.616567 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 063999d856bf4ac28bb7ec01509d04a4 with delete_type TABLET_DATA_DELETED (replaced by tablet 35b90c4b0ef341c8a7c1bc517a1231ed) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.616703 14819 tablet_replica.cc:331] T 063999d856bf4ac28bb7ec01509d04a4 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:11.616766 14671 tablet_service.cc:1467] Processing CreateTablet for tablet f64a668e2f4548f3bd4f2b0be32e8682 (DEFAULT_TABLE table=default.Table_03a772b18f874e3fb42b5dc7327e4543 [id=488a9a6268f64669ae17e1a67caa73a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.616816 14812 tablet_bootstrap.cc:492] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.616815 14819 raft_consensus.cc:2238] T 063999d856bf4ac28bb7ec01509d04a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.616883 14812 ts_tablet_manager.cc:1397] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20250115 03:06:11.616722 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ae1c21e6b564e08b6e7eec75290b117. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.616997 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f64a668e2f4548f3bd4f2b0be32e8682. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.617277 14812 raft_consensus.cc:357] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.617355 14812 raft_consensus.cc:383] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.617383 14812 raft_consensus.cc:738] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.617487 14812 consensus_queue.cc:260] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.617799 14812 ts_tablet_manager.cc:1428] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.617942 14812 tablet_bootstrap.cc:492] T fd5703df37db41daafc6697793355f31 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.618618 14812 tablet_bootstrap.cc:654] T fd5703df37db41daafc6697793355f31 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.618690 14670 tablet_service.cc:1514] Processing DeleteTablet for tablet 063999d856bf4ac28bb7ec01509d04a4 with delete_type TABLET_DATA_DELETED (replaced by tablet 35b90c4b0ef341c8a7c1bc517a1231ed) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:11.618956 14809 tablet_bootstrap.cc:492] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.618920 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 3ae1c21e6b564e08b6e7eec75290b117 (DEFAULT_TABLE table=default.Table_dee96a49a9754f88b4298601381f34c5 [id=9dcc0154ed914304a15cf96049afaf52]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.619107 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ae1c21e6b564e08b6e7eec75290b117. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.619580 14812 tablet_bootstrap.cc:492] T fd5703df37db41daafc6697793355f31 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.619652 14812 ts_tablet_manager.cc:1397] T fd5703df37db41daafc6697793355f31 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:11.619891 14809 tablet_bootstrap.cc:654] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.620049 14812 raft_consensus.cc:357] T fd5703df37db41daafc6697793355f31 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.620146 14812 raft_consensus.cc:383] T fd5703df37db41daafc6697793355f31 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.620178 14812 raft_consensus.cc:738] T fd5703df37db41daafc6697793355f31 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.620278 14812 consensus_queue.cc:260] T fd5703df37db41daafc6697793355f31 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.620537 14812 ts_tablet_manager.cc:1428] T fd5703df37db41daafc6697793355f31 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.616880 14402 tablet_service.cc:1467] Processing CreateTablet for tablet f64a668e2f4548f3bd4f2b0be32e8682 (DEFAULT_TABLE table=default.Table_03a772b18f874e3fb42b5dc7327e4543 [id=488a9a6268f64669ae17e1a67caa73a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.620677 14812 tablet_bootstrap.cc:492] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.620784 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f64a668e2f4548f3bd4f2b0be32e8682. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.620913 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 3ae1c21e6b564e08b6e7eec75290b117 (DEFAULT_TABLE table=default.Table_dee96a49a9754f88b4298601381f34c5 [id=9dcc0154ed914304a15cf96049afaf52]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.621124 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ae1c21e6b564e08b6e7eec75290b117. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.621505 14812 tablet_bootstrap.cc:654] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.622428 14535 tablet_service.cc:1467] Processing CreateTablet for tablet f64a668e2f4548f3bd4f2b0be32e8682 (DEFAULT_TABLE table=default.Table_03a772b18f874e3fb42b5dc7327e4543 [id=488a9a6268f64669ae17e1a67caa73a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.622563 14812 tablet_bootstrap.cc:492] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.622627 14812 ts_tablet_manager.cc:1397] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:11.622637 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f64a668e2f4548f3bd4f2b0be32e8682. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.622834 14811 tablet_bootstrap.cc:492] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.616892 14819 raft_consensus.cc:2267] T 063999d856bf4ac28bb7ec01509d04a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.622987 14812 raft_consensus.cc:357] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.623063 14812 raft_consensus.cc:383] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.623087 14812 raft_consensus.cc:738] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.623180 14812 consensus_queue.cc:260] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.623499 14812 ts_tablet_manager.cc:1428] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.623646 14812 tablet_bootstrap.cc:492] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.623693 14811 tablet_bootstrap.cc:654] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.624027 14819 ts_tablet_manager.cc:1905] T 063999d856bf4ac28bb7ec01509d04a4 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.624357 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 063999d856bf4ac28bb7ec01509d04a4 with delete_type TABLET_DATA_DELETED (replaced by tablet 35b90c4b0ef341c8a7c1bc517a1231ed) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.624689 14812 tablet_bootstrap.cc:654] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.624698 14809 tablet_bootstrap.cc:492] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.624768 14809 ts_tablet_manager.cc:1397] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20250115 03:06:11.625201 14809 raft_consensus.cc:357] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.625289 14809 raft_consensus.cc:383] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.625319 14809 raft_consensus.cc:738] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.625413 14809 consensus_queue.cc:260] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.625650 14812 tablet_bootstrap.cc:492] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.625651 14809 ts_tablet_manager.cc:1428] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.625718 14812 ts_tablet_manager.cc:1397] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20250115 03:06:11.625703 14819 ts_tablet_manager.cc:1918] T 063999d856bf4ac28bb7ec01509d04a4 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.625763 14819 log.cc:1198] T 063999d856bf4ac28bb7ec01509d04a4 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/063999d856bf4ac28bb7ec01509d04a4
I20250115 03:06:11.625797 14809 tablet_bootstrap.cc:492] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.625936 14819 ts_tablet_manager.cc:1939] T 063999d856bf4ac28bb7ec01509d04a4 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:11.626080 14812 raft_consensus.cc:357] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.626163 14812 raft_consensus.cc:383] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.626191 14812 raft_consensus.cc:738] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.626286 14812 consensus_queue.cc:260] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.626394 14820 tablet_replica.cc:331] T 063999d856bf4ac28bb7ec01509d04a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:11.626489 14820 raft_consensus.cc:2238] T 063999d856bf4ac28bb7ec01509d04a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.626534 14812 ts_tablet_manager.cc:1428] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:11.626581 14820 raft_consensus.cc:2267] T 063999d856bf4ac28bb7ec01509d04a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.626610 14809 tablet_bootstrap.cc:654] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.627004 14820 ts_tablet_manager.cc:1905] T 063999d856bf4ac28bb7ec01509d04a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.627053 14823 tablet_replica.cc:331] T 063999d856bf4ac28bb7ec01509d04a4 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:11.627148 14823 raft_consensus.cc:2238] T 063999d856bf4ac28bb7ec01509d04a4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.627229 14823 raft_consensus.cc:2267] T 063999d856bf4ac28bb7ec01509d04a4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.627418 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 063999d856bf4ac28bb7ec01509d04a4 (table default.Table_db8eb767c1214ff89ccfb64f0ad123ae [id=1b4a3ef0246a4341869bf5163487cac6]) successfully deleted
I20250115 03:06:11.627697 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 44585e70c3434fa1915d31c46b1c0d50
I20250115 03:06:11.628562 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 44585e70c3434fa1915d31c46b1c0d50 with delete_type TABLET_DATA_DELETED (replaced by tablet 3c19a41bd4d149bebdd81cba3fa87d01) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:11.628949 14809 tablet_bootstrap.cc:492] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.629034 14809 ts_tablet_manager.cc:1397] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:11.629420 14809 raft_consensus.cc:357] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.629514 14809 raft_consensus.cc:383] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.629544 14809 raft_consensus.cc:738] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.629627 14170 catalog_manager.cc:6596] ReplaceTablet: tablet 44585e70c3434fa1915d31c46b1c0d50 deleted and replaced by tablet 3c19a41bd4d149bebdd81cba3fa87d01
I20250115 03:06:11.629640 14809 consensus_queue.cc:260] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.629874 14820 ts_tablet_manager.cc:1918] T 063999d856bf4ac28bb7ec01509d04a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.629940 14820 log.cc:1198] T 063999d856bf4ac28bb7ec01509d04a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/063999d856bf4ac28bb7ec01509d04a4
I20250115 03:06:11.630044 14402 tablet_service.cc:1514] Processing DeleteTablet for tablet 44585e70c3434fa1915d31c46b1c0d50 with delete_type TABLET_DATA_DELETED (replaced by tablet 3c19a41bd4d149bebdd81cba3fa87d01) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.630116 14820 ts_tablet_manager.cc:1939] T 063999d856bf4ac28bb7ec01509d04a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:11.630290 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 44585e70c3434fa1915d31c46b1c0d50 with delete_type TABLET_DATA_DELETED (replaced by tablet 3c19a41bd4d149bebdd81cba3fa87d01) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.630575 14820 tablet_replica.cc:331] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:11.630673 14820 raft_consensus.cc:2238] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:11.630882 14820 pending_rounds.cc:62] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1: Trying to abort 1 pending ops.
I20250115 03:06:11.630946 14809 ts_tablet_manager.cc:1428] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:11.630978 14820 pending_rounds.cc:69] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384882036101120 op_type: NO_OP noop_request { }
I20250115 03:06:11.631062 14820 raft_consensus.cc:2883] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:11.631129 14820 raft_consensus.cc:2267] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.631263 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 4b21178f74d1465f9d6b64ac1b3a854b
I20250115 03:06:11.631415 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 063999d856bf4ac28bb7ec01509d04a4 (table default.Table_db8eb767c1214ff89ccfb64f0ad123ae [id=1b4a3ef0246a4341869bf5163487cac6]) successfully deleted
I20250115 03:06:11.631526 14166 catalog_manager.cc:6596] ReplaceTablet: tablet 4b21178f74d1465f9d6b64ac1b3a854b deleted and replaced by tablet 3a786573d49e45669a3a2ef2385afc19
I20250115 03:06:11.631740 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b21178f74d1465f9d6b64ac1b3a854b with delete_type TABLET_DATA_DELETED (replaced by tablet 3a786573d49e45669a3a2ef2385afc19) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.632119 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet fd5703df37db41daafc6697793355f31
I20250115 03:06:11.632153 14820 ts_tablet_manager.cc:1905] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.632360 14168 catalog_manager.cc:6596] ReplaceTablet: tablet fd5703df37db41daafc6697793355f31 deleted and replaced by tablet 496d3bd59f174412b7c26d9903972366
I20250115 03:06:11.632517 14819 tablet_replica.cc:331] T 44585e70c3434fa1915d31c46b1c0d50 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:11.632614 14819 raft_consensus.cc:2238] T 44585e70c3434fa1915d31c46b1c0d50 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.632445 14171 catalog_manager.cc:5526] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:11.632690 14819 raft_consensus.cc:2267] T 44585e70c3434fa1915d31c46b1c0d50 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.632918 14171 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet cc88653cdfec4120b37097b361f441d4 (table default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd]). Expected version 2 got 0
I20250115 03:06:11.633090 14171 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 4f32d3e059e44722af83c55b71f14c26 (table default.Table_c1020680eef344a49998eddac921fea5 [id=649880f1ff0d4cfda2358b408110d8dc]). Expected version 1 got 0
I20250115 03:06:11.633093 14819 ts_tablet_manager.cc:1905] T 44585e70c3434fa1915d31c46b1c0d50 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.633802 14820 ts_tablet_manager.cc:1918] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:11.633867 14820 log.cc:1198] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/44585e70c3434fa1915d31c46b1c0d50
I20250115 03:06:11.634198 14670 tablet_service.cc:1514] Processing DeleteTablet for tablet fd5703df37db41daafc6697793355f31 with delete_type TABLET_DATA_DELETED (replaced by tablet 496d3bd59f174412b7c26d9903972366) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:11.634212 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet fd5703df37db41daafc6697793355f31 with delete_type TABLET_DATA_DELETED (replaced by tablet 496d3bd59f174412b7c26d9903972366) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.634256 14853 raft_consensus.cc:491] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.634303 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b21178f74d1465f9d6b64ac1b3a854b with delete_type TABLET_DATA_DELETED (replaced by tablet 3a786573d49e45669a3a2ef2385afc19) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:11.634312 14853 raft_consensus.cc:513] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.634522 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet fd5703df37db41daafc6697793355f31 with delete_type TABLET_DATA_DELETED (replaced by tablet 496d3bd59f174412b7c26d9903972366) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.634673 14819 ts_tablet_manager.cc:1918] T 44585e70c3434fa1915d31c46b1c0d50 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.634209 14820 ts_tablet_manager.cc:1939] T 44585e70c3434fa1915d31c46b1c0d50 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:11.634904 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4379cf2eb12844a29c5f2e36d1e1dffc" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:11.635033 14422 raft_consensus.cc:2463] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:11.635259 14823 ts_tablet_manager.cc:1905] T 063999d856bf4ac28bb7ec01509d04a4 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.635275 14493 leader_election.cc:304] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.634685 14402 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b21178f74d1465f9d6b64ac1b3a854b with delete_type TABLET_DATA_DELETED (replaced by tablet 3a786573d49e45669a3a2ef2385afc19) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.634943 14819 log.cc:1198] T 44585e70c3434fa1915d31c46b1c0d50 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/44585e70c3434fa1915d31c46b1c0d50
I20250115 03:06:11.635424 14858 raft_consensus.cc:2798] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:11.635479 14858 raft_consensus.cc:491] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.635531 14858 raft_consensus.cc:3054] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.635886 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 44585e70c3434fa1915d31c46b1c0d50 (table default.Table_d46d2affc6e74c7a8d3cc2a8ec0233e6 [id=d9fda51f2bed4c1ca720fe30ec97f440]) successfully deleted
I20250115 03:06:11.636142 14819 ts_tablet_manager.cc:1939] T 44585e70c3434fa1915d31c46b1c0d50 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:11.636277 14858 raft_consensus.cc:513] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.636605 14858 leader_election.cc:290] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.636672 14811 tablet_bootstrap.cc:492] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.636747 14811 ts_tablet_manager.cc:1397] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20250115 03:06:11.636831 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 44585e70c3434fa1915d31c46b1c0d50 (table default.Table_d46d2affc6e74c7a8d3cc2a8ec0233e6 [id=d9fda51f2bed4c1ca720fe30ec97f440]) successfully deleted
I20250115 03:06:11.637081 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4379cf2eb12844a29c5f2e36d1e1dffc" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:11.637187 14692 raft_consensus.cc:3054] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.637168 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4379cf2eb12844a29c5f2e36d1e1dffc" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:11.637259 14422 raft_consensus.cc:3054] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.637269 14169 catalog_manager.cc:5526] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:11.637430 14169 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 180df492348a4cc09afa03c8a62a79c1 (table default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]). Expected version 1 got 0
I20250115 03:06:11.637559 14169 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 4379cf2eb12844a29c5f2e36d1e1dffc (table default.Table_4009ecd0a7ed495c99e5e4fd76c4774f [id=05f9fd2ebec34d2299ea59c47a74c828]). Expected version 1 got 0
I20250115 03:06:11.637624 14169 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 4f32d3e059e44722af83c55b71f14c26 (table default.Table_c1020680eef344a49998eddac921fea5 [id=649880f1ff0d4cfda2358b408110d8dc]). Expected version 1 got 0
I20250115 03:06:11.638006 14422 raft_consensus.cc:2463] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:11.638012 14692 raft_consensus.cc:2463] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:11.638164 14823 ts_tablet_manager.cc:1918] T 063999d856bf4ac28bb7ec01509d04a4 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.638144 14811 raft_consensus.cc:357] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.638221 14823 log.cc:1198] T 063999d856bf4ac28bb7ec01509d04a4 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/063999d856bf4ac28bb7ec01509d04a4
I20250115 03:06:11.638228 14811 raft_consensus.cc:383] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.638260 14811 raft_consensus.cc:738] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.638278 14493 leader_election.cc:304] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.638360 14811 consensus_queue.cc:260] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.638391 14823 ts_tablet_manager.cc:1939] T 063999d856bf4ac28bb7ec01509d04a4 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:11.638545 14858 raft_consensus.cc:2798] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:11.638599 14811 ts_tablet_manager.cc:1428] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:11.638761 14811 tablet_bootstrap.cc:492] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.638868 14823 tablet_replica.cc:331] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:11.638967 14820 tablet_replica.cc:331] T 4b21178f74d1465f9d6b64ac1b3a854b P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:11.638957 14823 raft_consensus.cc:2238] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.639046 14820 raft_consensus.cc:2238] T 4b21178f74d1465f9d6b64ac1b3a854b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.639032 14823 raft_consensus.cc:2267] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.639127 14820 raft_consensus.cc:2267] T 4b21178f74d1465f9d6b64ac1b3a854b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.639446 14823 ts_tablet_manager.cc:1905] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.639503 14820 ts_tablet_manager.cc:1905] T 4b21178f74d1465f9d6b64ac1b3a854b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.639717 14811 tablet_bootstrap.cc:654] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.639894 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 063999d856bf4ac28bb7ec01509d04a4 (table default.Table_db8eb767c1214ff89ccfb64f0ad123ae [id=1b4a3ef0246a4341869bf5163487cac6]) successfully deleted
I20250115 03:06:11.638614 14858 raft_consensus.cc:695] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:11.640969 14858 consensus_queue.cc:237] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.641060 14820 ts_tablet_manager.cc:1918] T 4b21178f74d1465f9d6b64ac1b3a854b P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.641060 14823 ts_tablet_manager.cc:1918] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.641117 14823 log.cc:1198] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/44585e70c3434fa1915d31c46b1c0d50
I20250115 03:06:11.641117 14820 log.cc:1198] T 4b21178f74d1465f9d6b64ac1b3a854b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/4b21178f74d1465f9d6b64ac1b3a854b
I20250115 03:06:11.641287 14820 ts_tablet_manager.cc:1939] T 4b21178f74d1465f9d6b64ac1b3a854b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:11.641287 14823 ts_tablet_manager.cc:1939] T 44585e70c3434fa1915d31c46b1c0d50 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:11.641405 14819 tablet_replica.cc:331] T fd5703df37db41daafc6697793355f31 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:11.641539 14819 raft_consensus.cc:2238] T fd5703df37db41daafc6697793355f31 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.641614 14853 leader_election.cc:290] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.641785 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4379cf2eb12844a29c5f2e36d1e1dffc" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:11.641898 14692 raft_consensus.cc:2371] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1. Re-sending same reply.
I20250115 03:06:11.642262 14823 tablet_replica.cc:331] T fd5703df37db41daafc6697793355f31 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:11.642294 14819 raft_consensus.cc:2267] T fd5703df37db41daafc6697793355f31 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.642343 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 44585e70c3434fa1915d31c46b1c0d50 (table default.Table_d46d2affc6e74c7a8d3cc2a8ec0233e6 [id=d9fda51f2bed4c1ca720fe30ec97f440]) successfully deleted
I20250115 03:06:11.642361 14823 raft_consensus.cc:2238] T fd5703df37db41daafc6697793355f31 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.642436 14823 raft_consensus.cc:2267] T fd5703df37db41daafc6697793355f31 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.643028 14823 ts_tablet_manager.cc:1905] T fd5703df37db41daafc6697793355f31 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.643482 14819 ts_tablet_manager.cc:1905] T fd5703df37db41daafc6697793355f31 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.644593 14823 ts_tablet_manager.cc:1918] T fd5703df37db41daafc6697793355f31 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.644688 14823 log.cc:1198] T fd5703df37db41daafc6697793355f31 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/fd5703df37db41daafc6697793355f31
I20250115 03:06:11.644888 14823 ts_tablet_manager.cc:1939] T fd5703df37db41daafc6697793355f31 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:11.645242 14819 ts_tablet_manager.cc:1918] T fd5703df37db41daafc6697793355f31 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.645327 14819 log.cc:1198] T fd5703df37db41daafc6697793355f31 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/fd5703df37db41daafc6697793355f31
I20250115 03:06:11.645391 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet fd5703df37db41daafc6697793355f31 (table default.Table_b12aff7a3c8d41f79392773a151c463a [id=2c420d8e08fd407386e8566fa323a55f]) successfully deleted
I20250115 03:06:11.645561 14819 ts_tablet_manager.cc:1939] T fd5703df37db41daafc6697793355f31 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:11.645581 14823 tablet_replica.cc:331] T 4b21178f74d1465f9d6b64ac1b3a854b P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:11.645665 14823 raft_consensus.cc:2238] T 4b21178f74d1465f9d6b64ac1b3a854b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.645756 14823 raft_consensus.cc:2267] T 4b21178f74d1465f9d6b64ac1b3a854b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.646008 14819 tablet_replica.cc:331] T 4b21178f74d1465f9d6b64ac1b3a854b P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:11.646106 14819 raft_consensus.cc:2238] T 4b21178f74d1465f9d6b64ac1b3a854b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.646188 14820 tablet_replica.cc:331] T fd5703df37db41daafc6697793355f31 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:11.646198 14819 raft_consensus.cc:2267] T 4b21178f74d1465f9d6b64ac1b3a854b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.646235 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 4b21178f74d1465f9d6b64ac1b3a854b (table default.Table_1132d16f9e04448689aab85d1952013b [id=9f650455cb3440a49a11010f541084db]) successfully deleted
I20250115 03:06:11.646286 14820 raft_consensus.cc:2238] T fd5703df37db41daafc6697793355f31 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.646400 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet fd5703df37db41daafc6697793355f31 (table default.Table_b12aff7a3c8d41f79392773a151c463a [id=2c420d8e08fd407386e8566fa323a55f]) successfully deleted
I20250115 03:06:11.646534 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 44585e70c3434fa1915d31c46b1c0d50 with delete_type TABLET_DATA_DELETED (replaced by tablet 3c19a41bd4d149bebdd81cba3fa87d01) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.646797 14819 ts_tablet_manager.cc:1905] T 4b21178f74d1465f9d6b64ac1b3a854b P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.647260 14823 ts_tablet_manager.cc:1905] T 4b21178f74d1465f9d6b64ac1b3a854b P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.648314 14819 ts_tablet_manager.cc:1918] T 4b21178f74d1465f9d6b64ac1b3a854b P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.648379 14819 log.cc:1198] T 4b21178f74d1465f9d6b64ac1b3a854b P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/4b21178f74d1465f9d6b64ac1b3a854b
I20250115 03:06:11.648566 14819 ts_tablet_manager.cc:1939] T 4b21178f74d1465f9d6b64ac1b3a854b P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:11.649001 14811 tablet_bootstrap.cc:492] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.649071 14811 ts_tablet_manager.cc:1397] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20250115 03:06:11.649302 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 4379cf2eb12844a29c5f2e36d1e1dffc (table default.Table_4009ecd0a7ed495c99e5e4fd76c4774f [id=05f9fd2ebec34d2299ea59c47a74c828]). Expected version 1 got 0
I20250115 03:06:11.649430 14811 raft_consensus.cc:357] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.649515 14811 raft_consensus.cc:383] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.649539 14811 raft_consensus.cc:738] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.649633 14811 consensus_queue.cc:260] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.649848 14811 ts_tablet_manager.cc:1428] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.650159 14820 raft_consensus.cc:2267] T fd5703df37db41daafc6697793355f31 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.650300 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 35b90c4b0ef341c8a7c1bc517a1231ed (DEFAULT_TABLE table=default.Table_db8eb767c1214ff89ccfb64f0ad123ae [id=1b4a3ef0246a4341869bf5163487cac6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.650512 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35b90c4b0ef341c8a7c1bc517a1231ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.650552 14820 ts_tablet_manager.cc:1905] T fd5703df37db41daafc6697793355f31 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.652640 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 4b21178f74d1465f9d6b64ac1b3a854b (table default.Table_1132d16f9e04448689aab85d1952013b [id=9f650455cb3440a49a11010f541084db]) successfully deleted
I20250115 03:06:11.652658 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 35b90c4b0ef341c8a7c1bc517a1231ed (DEFAULT_TABLE table=default.Table_db8eb767c1214ff89ccfb64f0ad123ae [id=1b4a3ef0246a4341869bf5163487cac6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.652873 14809 tablet_bootstrap.cc:492] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.653131 14823 ts_tablet_manager.cc:1918] T 4b21178f74d1465f9d6b64ac1b3a854b P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.653199 14823 log.cc:1198] T 4b21178f74d1465f9d6b64ac1b3a854b P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/4b21178f74d1465f9d6b64ac1b3a854b
I20250115 03:06:11.653380 14823 ts_tablet_manager.cc:1939] T 4b21178f74d1465f9d6b64ac1b3a854b P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:11.654011 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 35b90c4b0ef341c8a7c1bc517a1231ed (DEFAULT_TABLE table=default.Table_db8eb767c1214ff89ccfb64f0ad123ae [id=1b4a3ef0246a4341869bf5163487cac6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.654232 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35b90c4b0ef341c8a7c1bc517a1231ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.654707 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35b90c4b0ef341c8a7c1bc517a1231ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.655076 14692 raft_consensus.cc:1270] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:11.655542 14169 catalog_manager.cc:5526] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:11.655642 14861 consensus_queue.cc:1035] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:11.655718 14169 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 4379cf2eb12844a29c5f2e36d1e1dffc (table default.Table_4009ecd0a7ed495c99e5e4fd76c4774f [id=05f9fd2ebec34d2299ea59c47a74c828]). Expected version 1 got 0
I20250115 03:06:11.656139 14812 tablet_bootstrap.cc:492] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.656419 14811 tablet_bootstrap.cc:492] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.657016 14812 tablet_bootstrap.cc:654] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.657274 14811 tablet_bootstrap.cc:654] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.658154 14812 tablet_bootstrap.cc:492] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.658226 14812 ts_tablet_manager.cc:1397] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20250115 03:06:11.658236 14820 ts_tablet_manager.cc:1918] T fd5703df37db41daafc6697793355f31 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.658418 14820 log.cc:1198] T fd5703df37db41daafc6697793355f31 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/fd5703df37db41daafc6697793355f31
I20250115 03:06:11.658601 14820 ts_tablet_manager.cc:1939] T fd5703df37db41daafc6697793355f31 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:11.658624 14812 raft_consensus.cc:357] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.658711 14812 raft_consensus.cc:383] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.658752 14812 raft_consensus.cc:738] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.658870 14812 consensus_queue.cc:260] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.658991 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 3a786573d49e45669a3a2ef2385afc19 (DEFAULT_TABLE table=default.Table_1132d16f9e04448689aab85d1952013b [id=9f650455cb3440a49a11010f541084db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.659262 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a786573d49e45669a3a2ef2385afc19. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.660401 14812 ts_tablet_manager.cc:1428] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:11.660773 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 3a786573d49e45669a3a2ef2385afc19 (DEFAULT_TABLE table=default.Table_1132d16f9e04448689aab85d1952013b [id=9f650455cb3440a49a11010f541084db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.660830 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 3c19a41bd4d149bebdd81cba3fa87d01 (DEFAULT_TABLE table=default.Table_d46d2affc6e74c7a8d3cc2a8ec0233e6 [id=d9fda51f2bed4c1ca720fe30ec97f440]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.660982 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a786573d49e45669a3a2ef2385afc19. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.661056 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c19a41bd4d149bebdd81cba3fa87d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.661778 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 3c19a41bd4d149bebdd81cba3fa87d01 (DEFAULT_TABLE table=default.Table_d46d2affc6e74c7a8d3cc2a8ec0233e6 [id=d9fda51f2bed4c1ca720fe30ec97f440]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.662005 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c19a41bd4d149bebdd81cba3fa87d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.662104 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 4f32d3e059e44722af83c55b71f14c26 (table default.Table_c1020680eef344a49998eddac921fea5 [id=649880f1ff0d4cfda2358b408110d8dc]). Expected version 1 got 0
I20250115 03:06:11.662580 14812 tablet_bootstrap.cc:492] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.663363 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet fd5703df37db41daafc6697793355f31 (table default.Table_b12aff7a3c8d41f79392773a151c463a [id=2c420d8e08fd407386e8566fa323a55f]) successfully deleted
I20250115 03:06:11.663388 14812 tablet_bootstrap.cc:654] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:11.663614 14156 catalog_manager.cc:4839] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 44585e70c3434fa1915d31c46b1c0d50 because the tablet was not found. No further retry: Not found: Tablet not found: 44585e70c3434fa1915d31c46b1c0d50
I20250115 03:06:11.664466 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 3a786573d49e45669a3a2ef2385afc19 (DEFAULT_TABLE table=default.Table_1132d16f9e04448689aab85d1952013b [id=9f650455cb3440a49a11010f541084db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.664873 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 496d3bd59f174412b7c26d9903972366 (DEFAULT_TABLE table=default.Table_b12aff7a3c8d41f79392773a151c463a [id=2c420d8e08fd407386e8566fa323a55f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.665143 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 496d3bd59f174412b7c26d9903972366. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.665421 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet cc88653cdfec4120b37097b361f441d4 (table default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd]). Expected version 2 got 0
I20250115 03:06:11.665515 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 35b90c4b0ef341c8a7c1bc517a1231ed (table default.Table_db8eb767c1214ff89ccfb64f0ad123ae [id=1b4a3ef0246a4341869bf5163487cac6]). Expected version 2 got 0
I20250115 03:06:11.665530 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 3c19a41bd4d149bebdd81cba3fa87d01 (DEFAULT_TABLE table=default.Table_d46d2affc6e74c7a8d3cc2a8ec0233e6 [id=d9fda51f2bed4c1ca720fe30ec97f440]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.665724 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c19a41bd4d149bebdd81cba3fa87d01. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:11.665825 14167 catalog_manager.cc:5718] Failed to send AlterTable RPC for tablet 35b90c4b0ef341c8a7c1bc517a1231ed (table default.Table_db8eb767c1214ff89ccfb64f0ad123ae [id=1b4a3ef0246a4341869bf5163487cac6], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 35b90c4b0ef341c8a7c1bc517a1231ed consensus state has no leader
I20250115 03:06:11.666244 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 496d3bd59f174412b7c26d9903972366 (DEFAULT_TABLE table=default.Table_b12aff7a3c8d41f79392773a151c463a [id=2c420d8e08fd407386e8566fa323a55f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.666436 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 496d3bd59f174412b7c26d9903972366. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.664713 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a786573d49e45669a3a2ef2385afc19. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.668066 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 4b21178f74d1465f9d6b64ac1b3a854b (table default.Table_1132d16f9e04448689aab85d1952013b [id=9f650455cb3440a49a11010f541084db]) successfully deleted
I20250115 03:06:11.668251 14809 tablet_bootstrap.cc:654] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.668257 14883 tablet.cc:1721] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:11.669306 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 496d3bd59f174412b7c26d9903972366 (DEFAULT_TABLE table=default.Table_b12aff7a3c8d41f79392773a151c463a [id=2c420d8e08fd407386e8566fa323a55f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.669497 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 496d3bd59f174412b7c26d9903972366. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.669554 14556 raft_consensus.cc:1270] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:11.669752 14812 tablet_bootstrap.cc:492] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.669819 14812 ts_tablet_manager.cc:1397] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20250115 03:06:11.670060 14861 consensus_queue.cc:1035] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:11.670193 14812 raft_consensus.cc:357] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.670274 14812 raft_consensus.cc:383] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.670305 14812 raft_consensus.cc:738] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.670406 14812 consensus_queue.cc:260] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.670645 14812 ts_tablet_manager.cc:1428] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.671263 14877 mvcc.cc:204] Tried to move back new op lower bound from 7114384882291326976 to 7114384882045562880. Current Snapshot: MvccSnapshot[applied={T|T < 7114384882291326976}]
I20250115 03:06:11.671756 14812 tablet_bootstrap.cc:492] T 3c19a41bd4d149bebdd81cba3fa87d01 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.671815 14811 tablet_bootstrap.cc:492] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.671881 14811 ts_tablet_manager.cc:1397] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20250115 03:06:11.672266 14811 raft_consensus.cc:357] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.672351 14811 raft_consensus.cc:383] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.672382 14811 raft_consensus.cc:738] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.672489 14811 consensus_queue.cc:260] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.672626 14812 tablet_bootstrap.cc:654] T 3c19a41bd4d149bebdd81cba3fa87d01 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.672732 14811 ts_tablet_manager.cc:1428] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.672896 14811 tablet_bootstrap.cc:492] T 3c19a41bd4d149bebdd81cba3fa87d01 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.673738 14811 tablet_bootstrap.cc:654] T 3c19a41bd4d149bebdd81cba3fa87d01 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.674588 14169 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 35b90c4b0ef341c8a7c1bc517a1231ed (table default.Table_db8eb767c1214ff89ccfb64f0ad123ae [id=1b4a3ef0246a4341869bf5163487cac6]). Expected version 2 got 0
I20250115 03:06:11.674686 14169 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet cc88653cdfec4120b37097b361f441d4 (table default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd]). Expected version 2 got 0
I20250115 03:06:11.675402 14171 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet cc88653cdfec4120b37097b361f441d4 (table default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd]). Expected version 2 got 0
I20250115 03:06:11.676374 14809 tablet_bootstrap.cc:492] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.676445 14809 ts_tablet_manager.cc:1397] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.000s
I20250115 03:06:11.676839 14809 raft_consensus.cc:357] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.676951 14809 raft_consensus.cc:383] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.676985 14809 raft_consensus.cc:738] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.676918 14812 tablet_bootstrap.cc:492] T 3c19a41bd4d149bebdd81cba3fa87d01 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.677073 14809 consensus_queue.cc:260] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.677150 14812 ts_tablet_manager.cc:1397] T 3c19a41bd4d149bebdd81cba3fa87d01 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20250115 03:06:11.677310 14809 ts_tablet_manager.cc:1428] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.677465 14809 tablet_bootstrap.cc:492] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.677556 14812 raft_consensus.cc:357] T 3c19a41bd4d149bebdd81cba3fa87d01 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.677642 14812 raft_consensus.cc:383] T 3c19a41bd4d149bebdd81cba3fa87d01 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.677671 14812 raft_consensus.cc:738] T 3c19a41bd4d149bebdd81cba3fa87d01 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.677768 14812 consensus_queue.cc:260] T 3c19a41bd4d149bebdd81cba3fa87d01 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.677990 14812 ts_tablet_manager.cc:1428] T 3c19a41bd4d149bebdd81cba3fa87d01 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.678129 14812 tablet_bootstrap.cc:492] T 496d3bd59f174412b7c26d9903972366 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.678279 14809 tablet_bootstrap.cc:654] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.678928 14812 tablet_bootstrap.cc:654] T 496d3bd59f174412b7c26d9903972366 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.679281 14880 mvcc.cc:204] Tried to move back new op lower bound from 7114384882291326976 to 7114384882045562880. Current Snapshot: MvccSnapshot[applied={T|T < 7114384882291326976}]
I20250115 03:06:11.679350 14167 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd] alter complete (version 2)
I20250115 03:06:11.679844 14811 tablet_bootstrap.cc:492] T 3c19a41bd4d149bebdd81cba3fa87d01 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.679917 14811 ts_tablet_manager.cc:1397] T 3c19a41bd4d149bebdd81cba3fa87d01 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20250115 03:06:11.680279 14811 raft_consensus.cc:357] T 3c19a41bd4d149bebdd81cba3fa87d01 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.680369 14811 raft_consensus.cc:383] T 3c19a41bd4d149bebdd81cba3fa87d01 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.680398 14811 raft_consensus.cc:738] T 3c19a41bd4d149bebdd81cba3fa87d01 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.680495 14811 consensus_queue.cc:260] T 3c19a41bd4d149bebdd81cba3fa87d01 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.680748 14811 ts_tablet_manager.cc:1428] T 3c19a41bd4d149bebdd81cba3fa87d01 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:11.681078 14811 tablet_bootstrap.cc:492] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.681248 14809 tablet_bootstrap.cc:492] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.681310 14809 ts_tablet_manager.cc:1397] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:11.681684 14809 raft_consensus.cc:357] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.681766 14809 raft_consensus.cc:383] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.681794 14809 raft_consensus.cc:738] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.681893 14809 consensus_queue.cc:260] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.682030 14811 tablet_bootstrap.cc:654] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.682128 14809 ts_tablet_manager.cc:1428] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.682188 14171 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 35b90c4b0ef341c8a7c1bc517a1231ed (table default.Table_db8eb767c1214ff89ccfb64f0ad123ae [id=1b4a3ef0246a4341869bf5163487cac6]). Expected version 2 got 0
I20250115 03:06:11.680809 14886 mvcc.cc:204] Tried to move back new op lower bound from 7114384882291326976 to 7114384882045562880. Current Snapshot: MvccSnapshot[applied={T|T < 7114384882291326976}]
I20250115 03:06:11.682279 14809 tablet_bootstrap.cc:492] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.682279 14171 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet cc88653cdfec4120b37097b361f441d4 (table default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd]). Expected version 2 got 0
I20250115 03:06:11.682922 14891 tablet.cc:1721] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:11.683153 14809 tablet_bootstrap.cc:654] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.684412 14812 tablet_bootstrap.cc:492] T 496d3bd59f174412b7c26d9903972366 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.684437 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 3c19a41bd4d149bebdd81cba3fa87d01 (table default.Table_d46d2affc6e74c7a8d3cc2a8ec0233e6 [id=d9fda51f2bed4c1ca720fe30ec97f440]). Expected version 1 got 0
I20250115 03:06:11.684502 14812 ts_tablet_manager.cc:1397] T 496d3bd59f174412b7c26d9903972366 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:11.684530 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 35b90c4b0ef341c8a7c1bc517a1231ed (table default.Table_db8eb767c1214ff89ccfb64f0ad123ae [id=1b4a3ef0246a4341869bf5163487cac6]). Expected version 2 got 0
I20250115 03:06:11.684903 14812 raft_consensus.cc:357] T 496d3bd59f174412b7c26d9903972366 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:11.685019 14812 raft_consensus.cc:383] T 496d3bd59f174412b7c26d9903972366 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.685046 14812 raft_consensus.cc:738] T 496d3bd59f174412b7c26d9903972366 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.685149 14812 consensus_queue.cc:260] T 496d3bd59f174412b7c26d9903972366 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:11.685369 14812 ts_tablet_manager.cc:1428] T 496d3bd59f174412b7c26d9903972366 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:11.685375 14811 tablet_bootstrap.cc:492] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.685444 14811 ts_tablet_manager.cc:1397] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:11.685842 14811 raft_consensus.cc:357] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.685940 14811 raft_consensus.cc:383] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.685967 14811 raft_consensus.cc:738] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.686051 14811 consensus_queue.cc:260] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.686179 14893 tablet.cc:1721] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:11.686285 14811 ts_tablet_manager.cc:1428] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.686435 14811 tablet_bootstrap.cc:492] T 496d3bd59f174412b7c26d9903972366 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.686533 14809 tablet_bootstrap.cc:492] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.686602 14809 ts_tablet_manager.cc:1397] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
W20250115 03:06:11.684912 14169 catalog_manager.cc:5718] Failed to send AlterTable RPC for tablet 3c19a41bd4d149bebdd81cba3fa87d01 (table default.Table_d46d2affc6e74c7a8d3cc2a8ec0233e6 [id=d9fda51f2bed4c1ca720fe30ec97f440], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 3c19a41bd4d149bebdd81cba3fa87d01 consensus state has no leader
I20250115 03:06:11.687003 14809 raft_consensus.cc:357] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.687085 14809 raft_consensus.cc:383] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.687114 14809 raft_consensus.cc:738] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.687239 14811 tablet_bootstrap.cc:654] T 496d3bd59f174412b7c26d9903972366 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.687213 14809 consensus_queue.cc:260] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.687448 14809 ts_tablet_manager.cc:1428] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.687599 14809 tablet_bootstrap.cc:492] T 496d3bd59f174412b7c26d9903972366 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.688371 14809 tablet_bootstrap.cc:654] T 496d3bd59f174412b7c26d9903972366 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.685978 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 3c19a41bd4d149bebdd81cba3fa87d01 (table default.Table_d46d2affc6e74c7a8d3cc2a8ec0233e6 [id=d9fda51f2bed4c1ca720fe30ec97f440]). Expected version 1 got 0
I20250115 03:06:11.689311 14171 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 3c19a41bd4d149bebdd81cba3fa87d01 (table default.Table_d46d2affc6e74c7a8d3cc2a8ec0233e6 [id=d9fda51f2bed4c1ca720fe30ec97f440]). Expected version 1 got 0
I20250115 03:06:11.690300 14809 tablet_bootstrap.cc:492] T 496d3bd59f174412b7c26d9903972366 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.690377 14809 ts_tablet_manager.cc:1397] T 496d3bd59f174412b7c26d9903972366 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:11.690466 14811 tablet_bootstrap.cc:492] T 496d3bd59f174412b7c26d9903972366 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.690539 14811 ts_tablet_manager.cc:1397] T 496d3bd59f174412b7c26d9903972366 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20250115 03:06:11.690765 14809 raft_consensus.cc:357] T 496d3bd59f174412b7c26d9903972366 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:11.690924 14811 raft_consensus.cc:357] T 496d3bd59f174412b7c26d9903972366 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:11.691007 14811 raft_consensus.cc:383] T 496d3bd59f174412b7c26d9903972366 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.691036 14811 raft_consensus.cc:738] T 496d3bd59f174412b7c26d9903972366 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.691140 14811 consensus_queue.cc:260] T 496d3bd59f174412b7c26d9903972366 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:11.690855 14809 raft_consensus.cc:383] T 496d3bd59f174412b7c26d9903972366 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.691375 14811 ts_tablet_manager.cc:1428] T 496d3bd59f174412b7c26d9903972366 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.691412 14809 raft_consensus.cc:738] T 496d3bd59f174412b7c26d9903972366 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.691632 14809 consensus_queue.cc:260] T 496d3bd59f174412b7c26d9903972366 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:11.692185 14169 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 496d3bd59f174412b7c26d9903972366 (table default.Table_b12aff7a3c8d41f79392773a151c463a [id=2c420d8e08fd407386e8566fa323a55f]). Expected version 1 got 0
W20250115 03:06:11.692503 14169 catalog_manager.cc:5718] Failed to send AlterTable RPC for tablet 496d3bd59f174412b7c26d9903972366 (table default.Table_b12aff7a3c8d41f79392773a151c463a [id=2c420d8e08fd407386e8566fa323a55f], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 496d3bd59f174412b7c26d9903972366 consensus state has no leader
I20250115 03:06:11.693117 14809 ts_tablet_manager.cc:1428] T 496d3bd59f174412b7c26d9903972366 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:11.694489 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 496d3bd59f174412b7c26d9903972366 (table default.Table_b12aff7a3c8d41f79392773a151c463a [id=2c420d8e08fd407386e8566fa323a55f]). Expected version 1 got 0
I20250115 03:06:11.695292 14167 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 496d3bd59f174412b7c26d9903972366 (table default.Table_b12aff7a3c8d41f79392773a151c463a [id=2c420d8e08fd407386e8566fa323a55f]). Expected version 1 got 0
I20250115 03:06:11.717813 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_075e1f04509641e989ddbef0880de5d8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:11.718542 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_075e1f04509641e989ddbef0880de5d8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.718621 14692 raft_consensus.cc:1270] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:11.718647 14422 raft_consensus.cc:1270] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:11.718657 14824 raft_consensus.cc:491] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.718736 14824 raft_consensus.cc:513] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.719128 14853 consensus_queue.cc:1035] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:11.719270 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f46b54e47cbe444b94710d45f918d270"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.719444 14853 consensus_queue.cc:1035] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:11.719568 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f46b54e47cbe444b94710d45f918d270 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.719734 14824 leader_election.cc:290] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:11.722002 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_17695f503db342d2975cc920bdcda1d7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.722206 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f6b447ccf7e4426484d99557dd1591d7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:11.722294 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_17695f503db342d2975cc920bdcda1d7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.722312 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0bdf66c6040b4709b0a6be57188b9d86"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:11.722440 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f6b447ccf7e4426484d99557dd1591d7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:11.722582 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0bdf66c6040b4709b0a6be57188b9d86 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.722954 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_4d3670427bb242829ca73ccc85e5a887"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:11.723206 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d3670427bb242829ca73ccc85e5a887 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.724181 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_1e37785047d64a1fb98662234aea76a5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:11.724447 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e37785047d64a1fb98662234aea76a5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.724789 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 180df492348a4cc09afa03c8a62a79c1 (table default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]). Expected version 1 got 0
I20250115 03:06:11.726712 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f32d3e059e44722af83c55b71f14c26" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:11.726823 14421 raft_consensus.cc:2463] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:11.728255 14628 leader_election.cc:304] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.728539 14824 raft_consensus.cc:2798] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:11.728595 14824 raft_consensus.cc:491] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.728631 14824 raft_consensus.cc:3054] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.729228 14893 tablet.cc:1721] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:11.729554 14824 raft_consensus.cc:513] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.729861 14824 leader_election.cc:290] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:11.730290 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f32d3e059e44722af83c55b71f14c26" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:11.730386 14421 raft_consensus.cc:3054] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.730645 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 180df492348a4cc09afa03c8a62a79c1 (table default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]). Expected version 1 got 0
I20250115 03:06:11.731280 14421 raft_consensus.cc:2463] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:11.731377 14163 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 180df492348a4cc09afa03c8a62a79c1 (table default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]). Expected version 1 got 0
I20250115 03:06:11.734292 14883 tablet.cc:1721] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:11.734850 14628 leader_election.cc:304] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.735181 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f32d3e059e44722af83c55b71f14c26" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:11.735286 14556 raft_consensus.cc:2463] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:11.735474 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f32d3e059e44722af83c55b71f14c26" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:11.735556 14557 raft_consensus.cc:3054] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.735994 14824 raft_consensus.cc:2798] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:11.736074 14824 raft_consensus.cc:695] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:11.736265 14824 consensus_queue.cc:237] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.736413 14557 raft_consensus.cc:2463] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:11.738803 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_194d150ada7243d58b5e8d31c3a15afc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:11.739238 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_194d150ada7243d58b5e8d31c3a15afc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.740661 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 70a6241a383c461ca646a7d33911d654 (DEFAULT_TABLE table=default.Table_075e1f04509641e989ddbef0880de5d8 [id=f56a0de6e56441bfa620902f175feb4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.740669 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 70a6241a383c461ca646a7d33911d654 (DEFAULT_TABLE table=default.Table_075e1f04509641e989ddbef0880de5d8 [id=f56a0de6e56441bfa620902f175feb4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.740887 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70a6241a383c461ca646a7d33911d654. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.740895 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70a6241a383c461ca646a7d33911d654. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.742817 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 70a6241a383c461ca646a7d33911d654 (DEFAULT_TABLE table=default.Table_075e1f04509641e989ddbef0880de5d8 [id=f56a0de6e56441bfa620902f175feb4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.742863 14812 tablet_bootstrap.cc:492] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.743026 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70a6241a383c461ca646a7d33911d654. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.743748 14812 tablet_bootstrap.cc:654] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.743861 14891 tablet.cc:1721] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:11.745272 14811 tablet_bootstrap.cc:492] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.746142 14692 raft_consensus.cc:1270] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:11.746310 14811 tablet_bootstrap.cc:654] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.746515 14853 consensus_queue.cc:1035] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:11.748160 14170 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157] alter complete (version 1)
I20250115 03:06:11.748224 14422 raft_consensus.cc:1270] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:11.748536 14165 catalog_manager.cc:5526] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:11.748728 14165 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 4f32d3e059e44722af83c55b71f14c26 (table default.Table_c1020680eef344a49998eddac921fea5 [id=649880f1ff0d4cfda2358b408110d8dc]). Expected version 1 got 0
I20250115 03:06:11.748800 14853 consensus_queue.cc:1035] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:11.749264 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d46d2affc6e74c7a8d3cc2a8ec0233e6" } new_table_name: "default.Table_d3b0da332ade44a98e9e45c9cb294874" modify_external_catalogs: true
I20250115 03:06:11.749332 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_db8eb767c1214ff89ccfb64f0ad123ae" } new_table_name: "default.Table_4fc6e70f174043999eccc3ec2b20474d" modify_external_catalogs: true
I20250115 03:06:11.749334 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 9f4f3df7129b4fb79c67de414e7bac88 (DEFAULT_TABLE table=default.Table_0bdf66c6040b4709b0a6be57188b9d86 [id=c8fdb0f9f9064f3eb8d19f4adf3c439f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.749536 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f4f3df7129b4fb79c67de414e7bac88. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.749971 14171 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_03a772b18f874e3fb42b5dc7327e4543" } new_table_name: "default.Table_a3860130ddd44dd1bf2eff20f30d374b" modify_external_catalogs: true
I20250115 03:06:11.750350 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 642e5f4889ec46adb023fdaa45e99773 (DEFAULT_TABLE table=default.Table_1e37785047d64a1fb98662234aea76a5 [id=f00e0e2286064402802dce72256e92b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.750568 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 642e5f4889ec46adb023fdaa45e99773. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.750748 14533 tablet_service.cc:1467] Processing CreateTablet for tablet aa88f604011741f2b3eb2699b9328eae (DEFAULT_TABLE table=default.Table_4d3670427bb242829ca73ccc85e5a887 [id=46f096547b3a4de9a46522b978ac4b8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.750955 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa88f604011741f2b3eb2699b9328eae. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.751694 14397 tablet_service.cc:1467] Processing CreateTablet for tablet aa88f604011741f2b3eb2699b9328eae (DEFAULT_TABLE table=default.Table_4d3670427bb242829ca73ccc85e5a887 [id=46f096547b3a4de9a46522b978ac4b8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.751902 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa88f604011741f2b3eb2699b9328eae. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.752521 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 9f4f3df7129b4fb79c67de414e7bac88 (DEFAULT_TABLE table=default.Table_0bdf66c6040b4709b0a6be57188b9d86 [id=c8fdb0f9f9064f3eb8d19f4adf3c439f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.752720 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f4f3df7129b4fb79c67de414e7bac88. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.749276 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f46b54e47cbe444b94710d45f918d270" } new_table_name: "default.Table_6098a167e4524987bbc24d8dfa47f5e4" modify_external_catalogs: true
I20250115 03:06:11.753266 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_6a0bb9a04eb04a809c416bbd3f662565"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:11.753513 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6a0bb9a04eb04a809c416bbd3f662565 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.754177 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 9f4f3df7129b4fb79c67de414e7bac88 (DEFAULT_TABLE table=default.Table_0bdf66c6040b4709b0a6be57188b9d86 [id=c8fdb0f9f9064f3eb8d19f4adf3c439f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.754372 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f4f3df7129b4fb79c67de414e7bac88. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.754350 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b12aff7a3c8d41f79392773a151c463a" } modify_external_catalogs: true
I20250115 03:06:11.754436 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b12aff7a3c8d41f79392773a151c463a" } modify_external_catalogs: true
I20250115 03:06:11.755690 14812 tablet_bootstrap.cc:492] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.755668 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4009ecd0a7ed495c99e5e4fd76c4774f" } new_table_name: "default.Table_8f3bf985bedd43918313ea7504253e48" modify_external_catalogs: true
I20250115 03:06:11.755772 14812 ts_tablet_manager.cc:1397] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.001s
I20250115 03:06:11.755753 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 642e5f4889ec46adb023fdaa45e99773 (DEFAULT_TABLE table=default.Table_1e37785047d64a1fb98662234aea76a5 [id=f00e0e2286064402802dce72256e92b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.755937 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 642e5f4889ec46adb023fdaa45e99773. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.756179 14812 raft_consensus.cc:357] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.756268 14812 raft_consensus.cc:383] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.756294 14812 raft_consensus.cc:738] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.756388 14812 consensus_queue.cc:260] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.756629 14812 ts_tablet_manager.cc:1428] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.757051 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 642e5f4889ec46adb023fdaa45e99773 (DEFAULT_TABLE table=default.Table_1e37785047d64a1fb98662234aea76a5 [id=f00e0e2286064402802dce72256e92b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.757267 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 642e5f4889ec46adb023fdaa45e99773. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.757419 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_77f0e54f01874d84ac9475bf9d819946"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:11.757681 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_77f0e54f01874d84ac9475bf9d819946 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.759370 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_1e37785047d64a1fb98662234aea76a5" } modify_external_catalogs: true
I20250115 03:06:11.759471 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_1e37785047d64a1fb98662234aea76a5" } modify_external_catalogs: true
W20250115 03:06:11.759732 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 35b90c4b0ef341c8a7c1bc517a1231ed consensus state has no leader
I20250115 03:06:11.760025 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_017cebf1769e47a997604ce80cd74d9b" } modify_external_catalogs: true
I20250115 03:06:11.760108 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_017cebf1769e47a997604ce80cd74d9b" } modify_external_catalogs: true
I20250115 03:06:11.760550 14812 tablet_bootstrap.cc:492] T 9f4f3df7129b4fb79c67de414e7bac88 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.761422 14812 tablet_bootstrap.cc:654] T 9f4f3df7129b4fb79c67de414e7bac88 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.762015 14809 tablet_bootstrap.cc:492] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.762961 14809 tablet_bootstrap.cc:654] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.763250 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 9a863928b49443329dca51c60cc7fae0 (DEFAULT_TABLE table=default.Table_17695f503db342d2975cc920bdcda1d7 [id=060ecaa51ce14dffa42d21f7c72859aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.763269 14671 tablet_service.cc:1467] Processing CreateTablet for tablet 026bf50b4721424aa06c59455fe7d07e (DEFAULT_TABLE table=default.Table_f46b54e47cbe444b94710d45f918d270 [id=eb34ed3c30694c39bf957a6af751a9b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.763447 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a863928b49443329dca51c60cc7fae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.763509 14669 tablet_service.cc:1467] Processing CreateTablet for tablet afa7cecc384f495c9ff4723f553acc24 (DEFAULT_TABLE table=default.Table_f6b447ccf7e4426484d99557dd1591d7 [id=f4597c64de65413a9885be2bb4301d44]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.763931 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afa7cecc384f495c9ff4723f553acc24. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.764328 14672 tablet_service.cc:1467] Processing CreateTablet for tablet aa88f604011741f2b3eb2699b9328eae (DEFAULT_TABLE table=default.Table_4d3670427bb242829ca73ccc85e5a887 [id=46f096547b3a4de9a46522b978ac4b8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.764521 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa88f604011741f2b3eb2699b9328eae. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.765493 14811 tablet_bootstrap.cc:492] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.765565 14811 ts_tablet_manager.cc:1397] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.001s
I20250115 03:06:11.765760 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 026bf50b4721424aa06c59455fe7d07e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.765965 14811 raft_consensus.cc:357] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.766065 14811 raft_consensus.cc:383] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.766105 14811 raft_consensus.cc:738] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.766224 14811 consensus_queue.cc:260] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.766484 14811 ts_tablet_manager.cc:1428] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.766516 14812 tablet_bootstrap.cc:492] T 9f4f3df7129b4fb79c67de414e7bac88 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.766624 14812 ts_tablet_manager.cc:1397] T 9f4f3df7129b4fb79c67de414e7bac88 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250115 03:06:11.766636 14811 tablet_bootstrap.cc:492] T 642e5f4889ec46adb023fdaa45e99773 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.767025 14812 raft_consensus.cc:357] T 9f4f3df7129b4fb79c67de414e7bac88 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.767143 14812 raft_consensus.cc:383] T 9f4f3df7129b4fb79c67de414e7bac88 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.767194 14812 raft_consensus.cc:738] T 9f4f3df7129b4fb79c67de414e7bac88 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.767479 14811 tablet_bootstrap.cc:654] T 642e5f4889ec46adb023fdaa45e99773 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.767427 14812 consensus_queue.cc:260] T 9f4f3df7129b4fb79c67de414e7bac88 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.767757 14812 ts_tablet_manager.cc:1428] T 9f4f3df7129b4fb79c67de414e7bac88 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.768056 14812 tablet_bootstrap.cc:492] T 642e5f4889ec46adb023fdaa45e99773 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.768065 14891 tablet.cc:1721] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:11.768975 14812 tablet_bootstrap.cc:654] T 642e5f4889ec46adb023fdaa45e99773 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.769227 14809 tablet_bootstrap.cc:492] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.769296 14809 ts_tablet_manager.cc:1397] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20250115 03:06:11.769716 14809 raft_consensus.cc:357] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.769755 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 026bf50b4721424aa06c59455fe7d07e (DEFAULT_TABLE table=default.Table_f46b54e47cbe444b94710d45f918d270 [id=eb34ed3c30694c39bf957a6af751a9b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.769814 14809 raft_consensus.cc:383] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.769845 14809 raft_consensus.cc:738] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.769912 14812 tablet_bootstrap.cc:492] T 642e5f4889ec46adb023fdaa45e99773 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.769939 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 026bf50b4721424aa06c59455fe7d07e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.769974 14812 ts_tablet_manager.cc:1397] T 642e5f4889ec46adb023fdaa45e99773 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:11.770355 14812 raft_consensus.cc:357] T 642e5f4889ec46adb023fdaa45e99773 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.770438 14812 raft_consensus.cc:383] T 642e5f4889ec46adb023fdaa45e99773 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.770468 14812 raft_consensus.cc:738] T 642e5f4889ec46adb023fdaa45e99773 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.770555 14812 consensus_queue.cc:260] T 642e5f4889ec46adb023fdaa45e99773 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.770676 14809 consensus_queue.cc:260] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.770792 14812 ts_tablet_manager.cc:1428] T 642e5f4889ec46adb023fdaa45e99773 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.770928 14812 tablet_bootstrap.cc:492] T 9a863928b49443329dca51c60cc7fae0 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.771792 14812 tablet_bootstrap.cc:654] T 9a863928b49443329dca51c60cc7fae0 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.771803 14397 tablet_service.cc:1467] Processing CreateTablet for tablet afa7cecc384f495c9ff4723f553acc24 (DEFAULT_TABLE table=default.Table_f6b447ccf7e4426484d99557dd1591d7 [id=f4597c64de65413a9885be2bb4301d44]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.771979 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afa7cecc384f495c9ff4723f553acc24. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.772780 14809 ts_tablet_manager.cc:1428] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:11.773726 14809 tablet_bootstrap.cc:492] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.773957 14893 tablet.cc:1721] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:11.774559 14809 tablet_bootstrap.cc:654] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.775377 14883 tablet.cc:1721] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:11.775983 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 9a863928b49443329dca51c60cc7fae0 (DEFAULT_TABLE table=default.Table_17695f503db342d2975cc920bdcda1d7 [id=060ecaa51ce14dffa42d21f7c72859aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.776178 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a863928b49443329dca51c60cc7fae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.776382 14535 tablet_service.cc:1467] Processing CreateTablet for tablet afa7cecc384f495c9ff4723f553acc24 (DEFAULT_TABLE table=default.Table_f6b447ccf7e4426484d99557dd1591d7 [id=f4597c64de65413a9885be2bb4301d44]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.776445 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 026bf50b4721424aa06c59455fe7d07e (DEFAULT_TABLE table=default.Table_f46b54e47cbe444b94710d45f918d270 [id=eb34ed3c30694c39bf957a6af751a9b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.776583 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afa7cecc384f495c9ff4723f553acc24. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.776791 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 026bf50b4721424aa06c59455fe7d07e. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:11.779107 14163 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3c19a41bd4d149bebdd81cba3fa87d01 consensus state has no leader
W20250115 03:06:11.779129 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 026bf50b4721424aa06c59455fe7d07e consensus state has no leader
W20250115 03:06:11.779536 14171 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f64a668e2f4548f3bd4f2b0be32e8682 consensus state has no leader
I20250115 03:06:11.780532 14811 tablet_bootstrap.cc:492] T 642e5f4889ec46adb023fdaa45e99773 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.780607 14811 ts_tablet_manager.cc:1397] T 642e5f4889ec46adb023fdaa45e99773 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.001s
I20250115 03:06:11.780660 14812 tablet_bootstrap.cc:492] T 9a863928b49443329dca51c60cc7fae0 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.780741 14812 ts_tablet_manager.cc:1397] T 9a863928b49443329dca51c60cc7fae0 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20250115 03:06:11.781140 14811 raft_consensus.cc:357] T 642e5f4889ec46adb023fdaa45e99773 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.781145 14812 raft_consensus.cc:357] T 9a863928b49443329dca51c60cc7fae0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.781231 14812 raft_consensus.cc:383] T 9a863928b49443329dca51c60cc7fae0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.781232 14811 raft_consensus.cc:383] T 642e5f4889ec46adb023fdaa45e99773 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.781262 14812 raft_consensus.cc:738] T 9a863928b49443329dca51c60cc7fae0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.781267 14811 raft_consensus.cc:738] T 642e5f4889ec46adb023fdaa45e99773 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.781363 14812 consensus_queue.cc:260] T 9a863928b49443329dca51c60cc7fae0 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.781371 14811 consensus_queue.cc:260] T 642e5f4889ec46adb023fdaa45e99773 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.781597 14812 ts_tablet_manager.cc:1428] T 9a863928b49443329dca51c60cc7fae0 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.781618 14811 ts_tablet_manager.cc:1428] T 642e5f4889ec46adb023fdaa45e99773 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.781754 14812 tablet_bootstrap.cc:492] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.781772 14811 tablet_bootstrap.cc:492] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.782689 14812 tablet_bootstrap.cc:654] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.782864 14811 tablet_bootstrap.cc:654] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.783963 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_17695f503db342d2975cc920bdcda1d7" } modify_external_catalogs: true
I20250115 03:06:11.784020 14811 tablet_bootstrap.cc:492] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.784061 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_17695f503db342d2975cc920bdcda1d7" } modify_external_catalogs: true
I20250115 03:06:11.784085 14811 ts_tablet_manager.cc:1397] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:11.784210 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 4379cf2eb12844a29c5f2e36d1e1dffc (table default.Table_4009ecd0a7ed495c99e5e4fd76c4774f [id=05f9fd2ebec34d2299ea59c47a74c828]). Expected version 1 got 0
I20250115 03:06:11.784467 14811 raft_consensus.cc:357] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.784551 14811 raft_consensus.cc:383] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.784581 14811 raft_consensus.cc:738] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.784703 14811 consensus_queue.cc:260] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.784951 14811 ts_tablet_manager.cc:1428] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.785058 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 4379cf2eb12844a29c5f2e36d1e1dffc (table default.Table_4009ecd0a7ed495c99e5e4fd76c4774f [id=05f9fd2ebec34d2299ea59c47a74c828]). Expected version 1 got 0
I20250115 03:06:11.785097 14811 tablet_bootstrap.cc:492] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.785920 14811 tablet_bootstrap.cc:654] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.787441 14812 tablet_bootstrap.cc:492] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.787516 14812 ts_tablet_manager.cc:1397] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250115 03:06:11.787902 14812 raft_consensus.cc:357] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.787990 14812 raft_consensus.cc:383] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.788019 14812 raft_consensus.cc:738] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.788125 14812 consensus_queue.cc:260] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.788350 14812 ts_tablet_manager.cc:1428] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.788491 14812 tablet_bootstrap.cc:492] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.789561 14812 tablet_bootstrap.cc:654] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.790536 14811 tablet_bootstrap.cc:492] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.790629 14811 ts_tablet_manager.cc:1397] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250115 03:06:11.791018 14811 raft_consensus.cc:357] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.791100 14811 raft_consensus.cc:383] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.791129 14811 raft_consensus.cc:738] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.791226 14811 consensus_queue.cc:260] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.791496 14811 ts_tablet_manager.cc:1428] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.791492 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 496d3bd59f174412b7c26d9903972366
I20250115 03:06:11.791639 14811 tablet_bootstrap.cc:492] T afa7cecc384f495c9ff4723f553acc24 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.792439 14811 tablet_bootstrap.cc:654] T afa7cecc384f495c9ff4723f553acc24 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.793064 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 496d3bd59f174412b7c26d9903972366 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.793207 14820 tablet_replica.cc:331] T 496d3bd59f174412b7c26d9903972366 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:11.793293 14820 raft_consensus.cc:2238] T 496d3bd59f174412b7c26d9903972366 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.793366 14820 raft_consensus.cc:2267] T 496d3bd59f174412b7c26d9903972366 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.793737 14820 ts_tablet_manager.cc:1905] T 496d3bd59f174412b7c26d9903972366 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.794538 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 9a863928b49443329dca51c60cc7fae0 (DEFAULT_TABLE table=default.Table_17695f503db342d2975cc920bdcda1d7 [id=060ecaa51ce14dffa42d21f7c72859aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.794739 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a863928b49443329dca51c60cc7fae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.795337 14812 tablet_bootstrap.cc:492] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.795401 14812 ts_tablet_manager.cc:1397] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20250115 03:06:11.795809 14812 raft_consensus.cc:357] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.795884 14812 raft_consensus.cc:383] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.795908 14812 raft_consensus.cc:738] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.796003 14812 consensus_queue.cc:260] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.796154 14820 ts_tablet_manager.cc:1918] T 496d3bd59f174412b7c26d9903972366 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.796205 14820 log.cc:1198] T 496d3bd59f174412b7c26d9903972366 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/496d3bd59f174412b7c26d9903972366
I20250115 03:06:11.796218 14812 ts_tablet_manager.cc:1428] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:11.796375 14820 ts_tablet_manager.cc:1939] T 496d3bd59f174412b7c26d9903972366 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:11.796375 14812 tablet_bootstrap.cc:492] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.797225 14812 tablet_bootstrap.cc:654] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.798713 14811 tablet_bootstrap.cc:492] T afa7cecc384f495c9ff4723f553acc24 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.798718 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 496d3bd59f174412b7c26d9903972366 (table default.Table_b12aff7a3c8d41f79392773a151c463a [id=2c420d8e08fd407386e8566fa323a55f]) successfully deleted
I20250115 03:06:11.798789 14811 ts_tablet_manager.cc:1397] T afa7cecc384f495c9ff4723f553acc24 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:11.799130 14811 raft_consensus.cc:357] T afa7cecc384f495c9ff4723f553acc24 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.799211 14811 raft_consensus.cc:383] T afa7cecc384f495c9ff4723f553acc24 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.799237 14811 raft_consensus.cc:738] T afa7cecc384f495c9ff4723f553acc24 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.799330 14811 consensus_queue.cc:260] T afa7cecc384f495c9ff4723f553acc24 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.799561 14811 ts_tablet_manager.cc:1428] T afa7cecc384f495c9ff4723f553acc24 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.799706 14811 tablet_bootstrap.cc:492] T 026bf50b4721424aa06c59455fe7d07e P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.800598 14811 tablet_bootstrap.cc:654] T 026bf50b4721424aa06c59455fe7d07e P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.802640 14812 tablet_bootstrap.cc:492] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.802726 14812 ts_tablet_manager.cc:1397] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:11.803066 14812 raft_consensus.cc:357] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.803153 14812 raft_consensus.cc:383] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.803179 14812 raft_consensus.cc:738] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.803472 14812 consensus_queue.cc:260] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.803550 14671 tablet_service.cc:1514] Processing DeleteTablet for tablet 496d3bd59f174412b7c26d9903972366 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:11.803678 14823 tablet_replica.cc:331] T 496d3bd59f174412b7c26d9903972366 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:11.803720 14812 ts_tablet_manager.cc:1428] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.803766 14823 raft_consensus.cc:2238] T 496d3bd59f174412b7c26d9903972366 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.803834 14823 raft_consensus.cc:2267] T 496d3bd59f174412b7c26d9903972366 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.804172 14823 ts_tablet_manager.cc:1905] T 496d3bd59f174412b7c26d9903972366 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.804482 14811 tablet_bootstrap.cc:492] T 026bf50b4721424aa06c59455fe7d07e P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.804548 14811 ts_tablet_manager.cc:1397] T 026bf50b4721424aa06c59455fe7d07e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:11.804981 14811 raft_consensus.cc:357] T 026bf50b4721424aa06c59455fe7d07e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.805068 14811 raft_consensus.cc:383] T 026bf50b4721424aa06c59455fe7d07e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.805097 14811 raft_consensus.cc:738] T 026bf50b4721424aa06c59455fe7d07e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.805193 14811 consensus_queue.cc:260] T 026bf50b4721424aa06c59455fe7d07e P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.805418 14811 ts_tablet_manager.cc:1428] T 026bf50b4721424aa06c59455fe7d07e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.805572 14811 tablet_bootstrap.cc:492] T 9a863928b49443329dca51c60cc7fae0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.805909 14823 ts_tablet_manager.cc:1918] T 496d3bd59f174412b7c26d9903972366 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.805961 14823 log.cc:1198] T 496d3bd59f174412b7c26d9903972366 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/496d3bd59f174412b7c26d9903972366
I20250115 03:06:11.806120 14823 ts_tablet_manager.cc:1939] T 496d3bd59f174412b7c26d9903972366 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:11.806414 14811 tablet_bootstrap.cc:654] T 9a863928b49443329dca51c60cc7fae0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.806638 14883 tablet.cc:1721] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:11.806668 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_194d150ada7243d58b5e8d31c3a15afc" } modify_external_catalogs: true
I20250115 03:06:11.806756 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_194d150ada7243d58b5e8d31c3a15afc" } modify_external_catalogs: true
I20250115 03:06:11.807334 14809 tablet_bootstrap.cc:492] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.807410 14809 ts_tablet_manager.cc:1397] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.000s
I20250115 03:06:11.807816 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 496d3bd59f174412b7c26d9903972366 (table default.Table_b12aff7a3c8d41f79392773a151c463a [id=2c420d8e08fd407386e8566fa323a55f]) successfully deleted
I20250115 03:06:11.807821 14809 raft_consensus.cc:357] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.807910 14809 raft_consensus.cc:383] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.807941 14809 raft_consensus.cc:738] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.808058 14809 consensus_queue.cc:260] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.808326 14809 ts_tablet_manager.cc:1428] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.808471 14809 tablet_bootstrap.cc:492] T 9f4f3df7129b4fb79c67de414e7bac88 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.809326 14809 tablet_bootstrap.cc:654] T 9f4f3df7129b4fb79c67de414e7bac88 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.810108 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 496d3bd59f174412b7c26d9903972366 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.810276 14819 tablet_replica.cc:331] T 496d3bd59f174412b7c26d9903972366 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:11.810364 14819 raft_consensus.cc:2238] T 496d3bd59f174412b7c26d9903972366 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.810431 14819 raft_consensus.cc:2267] T 496d3bd59f174412b7c26d9903972366 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.810793 14819 ts_tablet_manager.cc:1905] T 496d3bd59f174412b7c26d9903972366 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.810930 14809 tablet_bootstrap.cc:492] T 9f4f3df7129b4fb79c67de414e7bac88 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.810997 14809 ts_tablet_manager.cc:1397] T 9f4f3df7129b4fb79c67de414e7bac88 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20250115 03:06:11.811394 14809 raft_consensus.cc:357] T 9f4f3df7129b4fb79c67de414e7bac88 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.811477 14809 raft_consensus.cc:383] T 9f4f3df7129b4fb79c67de414e7bac88 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.811506 14809 raft_consensus.cc:738] T 9f4f3df7129b4fb79c67de414e7bac88 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.811623 14809 consensus_queue.cc:260] T 9f4f3df7129b4fb79c67de414e7bac88 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.811857 14809 ts_tablet_manager.cc:1428] T 9f4f3df7129b4fb79c67de414e7bac88 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:11.812008 14809 tablet_bootstrap.cc:492] T 642e5f4889ec46adb023fdaa45e99773 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.812479 14819 ts_tablet_manager.cc:1918] T 496d3bd59f174412b7c26d9903972366 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.812530 14819 log.cc:1198] T 496d3bd59f174412b7c26d9903972366 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/496d3bd59f174412b7c26d9903972366
I20250115 03:06:11.812693 14819 ts_tablet_manager.cc:1939] T 496d3bd59f174412b7c26d9903972366 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:11.812875 14809 tablet_bootstrap.cc:654] T 642e5f4889ec46adb023fdaa45e99773 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.815825 14893 tablet.cc:1721] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:11.816205 14809 tablet_bootstrap.cc:492] T 642e5f4889ec46adb023fdaa45e99773 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.816283 14809 ts_tablet_manager.cc:1397] T 642e5f4889ec46adb023fdaa45e99773 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:11.816650 14809 raft_consensus.cc:357] T 642e5f4889ec46adb023fdaa45e99773 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.816754 14809 raft_consensus.cc:383] T 642e5f4889ec46adb023fdaa45e99773 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.816789 14809 raft_consensus.cc:738] T 642e5f4889ec46adb023fdaa45e99773 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.816896 14809 consensus_queue.cc:260] T 642e5f4889ec46adb023fdaa45e99773 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.817147 14809 ts_tablet_manager.cc:1428] T 642e5f4889ec46adb023fdaa45e99773 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.817309 14809 tablet_bootstrap.cc:492] T 026bf50b4721424aa06c59455fe7d07e P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.817816 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet 4379cf2eb12844a29c5f2e36d1e1dffc from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:11.818120 14809 tablet_bootstrap.cc:654] T 026bf50b4721424aa06c59455fe7d07e P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.819201 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 496d3bd59f174412b7c26d9903972366 (table default.Table_b12aff7a3c8d41f79392773a151c463a [id=2c420d8e08fd407386e8566fa323a55f]) successfully deleted
I20250115 03:06:11.820971 14809 tablet_bootstrap.cc:492] T 026bf50b4721424aa06c59455fe7d07e P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.821048 14809 ts_tablet_manager.cc:1397] T 026bf50b4721424aa06c59455fe7d07e P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250115 03:06:11.821472 14809 raft_consensus.cc:357] T 026bf50b4721424aa06c59455fe7d07e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.821560 14809 raft_consensus.cc:383] T 026bf50b4721424aa06c59455fe7d07e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.821591 14809 raft_consensus.cc:738] T 026bf50b4721424aa06c59455fe7d07e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.821697 14809 consensus_queue.cc:260] T 026bf50b4721424aa06c59455fe7d07e P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.821738 14891 tablet.cc:1721] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:11.821955 14809 ts_tablet_manager.cc:1428] T 026bf50b4721424aa06c59455fe7d07e P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.822108 14809 tablet_bootstrap.cc:492] T afa7cecc384f495c9ff4723f553acc24 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.822098 14165 master_service.cc:946] ReplaceTablet: received request to replace tablet 4379cf2eb12844a29c5f2e36d1e1dffc from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:11.822892 14809 tablet_bootstrap.cc:654] T afa7cecc384f495c9ff4723f553acc24 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.823874 14811 tablet_bootstrap.cc:492] T 9a863928b49443329dca51c60cc7fae0 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.823951 14811 ts_tablet_manager.cc:1397] T 9a863928b49443329dca51c60cc7fae0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20250115 03:06:11.824097 14809 tablet_bootstrap.cc:492] T afa7cecc384f495c9ff4723f553acc24 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.824159 14809 ts_tablet_manager.cc:1397] T afa7cecc384f495c9ff4723f553acc24 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:11.824313 14811 raft_consensus.cc:357] T 9a863928b49443329dca51c60cc7fae0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.824397 14811 raft_consensus.cc:383] T 9a863928b49443329dca51c60cc7fae0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.824426 14811 raft_consensus.cc:738] T 9a863928b49443329dca51c60cc7fae0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.824539 14811 consensus_queue.cc:260] T 9a863928b49443329dca51c60cc7fae0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.824548 14809 raft_consensus.cc:357] T afa7cecc384f495c9ff4723f553acc24 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.824620 14809 raft_consensus.cc:383] T afa7cecc384f495c9ff4723f553acc24 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.824647 14809 raft_consensus.cc:738] T afa7cecc384f495c9ff4723f553acc24 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.824749 14809 consensus_queue.cc:260] T afa7cecc384f495c9ff4723f553acc24 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.824810 14811 ts_tablet_manager.cc:1428] T 9a863928b49443329dca51c60cc7fae0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.825078 14809 ts_tablet_manager.cc:1428] T afa7cecc384f495c9ff4723f553acc24 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.825228 14809 tablet_bootstrap.cc:492] T 9a863928b49443329dca51c60cc7fae0 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.826145 14809 tablet_bootstrap.cc:654] T 9a863928b49443329dca51c60cc7fae0 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.827977 14809 tablet_bootstrap.cc:492] T 9a863928b49443329dca51c60cc7fae0 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.828052 14809 ts_tablet_manager.cc:1397] T 9a863928b49443329dca51c60cc7fae0 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:11.828517 14809 raft_consensus.cc:357] T 9a863928b49443329dca51c60cc7fae0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.828717 14809 raft_consensus.cc:383] T 9a863928b49443329dca51c60cc7fae0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.828760 14809 raft_consensus.cc:738] T 9a863928b49443329dca51c60cc7fae0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.828881 14809 consensus_queue.cc:260] T 9a863928b49443329dca51c60cc7fae0 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.829649 14809 ts_tablet_manager.cc:1428] T 9a863928b49443329dca51c60cc7fae0 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:11.831226 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 4379cf2eb12844a29c5f2e36d1e1dffc (table default.Table_8f3bf985bedd43918313ea7504253e48 [id=05f9fd2ebec34d2299ea59c47a74c828]). Expected version 2 got 1
I20250115 03:06:11.832257 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 35df9fe895414d4e87c7923da9703e59 (DEFAULT_TABLE table=default.Table_194d150ada7243d58b5e8d31c3a15afc [id=3d0789d7623c43a58d33e9ede070f7c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.832495 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35df9fe895414d4e87c7923da9703e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.832515 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 35df9fe895414d4e87c7923da9703e59 (DEFAULT_TABLE table=default.Table_194d150ada7243d58b5e8d31c3a15afc [id=3d0789d7623c43a58d33e9ede070f7c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.832736 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35df9fe895414d4e87c7923da9703e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.833284 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet 4379cf2eb12844a29c5f2e36d1e1dffc from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:11.833905 14671 tablet_service.cc:1467] Processing CreateTablet for tablet 35df9fe895414d4e87c7923da9703e59 (DEFAULT_TABLE table=default.Table_194d150ada7243d58b5e8d31c3a15afc [id=3d0789d7623c43a58d33e9ede070f7c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.834124 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35df9fe895414d4e87c7923da9703e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.835512 14809 tablet_bootstrap.cc:492] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.835918 14811 tablet_bootstrap.cc:492] T 35df9fe895414d4e87c7923da9703e59 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.835984 14812 tablet_bootstrap.cc:492] T 35df9fe895414d4e87c7923da9703e59 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.836733 14811 tablet_bootstrap.cc:654] T 35df9fe895414d4e87c7923da9703e59 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.836861 14812 tablet_bootstrap.cc:654] T 35df9fe895414d4e87c7923da9703e59 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.837175 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 35df9fe895414d4e87c7923da9703e59
I20250115 03:06:11.837769 14671 tablet_service.cc:1514] Processing DeleteTablet for tablet 35df9fe895414d4e87c7923da9703e59 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:11.838162 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 35df9fe895414d4e87c7923da9703e59 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 35df9fe895414d4e87c7923da9703e59 already in progress: creating tablet
I20250115 03:06:11.838560 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 35df9fe895414d4e87c7923da9703e59 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.838603 14812 tablet_bootstrap.cc:492] T 35df9fe895414d4e87c7923da9703e59 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.838673 14812 ts_tablet_manager.cc:1397] T 35df9fe895414d4e87c7923da9703e59 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:11.838698 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 35df9fe895414d4e87c7923da9703e59 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.838866 14162 master_service.cc:946] ReplaceTablet: received request to replace tablet 9f4f3df7129b4fb79c67de414e7bac88 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:11.839067 14812 raft_consensus.cc:357] T 35df9fe895414d4e87c7923da9703e59 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:11.839116 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 35df9fe895414d4e87c7923da9703e59 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 35df9fe895414d4e87c7923da9703e59 already in progress: creating tablet
I20250115 03:06:11.839175 14812 raft_consensus.cc:383] T 35df9fe895414d4e87c7923da9703e59 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.839213 14812 raft_consensus.cc:738] T 35df9fe895414d4e87c7923da9703e59 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.839314 14812 consensus_queue.cc:260] T 35df9fe895414d4e87c7923da9703e59 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.839577 14812 ts_tablet_manager.cc:1428] T 35df9fe895414d4e87c7923da9703e59 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20250115 03:06:11.840166 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 35df9fe895414d4e87c7923da9703e59 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 35df9fe895414d4e87c7923da9703e59 already in progress: creating tablet
I20250115 03:06:11.840435 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 4379cf2eb12844a29c5f2e36d1e1dffc
I20250115 03:06:11.840579 14811 tablet_bootstrap.cc:492] T 35df9fe895414d4e87c7923da9703e59 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.840644 14811 ts_tablet_manager.cc:1397] T 35df9fe895414d4e87c7923da9703e59 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20250115 03:06:11.840700 14164 catalog_manager.cc:6596] ReplaceTablet: tablet 4379cf2eb12844a29c5f2e36d1e1dffc deleted and replaced by tablet c6cd63ac1c084a0d8f166e63b0ed85bc
I20250115 03:06:11.840811 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet cc88653cdfec4120b37097b361f441d4 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:11.841079 14811 raft_consensus.cc:357] T 35df9fe895414d4e87c7923da9703e59 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.841166 14811 raft_consensus.cc:383] T 35df9fe895414d4e87c7923da9703e59 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.841195 14811 raft_consensus.cc:738] T 35df9fe895414d4e87c7923da9703e59 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.841289 14811 consensus_queue.cc:260] T 35df9fe895414d4e87c7923da9703e59 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.841524 14811 ts_tablet_manager.cc:1428] T 35df9fe895414d4e87c7923da9703e59 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.841747 14671 tablet_service.cc:1514] Processing DeleteTablet for tablet 4379cf2eb12844a29c5f2e36d1e1dffc with delete_type TABLET_DATA_DELETED (replaced by tablet c6cd63ac1c084a0d8f166e63b0ed85bc) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:11.841872 14823 tablet_replica.cc:331] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:11.841974 14823 raft_consensus.cc:2238] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.842057 14823 raft_consensus.cc:2267] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.842371 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 4379cf2eb12844a29c5f2e36d1e1dffc with delete_type TABLET_DATA_DELETED (replaced by tablet c6cd63ac1c084a0d8f166e63b0ed85bc) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.842509 14820 tablet_replica.cc:331] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:11.842588 14820 raft_consensus.cc:2238] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:11.842770 14820 raft_consensus.cc:2267] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.842772 14823 ts_tablet_manager.cc:1905] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.842888 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 4379cf2eb12844a29c5f2e36d1e1dffc with delete_type TABLET_DATA_DELETED (replaced by tablet c6cd63ac1c084a0d8f166e63b0ed85bc) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.843020 14819 tablet_replica.cc:331] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:11.843075 14809 tablet_bootstrap.cc:654] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.843106 14819 raft_consensus.cc:2238] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.843183 14819 raft_consensus.cc:2267] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.843891 14820 ts_tablet_manager.cc:1905] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.843942 14809 tablet_bootstrap.cc:492] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.844003 14809 ts_tablet_manager.cc:1397] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20250115 03:06:11.844373 14809 raft_consensus.cc:357] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.844465 14809 raft_consensus.cc:383] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.844494 14809 raft_consensus.cc:738] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.844583 14823 ts_tablet_manager.cc:1918] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:11.844590 14809 consensus_queue.cc:260] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.844638 14823 log.cc:1198] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/4379cf2eb12844a29c5f2e36d1e1dffc
I20250115 03:06:11.844825 14809 ts_tablet_manager.cc:1428] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.844873 14823 ts_tablet_manager.cc:1939] T 4379cf2eb12844a29c5f2e36d1e1dffc P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:11.845722 14820 ts_tablet_manager.cc:1918] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:11.845779 14820 log.cc:1198] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/4379cf2eb12844a29c5f2e36d1e1dffc
I20250115 03:06:11.845986 14820 ts_tablet_manager.cc:1939] T 4379cf2eb12844a29c5f2e36d1e1dffc P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:11.846096 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 180df492348a4cc09afa03c8a62a79c1
I20250115 03:06:11.846354 14819 ts_tablet_manager.cc:1905] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.846415 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 642e5f4889ec46adb023fdaa45e99773
I20250115 03:06:11.847995 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 4379cf2eb12844a29c5f2e36d1e1dffc (table default.Table_8f3bf985bedd43918313ea7504253e48 [id=05f9fd2ebec34d2299ea59c47a74c828]) successfully deleted
I20250115 03:06:11.848171 14819 ts_tablet_manager.cc:1918] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:11.848228 14819 log.cc:1198] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/4379cf2eb12844a29c5f2e36d1e1dffc
I20250115 03:06:11.848438 14819 ts_tablet_manager.cc:1939] T 4379cf2eb12844a29c5f2e36d1e1dffc P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:11.848503 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 46e799d700db4803b79ed46273b99df6 (DEFAULT_TABLE table=default.Table_77f0e54f01874d84ac9475bf9d819946 [id=f5e54ac99a344e05a0be4c913472dd37]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.848555 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 642e5f4889ec46adb023fdaa45e99773 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.848762 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 9a863928b49443329dca51c60cc7fae0
I20250115 03:06:11.849232 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46e799d700db4803b79ed46273b99df6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.849447 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet 180df492348a4cc09afa03c8a62a79c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.849606 14819 tablet_replica.cc:331] T 642e5f4889ec46adb023fdaa45e99773 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:11.849643 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet 642e5f4889ec46adb023fdaa45e99773 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.849713 14819 raft_consensus.cc:2238] T 642e5f4889ec46adb023fdaa45e99773 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.849789 14819 raft_consensus.cc:2267] T 642e5f4889ec46adb023fdaa45e99773 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.849903 14167 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_8f3bf985bedd43918313ea7504253e48 [id=05f9fd2ebec34d2299ea59c47a74c828] alter complete (version 2)
I20250115 03:06:11.850154 14819 ts_tablet_manager.cc:1905] T 642e5f4889ec46adb023fdaa45e99773 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.850903 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 12a06b7730eb40fa8e73722cf75e9dce (DEFAULT_TABLE table=default.Table_6a0bb9a04eb04a809c416bbd3f662565 [id=cf66734746ce4d72a20b286b824f69a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.851094 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12a06b7730eb40fa8e73722cf75e9dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.851107 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 46e799d700db4803b79ed46273b99df6 (DEFAULT_TABLE table=default.Table_77f0e54f01874d84ac9475bf9d819946 [id=f5e54ac99a344e05a0be4c913472dd37]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.851329 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46e799d700db4803b79ed46273b99df6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.851620 14670 tablet_service.cc:1514] Processing DeleteTablet for tablet 642e5f4889ec46adb023fdaa45e99773 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:11.851768 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a863928b49443329dca51c60cc7fae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:11.851981 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 12a06b7730eb40fa8e73722cf75e9dce (DEFAULT_TABLE table=default.Table_6a0bb9a04eb04a809c416bbd3f662565 [id=cf66734746ce4d72a20b286b824f69a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.852205 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12a06b7730eb40fa8e73722cf75e9dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.851994 14809 tablet_bootstrap.cc:492] T 46e799d700db4803b79ed46273b99df6 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.853106 14669 tablet_service.cc:1514] Processing DeleteTablet for tablet 180df492348a4cc09afa03c8a62a79c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:11.853364 14671 tablet_service.cc:1467] Processing CreateTablet for tablet 12a06b7730eb40fa8e73722cf75e9dce (DEFAULT_TABLE table=default.Table_6a0bb9a04eb04a809c416bbd3f662565 [id=cf66734746ce4d72a20b286b824f69a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.853505 14809 tablet_bootstrap.cc:654] T 46e799d700db4803b79ed46273b99df6 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.853571 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12a06b7730eb40fa8e73722cf75e9dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.853892 14811 tablet_bootstrap.cc:492] T 12a06b7730eb40fa8e73722cf75e9dce P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.854491 14820 tablet_replica.cc:331] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:11.854602 14820 raft_consensus.cc:2238] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:11.854694 14819 ts_tablet_manager.cc:1918] T 642e5f4889ec46adb023fdaa45e99773 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.854733 14811 tablet_bootstrap.cc:654] T 12a06b7730eb40fa8e73722cf75e9dce P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.854748 14819 log.cc:1198] T 642e5f4889ec46adb023fdaa45e99773 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/642e5f4889ec46adb023fdaa45e99773
I20250115 03:06:11.854771 14820 raft_consensus.cc:2267] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.854856 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 4379cf2eb12844a29c5f2e36d1e1dffc (table default.Table_8f3bf985bedd43918313ea7504253e48 [id=05f9fd2ebec34d2299ea59c47a74c828]) successfully deleted
I20250115 03:06:11.854923 14819 ts_tablet_manager.cc:1939] T 642e5f4889ec46adb023fdaa45e99773 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:11.855311 14824 raft_consensus.cc:491] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.855388 14824 raft_consensus.cc:513] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:11.855603 14156 catalog_manager.cc:4945] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): alter failed for tablet 4379cf2eb12844a29c5f2e36d1e1dffc (table default.Table_8f3bf985bedd43918313ea7504253e48 [id=05f9fd2ebec34d2299ea59c47a74c828]),no further retry: Not found: Tablet not found: 4379cf2eb12844a29c5f2e36d1e1dffc
I20250115 03:06:11.855738 14824 leader_election.cc:290] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:11.855787 14809 tablet_bootstrap.cc:492] T 46e799d700db4803b79ed46273b99df6 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.855852 14809 ts_tablet_manager.cc:1397] T 46e799d700db4803b79ed46273b99df6 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:11.856225 14809 raft_consensus.cc:357] T 46e799d700db4803b79ed46273b99df6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.856312 14809 raft_consensus.cc:383] T 46e799d700db4803b79ed46273b99df6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.856343 14809 raft_consensus.cc:738] T 46e799d700db4803b79ed46273b99df6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.856364 14823 tablet_replica.cc:331] T 642e5f4889ec46adb023fdaa45e99773 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:11.856472 14823 raft_consensus.cc:2238] T 642e5f4889ec46adb023fdaa45e99773 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.856441 14809 consensus_queue.cc:260] T 46e799d700db4803b79ed46273b99df6 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.856552 14823 raft_consensus.cc:2267] T 642e5f4889ec46adb023fdaa45e99773 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.856580 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet cc88653cdfec4120b37097b361f441d4
I20250115 03:06:11.856688 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa7cecc384f495c9ff4723f553acc24" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:11.856796 14422 raft_consensus.cc:2463] T afa7cecc384f495c9ff4723f553acc24 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:11.856823 14164 catalog_manager.cc:6596] ReplaceTablet: tablet cc88653cdfec4120b37097b361f441d4 deleted and replaced by tablet 108aafc0c034498bbd300be9c6b1f8c4
I20250115 03:06:11.856700 14809 ts_tablet_manager.cc:1428] T 46e799d700db4803b79ed46273b99df6 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.857117 14628 leader_election.cc:304] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.857152 14809 tablet_bootstrap.cc:492] T 12a06b7730eb40fa8e73722cf75e9dce P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.857143 14823 ts_tablet_manager.cc:1905] T 642e5f4889ec46adb023fdaa45e99773 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.857263 14824 raft_consensus.cc:2798] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:11.857308 14824 raft_consensus.cc:491] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.857343 14824 raft_consensus.cc:3054] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.857996 14809 tablet_bootstrap.cc:654] T 12a06b7730eb40fa8e73722cf75e9dce P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.858774 14823 ts_tablet_manager.cc:1918] T 642e5f4889ec46adb023fdaa45e99773 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.858830 14823 log.cc:1198] T 642e5f4889ec46adb023fdaa45e99773 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/642e5f4889ec46adb023fdaa45e99773
I20250115 03:06:11.858829 14811 tablet_bootstrap.cc:492] T 12a06b7730eb40fa8e73722cf75e9dce P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.858901 14811 ts_tablet_manager.cc:1397] T 12a06b7730eb40fa8e73722cf75e9dce P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20250115 03:06:11.858999 14823 ts_tablet_manager.cc:1939] T 642e5f4889ec46adb023fdaa45e99773 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:11.859431 14811 raft_consensus.cc:357] T 12a06b7730eb40fa8e73722cf75e9dce P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.859596 14811 raft_consensus.cc:383] T 12a06b7730eb40fa8e73722cf75e9dce P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.859622 14402 tablet_service.cc:1514] Processing DeleteTablet for tablet 180df492348a4cc09afa03c8a62a79c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.859794 14819 tablet_replica.cc:331] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:11.859889 14819 raft_consensus.cc:2238] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.859866 14824 raft_consensus.cc:513] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.859977 14819 raft_consensus.cc:2267] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.860018 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 4379cf2eb12844a29c5f2e36d1e1dffc (table default.Table_8f3bf985bedd43918313ea7504253e48 [id=05f9fd2ebec34d2299ea59c47a74c828]) successfully deleted
I20250115 03:06:11.860172 14824 leader_election.cc:290] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:11.860179 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 642e5f4889ec46adb023fdaa45e99773 (table default.Table_1e37785047d64a1fb98662234aea76a5 [id=f00e0e2286064402802dce72256e92b8]) successfully deleted
I20250115 03:06:11.860308 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a863928b49443329dca51c60cc7fae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.860550 14819 ts_tablet_manager.cc:1905] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.860677 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa7cecc384f495c9ff4723f553acc24" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:11.860718 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa7cecc384f495c9ff4723f553acc24" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:11.860788 14557 raft_consensus.cc:2463] T afa7cecc384f495c9ff4723f553acc24 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:11.860806 14422 raft_consensus.cc:3054] T afa7cecc384f495c9ff4723f553acc24 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.860880 14812 tablet_bootstrap.cc:492] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.860970 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afa7cecc384f495c9ff4723f553acc24" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:11.861050 14556 raft_consensus.cc:3054] T afa7cecc384f495c9ff4723f553acc24 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.861620 14422 raft_consensus.cc:2463] T afa7cecc384f495c9ff4723f553acc24 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:11.861750 14812 tablet_bootstrap.cc:654] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.861852 14556 raft_consensus.cc:2463] T afa7cecc384f495c9ff4723f553acc24 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:11.861888 14628 leader_election.cc:304] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.860675 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet cc88653cdfec4120b37097b361f441d4 with delete_type TABLET_DATA_DELETED (replaced by tablet 108aafc0c034498bbd300be9c6b1f8c4) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.862028 14824 raft_consensus.cc:2798] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:11.862092 14824 raft_consensus.cc:695] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:11.862201 14819 ts_tablet_manager.cc:1918] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:11.862205 14824 consensus_queue.cc:237] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.862262 14819 log.cc:1198] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/180df492348a4cc09afa03c8a62a79c1
I20250115 03:06:11.862485 14819 ts_tablet_manager.cc:1939] T 180df492348a4cc09afa03c8a62a79c1 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:11.862725 14820 ts_tablet_manager.cc:1905] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.859635 14811 raft_consensus.cc:738] T 12a06b7730eb40fa8e73722cf75e9dce P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.863041 14811 consensus_queue.cc:260] T 12a06b7730eb40fa8e73722cf75e9dce P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.863399 14811 ts_tablet_manager.cc:1428] T 12a06b7730eb40fa8e73722cf75e9dce P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:11.863554 14811 tablet_bootstrap.cc:492] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.864392 14811 tablet_bootstrap.cc:654] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.864411 14820 ts_tablet_manager.cc:1918] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:11.864470 14820 log.cc:1198] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/180df492348a4cc09afa03c8a62a79c1
I20250115 03:06:11.864539 14671 tablet_service.cc:1514] Processing DeleteTablet for tablet cc88653cdfec4120b37097b361f441d4 with delete_type TABLET_DATA_DELETED (replaced by tablet 108aafc0c034498bbd300be9c6b1f8c4) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:11.864679 14820 ts_tablet_manager.cc:1939] T 180df492348a4cc09afa03c8a62a79c1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:11.864662 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 180df492348a4cc09afa03c8a62a79c1 (table default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]) successfully deleted
I20250115 03:06:11.864810 14402 tablet_service.cc:1514] Processing DeleteTablet for tablet cc88653cdfec4120b37097b361f441d4 with delete_type TABLET_DATA_DELETED (replaced by tablet 108aafc0c034498bbd300be9c6b1f8c4) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.864969 14819 tablet_replica.cc:331] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:11.865069 14819 raft_consensus.cc:2238] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:11.865330 14819 raft_consensus.cc:2267] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.865607 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a863928b49443329dca51c60cc7fae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.866009 14820 tablet_replica.cc:331] T 642e5f4889ec46adb023fdaa45e99773 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:11.866089 14819 ts_tablet_manager.cc:1905] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.866111 14820 raft_consensus.cc:2238] T 642e5f4889ec46adb023fdaa45e99773 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.866194 14820 raft_consensus.cc:2267] T 642e5f4889ec46adb023fdaa45e99773 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.866210 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 180df492348a4cc09afa03c8a62a79c1 (table default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]) successfully deleted
I20250115 03:06:11.866519 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 642e5f4889ec46adb023fdaa45e99773 (table default.Table_1e37785047d64a1fb98662234aea76a5 [id=f00e0e2286064402802dce72256e92b8]) successfully deleted
I20250115 03:06:11.866530 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 026bf50b4721424aa06c59455fe7d07e (table default.Table_6098a167e4524987bbc24d8dfa47f5e4 [id=eb34ed3c30694c39bf957a6af751a9b7]). Expected version 1 got 0
I20250115 03:06:11.867075 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 46e799d700db4803b79ed46273b99df6 (DEFAULT_TABLE table=default.Table_77f0e54f01874d84ac9475bf9d819946 [id=f5e54ac99a344e05a0be4c913472dd37]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.867282 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46e799d700db4803b79ed46273b99df6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.867714 14819 ts_tablet_manager.cc:1918] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:11.867769 14819 log.cc:1198] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/cc88653cdfec4120b37097b361f441d4
I20250115 03:06:11.867969 14819 ts_tablet_manager.cc:1939] T cc88653cdfec4120b37097b361f441d4 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:11.868239 14820 ts_tablet_manager.cc:1905] T 642e5f4889ec46adb023fdaa45e99773 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.868454 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet cc88653cdfec4120b37097b361f441d4 (table default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd]) successfully deleted
I20250115 03:06:11.868819 14853 raft_consensus.cc:491] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.868894 14853 raft_consensus.cc:513] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.868952 14163 catalog_manager.cc:5526] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:11.869227 14853 leader_election.cc:290] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.869499 14671 tablet_service.cc:1514] Processing DeleteTablet for tablet 35df9fe895414d4e87c7923da9703e59 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:11.869611 14533 tablet_service.cc:1467] Processing CreateTablet for tablet c6cd63ac1c084a0d8f166e63b0ed85bc (DEFAULT_TABLE table=default.Table_8f3bf985bedd43918313ea7504253e48 [id=05f9fd2ebec34d2299ea59c47a74c828]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.869820 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6cd63ac1c084a0d8f166e63b0ed85bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.869882 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4f3df7129b4fb79c67de414e7bac88" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:11.869975 14422 raft_consensus.cc:2463] T 9f4f3df7129b4fb79c67de414e7bac88 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:11.870225 14493 leader_election.cc:304] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.870381 14820 ts_tablet_manager.cc:1918] T 642e5f4889ec46adb023fdaa45e99773 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.870437 14820 log.cc:1198] T 642e5f4889ec46adb023fdaa45e99773 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/642e5f4889ec46adb023fdaa45e99773
I20250115 03:06:11.870486 14853 raft_consensus.cc:2798] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:11.870535 14853 raft_consensus.cc:491] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.870570 14853 raft_consensus.cc:3054] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.870553 14401 tablet_service.cc:1467] Processing CreateTablet for tablet c6cd63ac1c084a0d8f166e63b0ed85bc (DEFAULT_TABLE table=default.Table_8f3bf985bedd43918313ea7504253e48 [id=05f9fd2ebec34d2299ea59c47a74c828]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.870609 14820 ts_tablet_manager.cc:1939] T 642e5f4889ec46adb023fdaa45e99773 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:11.870745 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6cd63ac1c084a0d8f166e63b0ed85bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.870826 14823 tablet_replica.cc:331] T 9a863928b49443329dca51c60cc7fae0 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:11.870913 14823 raft_consensus.cc:2238] T 9a863928b49443329dca51c60cc7fae0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.870994 14823 raft_consensus.cc:2267] T 9a863928b49443329dca51c60cc7fae0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.871115 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 4379cf2eb12844a29c5f2e36d1e1dffc with delete_type TABLET_DATA_DELETED (replaced by tablet c6cd63ac1c084a0d8f166e63b0ed85bc) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:11.871362 14853 raft_consensus.cc:513] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.871490 14823 ts_tablet_manager.cc:1905] T 9a863928b49443329dca51c60cc7fae0 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.871600 14819 tablet_replica.cc:331] T 9a863928b49443329dca51c60cc7fae0 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:11.871650 14828 raft_consensus.cc:491] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.871661 14853 leader_election.cc:290] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.871726 14819 raft_consensus.cc:2238] T 9a863928b49443329dca51c60cc7fae0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.871713 14828 raft_consensus.cc:513] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.871804 14819 raft_consensus.cc:2267] T 9a863928b49443329dca51c60cc7fae0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.872013 14828 leader_election.cc:290] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:11.872071 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4f3df7129b4fb79c67de414e7bac88" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:11.872172 14691 raft_consensus.cc:3054] T 9f4f3df7129b4fb79c67de414e7bac88 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.872323 14820 tablet_replica.cc:331] T 9a863928b49443329dca51c60cc7fae0 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:11.872416 14820 raft_consensus.cc:2238] T 9a863928b49443329dca51c60cc7fae0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.872488 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 642e5f4889ec46adb023fdaa45e99773 (table default.Table_1e37785047d64a1fb98662234aea76a5 [id=f00e0e2286064402802dce72256e92b8]) successfully deleted
I20250115 03:06:11.872586 14820 raft_consensus.cc:2267] T 9a863928b49443329dca51c60cc7fae0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.873114 14820 ts_tablet_manager.cc:1905] T 9a863928b49443329dca51c60cc7fae0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.873143 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64a668e2f4548f3bd4f2b0be32e8682" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:11.873229 14422 raft_consensus.cc:2463] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:11.873409 14823 ts_tablet_manager.cc:1918] T 9a863928b49443329dca51c60cc7fae0 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.873466 14823 log.cc:1198] T 9a863928b49443329dca51c60cc7fae0 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/9a863928b49443329dca51c60cc7fae0
I20250115 03:06:11.873512 14628 leader_election.cc:304] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.873636 14823 ts_tablet_manager.cc:1939] T 9a863928b49443329dca51c60cc7fae0 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:11.873656 14828 raft_consensus.cc:2798] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:11.873684 14819 ts_tablet_manager.cc:1905] T 9a863928b49443329dca51c60cc7fae0 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.873701 14828 raft_consensus.cc:491] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.873736 14828 raft_consensus.cc:3054] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.874061 14691 raft_consensus.cc:2463] T 9f4f3df7129b4fb79c67de414e7bac88 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:11.874322 14809 tablet_bootstrap.cc:492] T 12a06b7730eb40fa8e73722cf75e9dce P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.874322 14811 tablet_bootstrap.cc:492] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.874387 14809 ts_tablet_manager.cc:1397] T 12a06b7730eb40fa8e73722cf75e9dce P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20250115 03:06:11.874387 14811 ts_tablet_manager.cc:1397] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20250115 03:06:11.874370 14493 leader_election.cc:304] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:11.874518 14817 raft_consensus.cc:491] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.874578 14817 raft_consensus.cc:513] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.874715 14820 ts_tablet_manager.cc:1918] T 9a863928b49443329dca51c60cc7fae0 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.874759 14809 raft_consensus.cc:357] T 12a06b7730eb40fa8e73722cf75e9dce P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.874841 14809 raft_consensus.cc:383] T 12a06b7730eb40fa8e73722cf75e9dce P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.874763 14811 raft_consensus.cc:357] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.874868 14809 raft_consensus.cc:738] T 12a06b7730eb40fa8e73722cf75e9dce P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.874898 14811 raft_consensus.cc:383] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.874927 14811 raft_consensus.cc:738] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.874962 14809 consensus_queue.cc:260] T 12a06b7730eb40fa8e73722cf75e9dce P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.874853 14817 leader_election.cc:290] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:11.875041 14811 consensus_queue.cc:260] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.875181 14809 ts_tablet_manager.cc:1428] T 12a06b7730eb40fa8e73722cf75e9dce P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.875257 14811 ts_tablet_manager.cc:1428] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.875249 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64a668e2f4548f3bd4f2b0be32e8682" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:11.875332 14809 tablet_bootstrap.cc:492] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.875355 14556 raft_consensus.cc:2463] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:11.875416 14811 tablet_bootstrap.cc:492] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.875411 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64a668e2f4548f3bd4f2b0be32e8682" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:11.876152 14809 tablet_bootstrap.cc:654] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.876272 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4f3df7129b4fb79c67de414e7bac88" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:11.876356 14422 raft_consensus.cc:3054] T 9f4f3df7129b4fb79c67de414e7bac88 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.876621 14811 tablet_bootstrap.cc:654] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.876669 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64a668e2f4548f3bd4f2b0be32e8682" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:11.876765 14421 raft_consensus.cc:2463] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:11.874765 14820 log.cc:1198] T 9a863928b49443329dca51c60cc7fae0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/9a863928b49443329dca51c60cc7fae0
I20250115 03:06:11.877146 14422 raft_consensus.cc:2463] T 9f4f3df7129b4fb79c67de414e7bac88 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:11.877288 14820 ts_tablet_manager.cc:1939] T 9a863928b49443329dca51c60cc7fae0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:11.877410 14493 leader_election.cc:304] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.877671 14817 raft_consensus.cc:2798] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:11.877718 14817 raft_consensus.cc:491] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.877751 14817 raft_consensus.cc:3054] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.877960 14812 tablet_bootstrap.cc:492] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.878032 14812 ts_tablet_manager.cc:1397] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20250115 03:06:11.878118 14819 ts_tablet_manager.cc:1918] T 9a863928b49443329dca51c60cc7fae0 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.878171 14819 log.cc:1198] T 9a863928b49443329dca51c60cc7fae0 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/9a863928b49443329dca51c60cc7fae0
I20250115 03:06:11.878288 14858 raft_consensus.cc:491] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.878335 14819 ts_tablet_manager.cc:1939] T 9a863928b49443329dca51c60cc7fae0 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:11.878350 14858 raft_consensus.cc:513] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.878427 14812 raft_consensus.cc:357] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.878474 14817 raft_consensus.cc:513] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.878508 14812 raft_consensus.cc:383] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.878538 14812 raft_consensus.cc:738] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.878633 14812 consensus_queue.cc:260] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.878667 14858 leader_election.cc:290] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.878697 14817 leader_election.cc:290] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:11.878897 14812 ts_tablet_manager.cc:1428] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.878986 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4f3df7129b4fb79c67de414e7bac88" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:11.879050 14812 tablet_bootstrap.cc:492] T 46e799d700db4803b79ed46273b99df6 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.879065 14692 raft_consensus.cc:2371] T 9f4f3df7129b4fb79c67de414e7bac88 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1. Re-sending same reply.
I20250115 03:06:11.879225 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64a668e2f4548f3bd4f2b0be32e8682" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:11.879346 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 9a863928b49443329dca51c60cc7fae0 (table default.Table_17695f503db342d2975cc920bdcda1d7 [id=060ecaa51ce14dffa42d21f7c72859aa]) successfully deleted
I20250115 03:06:11.879490 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70a6241a383c461ca646a7d33911d654" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:11.879580 14689 raft_consensus.cc:2463] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:11.879777 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 9a863928b49443329dca51c60cc7fae0 (table default.Table_17695f503db342d2975cc920bdcda1d7 [id=060ecaa51ce14dffa42d21f7c72859aa]) successfully deleted
I20250115 03:06:11.874845 14853 raft_consensus.cc:2798] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:11.879994 14853 raft_consensus.cc:695] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:11.880069 14812 tablet_bootstrap.cc:654] T 46e799d700db4803b79ed46273b99df6 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.880113 14853 consensus_queue.cc:237] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.880211 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70a6241a383c461ca646a7d33911d654" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:11.880298 14422 raft_consensus.cc:2463] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:11.880450 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64a668e2f4548f3bd4f2b0be32e8682" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:11.880534 14421 raft_consensus.cc:3054] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.881340 14421 raft_consensus.cc:2463] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:11.881632 14672 tablet_service.cc:1467] Processing CreateTablet for tablet c6cd63ac1c084a0d8f166e63b0ed85bc (DEFAULT_TABLE table=default.Table_8f3bf985bedd43918313ea7504253e48 [id=05f9fd2ebec34d2299ea59c47a74c828]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.881837 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6cd63ac1c084a0d8f166e63b0ed85bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.882243 14493 leader_election.cc:304] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.882390 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 9f4f3df7129b4fb79c67de414e7bac88
I20250115 03:06:11.882400 14493 leader_election.cc:304] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.882643 14162 catalog_manager.cc:6596] ReplaceTablet: tablet 9f4f3df7129b4fb79c67de414e7bac88 deleted and replaced by tablet b41d9a854bac49599d3390c15f31e0fa
I20250115 03:06:11.882804 14817 raft_consensus.cc:2798] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:11.882779 14828 raft_consensus.cc:513] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.882851 14817 raft_consensus.cc:491] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.882884 14817 raft_consensus.cc:3054] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.883090 14828 leader_election.cc:290] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:11.883117 14691 raft_consensus.cc:2388] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in current term 1: Already voted for candidate 5c0dae7aff4b44c699275b95ba639770 in this term.
I20250115 03:06:11.883646 14817 raft_consensus.cc:513] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.883680 14671 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f4f3df7129b4fb79c67de414e7bac88 with delete_type TABLET_DATA_DELETED (replaced by tablet b41d9a854bac49599d3390c15f31e0fa) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:11.883816 14823 tablet_replica.cc:331] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:11.883807 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 9a863928b49443329dca51c60cc7fae0 (table default.Table_17695f503db342d2975cc920bdcda1d7 [id=060ecaa51ce14dffa42d21f7c72859aa]) successfully deleted
I20250115 03:06:11.883919 14823 raft_consensus.cc:2238] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.883929 14817 leader_election.cc:290] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.884003 14823 raft_consensus.cc:2267] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.884025 14817 raft_consensus.cc:2798] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:11.884088 14817 raft_consensus.cc:695] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:11.884184 14817 consensus_queue.cc:237] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.884474 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f4f3df7129b4fb79c67de414e7bac88 with delete_type TABLET_DATA_DELETED (replaced by tablet b41d9a854bac49599d3390c15f31e0fa) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.884653 14819 tablet_replica.cc:331] T 9f4f3df7129b4fb79c67de414e7bac88 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:11.884752 14819 raft_consensus.cc:2238] T 9f4f3df7129b4fb79c67de414e7bac88 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.884824 14819 raft_consensus.cc:2267] T 9f4f3df7129b4fb79c67de414e7bac88 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.885282 14819 ts_tablet_manager.cc:1905] T 9f4f3df7129b4fb79c67de414e7bac88 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.885530 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f4f3df7129b4fb79c67de414e7bac88 with delete_type TABLET_DATA_DELETED (replaced by tablet b41d9a854bac49599d3390c15f31e0fa) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.885665 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64a668e2f4548f3bd4f2b0be32e8682" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:11.885912 14823 ts_tablet_manager.cc:1905] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.886790 14819 ts_tablet_manager.cc:1918] T 9f4f3df7129b4fb79c67de414e7bac88 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.886813 14167 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 026bf50b4721424aa06c59455fe7d07e (table default.Table_6098a167e4524987bbc24d8dfa47f5e4 [id=eb34ed3c30694c39bf957a6af751a9b7]). Expected version 1 got 0
I20250115 03:06:11.886844 14819 log.cc:1198] T 9f4f3df7129b4fb79c67de414e7bac88 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/9f4f3df7129b4fb79c67de414e7bac88
I20250115 03:06:11.887012 14819 ts_tablet_manager.cc:1939] T 9f4f3df7129b4fb79c67de414e7bac88 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:11.887059 14885 raft_consensus.cc:491] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.887167 14885 raft_consensus.cc:513] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.887213 14820 tablet_replica.cc:331] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:11.887312 14820 raft_consensus.cc:2238] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.887405 14820 raft_consensus.cc:2267] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.887519 14885 leader_election.cc:290] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.887529 14823 ts_tablet_manager.cc:1918] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:11.887583 14823 log.cc:1198] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/180df492348a4cc09afa03c8a62a79c1
I20250115 03:06:11.887797 14823 ts_tablet_manager.cc:1939] T 180df492348a4cc09afa03c8a62a79c1 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:11.887950 14820 ts_tablet_manager.cc:1905] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.888284 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 180df492348a4cc09afa03c8a62a79c1 (table default.Table_017cebf1769e47a997604ce80cd74d9b [id=06ba0b75b43c4c788971375a044ed157]) successfully deleted
I20250115 03:06:11.888350 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35df9fe895414d4e87c7923da9703e59" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:11.888396 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70a6241a383c461ca646a7d33911d654" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:11.888456 14691 raft_consensus.cc:2463] T 35df9fe895414d4e87c7923da9703e59 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:11.888458 14689 raft_consensus.cc:3054] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.888463 14823 tablet_replica.cc:331] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:11.888779 14823 raft_consensus.cc:2238] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.888870 14823 raft_consensus.cc:2267] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.888991 14809 tablet_bootstrap.cc:492] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.889060 14809 ts_tablet_manager.cc:1397] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20250115 03:06:11.889111 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70a6241a383c461ca646a7d33911d654" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:11.889197 14422 raft_consensus.cc:3054] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.889401 14689 raft_consensus.cc:2463] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:11.889453 14809 raft_consensus.cc:357] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.889549 14809 raft_consensus.cc:383] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.889549 14823 ts_tablet_manager.cc:1905] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.889578 14809 raft_consensus.cc:738] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.889665 14358 leader_election.cc:304] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.889740 14828 raft_consensus.cc:491] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.889799 14828 raft_consensus.cc:513] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.889873 14885 raft_consensus.cc:2798] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:11.889917 14885 raft_consensus.cc:491] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.889946 14422 raft_consensus.cc:2463] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:11.889953 14885 raft_consensus.cc:3054] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.890062 14828 leader_election.cc:290] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:11.889683 14809 consensus_queue.cc:260] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.890311 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64a668e2f4548f3bd4f2b0be32e8682" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:11.890440 14421 raft_consensus.cc:2388] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c0dae7aff4b44c699275b95ba639770 in current term 1: Already voted for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in this term.
I20250115 03:06:11.890532 14809 ts_tablet_manager.cc:1428] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.890847 14885 raft_consensus.cc:513] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.891127 14885 leader_election.cc:290] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.891459 14823 ts_tablet_manager.cc:1918] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:11.891518 14823 log.cc:1198] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/cc88653cdfec4120b37097b361f441d4
I20250115 03:06:11.891712 14628 leader_election.cc:304] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:11.891757 14823 ts_tablet_manager.cc:1939] T cc88653cdfec4120b37097b361f441d4 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:11.892007 14828 raft_consensus.cc:2743] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250115 03:06:11.892212 14823 tablet_replica.cc:331] T 35df9fe895414d4e87c7923da9703e59 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:11.892308 14823 raft_consensus.cc:2238] T 35df9fe895414d4e87c7923da9703e59 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.892385 14823 raft_consensus.cc:2267] T 35df9fe895414d4e87c7923da9703e59 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.892393 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12a06b7730eb40fa8e73722cf75e9dce" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:11.892503 14555 raft_consensus.cc:2463] T 12a06b7730eb40fa8e73722cf75e9dce P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:11.892755 14626 leader_election.cc:304] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:11.892895 14828 raft_consensus.cc:2798] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:11.892900 14811 tablet_bootstrap.cc:492] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.892952 14828 raft_consensus.cc:491] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.892990 14828 raft_consensus.cc:3054] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.893005 14811 ts_tablet_manager.cc:1397] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.001s
I20250115 03:06:11.893383 14811 raft_consensus.cc:357] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.893479 14811 raft_consensus.cc:383] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.893476 14059 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu
/tmp/dist-test-taskE2XgYg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.186.254:39901
--webserver_interface=127.13.186.254
--webserver_port=37481
--builtin_ntp_servers=127.13.186.212:37407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.186.254:39901,127.13.186.253:35411,127.13.186.252:41855
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20250115 03:06:11.893510 14811 raft_consensus.cc:738] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.893612 14811 consensus_queue.cc:260] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.893738 14828 raft_consensus.cc:513] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.893834 14811 ts_tablet_manager.cc:1428] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.894044 14828 leader_election.cc:290] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:11.894271 14823 ts_tablet_manager.cc:1905] T 35df9fe895414d4e87c7923da9703e59 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.894506 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet cc88653cdfec4120b37097b361f441d4 (table default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd]) successfully deleted
I20250115 03:06:11.894706 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35df9fe895414d4e87c7923da9703e59" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
W20250115 03:06:11.895056 14358 leader_election.cc:343] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: Tablet not RUNNING: STOPPED
I20250115 03:06:11.895133 14820 ts_tablet_manager.cc:1918] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:11.895190 14820 log.cc:1198] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/cc88653cdfec4120b37097b361f441d4
I20250115 03:06:11.895242 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35df9fe895414d4e87c7923da9703e59" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:11.895329 14557 raft_consensus.cc:3054] T 35df9fe895414d4e87c7923da9703e59 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.895398 14820 ts_tablet_manager.cc:1939] T cc88653cdfec4120b37097b361f441d4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:11.895797 14823 ts_tablet_manager.cc:1918] T 35df9fe895414d4e87c7923da9703e59 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.895917 14823 log.cc:1198] T 35df9fe895414d4e87c7923da9703e59 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/35df9fe895414d4e87c7923da9703e59
I20250115 03:06:11.895942 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet cc88653cdfec4120b37097b361f441d4 (table default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd]) successfully deleted
I20250115 03:06:11.895972 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 9f4f3df7129b4fb79c67de414e7bac88 (table default.Table_0bdf66c6040b4709b0a6be57188b9d86 [id=c8fdb0f9f9064f3eb8d19f4adf3c439f]) successfully deleted
I20250115 03:06:11.896106 14557 raft_consensus.cc:2463] T 35df9fe895414d4e87c7923da9703e59 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:11.896135 14820 tablet_replica.cc:331] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:11.896251 14820 raft_consensus.cc:2238] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:11.896275 14823 ts_tablet_manager.cc:1939] T 35df9fe895414d4e87c7923da9703e59 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:11.896373 14820 pending_rounds.cc:62] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1: Trying to abort 1 pending ops.
I20250115 03:06:11.896361 14356 leader_election.cc:304] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770
I20250115 03:06:11.896399 14820 pending_rounds.cc:69] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384883222089728 op_type: NO_OP noop_request { }
I20250115 03:06:11.896443 14820 raft_consensus.cc:2883] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:11.896497 14820 raft_consensus.cc:2267] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.896595 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35df9fe895414d4e87c7923da9703e59" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:11.896698 14885 raft_consensus.cc:2798] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:11.896713 14556 raft_consensus.cc:2371] T 35df9fe895414d4e87c7923da9703e59 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1. Re-sending same reply.
I20250115 03:06:11.896771 14885 raft_consensus.cc:695] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:11.896813 14823 tablet_replica.cc:331] T 9f4f3df7129b4fb79c67de414e7bac88 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:11.896912 14823 raft_consensus.cc:2238] T 9f4f3df7129b4fb79c67de414e7bac88 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.897006 14823 raft_consensus.cc:2267] T 9f4f3df7129b4fb79c67de414e7bac88 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.896875 14885 consensus_queue.cc:237] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.897130 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 35df9fe895414d4e87c7923da9703e59 (table default.Table_194d150ada7243d58b5e8d31c3a15afc [id=3d0789d7623c43a58d33e9ede070f7c5]) successfully deleted
W20250115 03:06:11.897266 14158 catalog_manager.cc:4839] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 4379cf2eb12844a29c5f2e36d1e1dffc because the tablet was not found. No further retry: Not found: Tablet not found: 4379cf2eb12844a29c5f2e36d1e1dffc
I20250115 03:06:11.897394 14823 ts_tablet_manager.cc:1905] T 9f4f3df7129b4fb79c67de414e7bac88 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.897485 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12a06b7730eb40fa8e73722cf75e9dce" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:11.897572 14557 raft_consensus.cc:3054] T 12a06b7730eb40fa8e73722cf75e9dce P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.898286 14557 raft_consensus.cc:2463] T 12a06b7730eb40fa8e73722cf75e9dce P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:11.898555 14626 leader_election.cc:304] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:11.898720 14828 raft_consensus.cc:2798] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:11.898792 14828 raft_consensus.cc:695] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:11.898895 14828 consensus_queue.cc:237] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.898949 14823 ts_tablet_manager.cc:1918] T 9f4f3df7129b4fb79c67de414e7bac88 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.899008 14823 log.cc:1198] T 9f4f3df7129b4fb79c67de414e7bac88 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/9f4f3df7129b4fb79c67de414e7bac88
I20250115 03:06:11.899173 14823 ts_tablet_manager.cc:1939] T 9f4f3df7129b4fb79c67de414e7bac88 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
W20250115 03:06:11.899214 14939 consensus_queue.cc:1159] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20250115 03:06:11.899708 14812 tablet_bootstrap.cc:492] T 46e799d700db4803b79ed46273b99df6 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.899729 14820 ts_tablet_manager.cc:1905] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.899781 14812 ts_tablet_manager.cc:1397] T 46e799d700db4803b79ed46273b99df6 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.000s
I20250115 03:06:11.900122 14812 raft_consensus.cc:357] T 46e799d700db4803b79ed46273b99df6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.900204 14812 raft_consensus.cc:383] T 46e799d700db4803b79ed46273b99df6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.900228 14812 raft_consensus.cc:738] T 46e799d700db4803b79ed46273b99df6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.900319 14812 consensus_queue.cc:260] T 46e799d700db4803b79ed46273b99df6 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.900518 14812 ts_tablet_manager.cc:1428] T 46e799d700db4803b79ed46273b99df6 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.900651 14812 tablet_bootstrap.cc:492] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.901372 14820 ts_tablet_manager.cc:1918] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:11.901429 14820 log.cc:1198] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/9f4f3df7129b4fb79c67de414e7bac88
I20250115 03:06:11.901579 14812 tablet_bootstrap.cc:654] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.901654 14820 ts_tablet_manager.cc:1939] T 9f4f3df7129b4fb79c67de414e7bac88 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:11.902089 14163 catalog_manager.cc:5526] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:11.902235 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 9f4f3df7129b4fb79c67de414e7bac88 (table default.Table_0bdf66c6040b4709b0a6be57188b9d86 [id=c8fdb0f9f9064f3eb8d19f4adf3c439f]) successfully deleted
I20250115 03:06:11.902720 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12a06b7730eb40fa8e73722cf75e9dce" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:11.902766 14812 tablet_bootstrap.cc:492] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.902782 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 9f4f3df7129b4fb79c67de414e7bac88 (table default.Table_0bdf66c6040b4709b0a6be57188b9d86 [id=c8fdb0f9f9064f3eb8d19f4adf3c439f]) successfully deleted
I20250115 03:06:11.902810 14422 raft_consensus.cc:3054] T 12a06b7730eb40fa8e73722cf75e9dce P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.902834 14812 ts_tablet_manager.cc:1397] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:11.903239 14812 raft_consensus.cc:357] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.903319 14812 raft_consensus.cc:383] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.903348 14812 raft_consensus.cc:738] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.903437 14812 consensus_queue.cc:260] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.903582 14422 raft_consensus.cc:2463] T 12a06b7730eb40fa8e73722cf75e9dce P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:11.903673 14812 ts_tablet_manager.cc:1428] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.904903 14493 leader_election.cc:304] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.905066 14853 raft_consensus.cc:2798] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:11.905129 14853 raft_consensus.cc:695] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:11.905236 14853 consensus_queue.cc:237] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.905433 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12a06b7730eb40fa8e73722cf75e9dce" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:11.905444 14171 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 026bf50b4721424aa06c59455fe7d07e (table default.Table_6098a167e4524987bbc24d8dfa47f5e4 [id=eb34ed3c30694c39bf957a6af751a9b7]). Expected version 1 got 0
I20250115 03:06:11.905526 14421 raft_consensus.cc:2371] T 12a06b7730eb40fa8e73722cf75e9dce P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1. Re-sending same reply.
I20250115 03:06:11.906610 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 35df9fe895414d4e87c7923da9703e59 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:11.906752 14820 tablet_replica.cc:331] T 35df9fe895414d4e87c7923da9703e59 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:11.906849 14820 raft_consensus.cc:2238] T 35df9fe895414d4e87c7923da9703e59 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:11.906930 14820 raft_consensus.cc:2267] T 35df9fe895414d4e87c7923da9703e59 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.907107 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f4f3df7129b4fb79c67de414e7bac88 with delete_type TABLET_DATA_DELETED (replaced by tablet b41d9a854bac49599d3390c15f31e0fa) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.907229 14402 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a863928b49443329dca51c60cc7fae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.907300 14820 ts_tablet_manager.cc:1905] T 35df9fe895414d4e87c7923da9703e59 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.907343 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 35df9fe895414d4e87c7923da9703e59 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.907580 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 642e5f4889ec46adb023fdaa45e99773 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.907711 14885 raft_consensus.cc:491] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.907766 14885 raft_consensus.cc:513] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.908051 14885 leader_election.cc:290] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.909194 14819 tablet_replica.cc:331] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:11.909312 14819 raft_consensus.cc:2238] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:11.909427 14819 pending_rounds.cc:62] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75: Trying to abort 1 pending ops.
I20250115 03:06:11.909492 14819 pending_rounds.cc:69] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384883291475968 op_type: NO_OP noop_request { }
I20250115 03:06:11.909544 14819 raft_consensus.cc:2883] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:11.909590 14819 raft_consensus.cc:2267] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:11.909816 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa88f604011741f2b3eb2699b9328eae" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:11.909924 14689 raft_consensus.cc:2463] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:11.910152 14358 leader_election.cc:304] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.910290 14885 raft_consensus.cc:2798] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20250115 03:06:11.910286 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 9f4f3df7129b4fb79c67de414e7bac88 because the tablet was not found. No further retry: Not found: Tablet not found: 9f4f3df7129b4fb79c67de414e7bac88
I20250115 03:06:11.910329 14885 raft_consensus.cc:491] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.910367 14885 raft_consensus.cc:3054] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:11.910418 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 9a863928b49443329dca51c60cc7fae0 because the tablet was not found. No further retry: Not found: Tablet not found: 9a863928b49443329dca51c60cc7fae0
I20250115 03:06:11.911165 14885 raft_consensus.cc:513] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.911494 14885 leader_election.cc:290] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.911962 14820 ts_tablet_manager.cc:1918] T 35df9fe895414d4e87c7923da9703e59 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:11.912026 14820 log.cc:1198] T 35df9fe895414d4e87c7923da9703e59 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/35df9fe895414d4e87c7923da9703e59
I20250115 03:06:11.912204 14820 ts_tablet_manager.cc:1939] T 35df9fe895414d4e87c7923da9703e59 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:11.912714 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 35df9fe895414d4e87c7923da9703e59 (table default.Table_194d150ada7243d58b5e8d31c3a15afc [id=3d0789d7623c43a58d33e9ede070f7c5]) successfully deleted
I20250115 03:06:11.913038 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa88f604011741f2b3eb2699b9328eae" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:11.913127 14689 raft_consensus.cc:3054] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.913209 14819 ts_tablet_manager.cc:1905] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:11.913908 14689 raft_consensus.cc:2463] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:11.914165 14358 leader_election.cc:304] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.914301 14885 raft_consensus.cc:2798] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:11.914363 14885 raft_consensus.cc:695] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:11.914466 14885 consensus_queue.cc:237] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.914985 14819 ts_tablet_manager.cc:1918] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:11.915083 14819 log.cc:1198] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/35df9fe895414d4e87c7923da9703e59
I20250115 03:06:11.915290 14819 ts_tablet_manager.cc:1939] T 35df9fe895414d4e87c7923da9703e59 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:11.915813 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 35df9fe895414d4e87c7923da9703e59 (table default.Table_194d150ada7243d58b5e8d31c3a15afc [id=3d0789d7623c43a58d33e9ede070f7c5]) successfully deleted
W20250115 03:06:11.916182 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 642e5f4889ec46adb023fdaa45e99773 because the tablet was not found. No further retry: Not found: Tablet not found: 642e5f4889ec46adb023fdaa45e99773
I20250115 03:06:11.917214 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa88f604011741f2b3eb2699b9328eae" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:11.917311 14556 raft_consensus.cc:3054] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.918085 14556 raft_consensus.cc:2463] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:11.918557 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa88f604011741f2b3eb2699b9328eae" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:11.918651 14557 raft_consensus.cc:2371] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1. Re-sending same reply.
I20250115 03:06:11.918859 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a863928b49443329dca51c60cc7fae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:11.919266 14156 catalog_manager.cc:4839] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 9a863928b49443329dca51c60cc7fae0 because the tablet was not found. No further retry: Not found: Tablet not found: 9a863928b49443329dca51c60cc7fae0
I20250115 03:06:11.925648 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 108aafc0c034498bbd300be9c6b1f8c4 (DEFAULT_TABLE table=default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.925812 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 108aafc0c034498bbd300be9c6b1f8c4 (DEFAULT_TABLE table=default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.925889 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 108aafc0c034498bbd300be9c6b1f8c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.926002 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 108aafc0c034498bbd300be9c6b1f8c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.927651 14809 tablet_bootstrap.cc:492] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.927651 14811 tablet_bootstrap.cc:492] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.928519 14811 tablet_bootstrap.cc:654] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.928519 14809 tablet_bootstrap.cc:654] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.928683 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 108aafc0c034498bbd300be9c6b1f8c4 (DEFAULT_TABLE table=default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.928947 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 108aafc0c034498bbd300be9c6b1f8c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.929564 14809 tablet_bootstrap.cc:492] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.929629 14809 ts_tablet_manager.cc:1397] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:11.929984 14809 raft_consensus.cc:357] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.930060 14809 raft_consensus.cc:383] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.930085 14809 raft_consensus.cc:738] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.930173 14809 consensus_queue.cc:260] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.930382 14809 ts_tablet_manager.cc:1428] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.930825 14812 tablet_bootstrap.cc:492] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.931706 14812 tablet_bootstrap.cc:654] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.932200 14824 raft_consensus.cc:491] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.932264 14537 tablet_service.cc:1467] Processing CreateTablet for tablet b41d9a854bac49599d3390c15f31e0fa (DEFAULT_TABLE table=default.Table_0bdf66c6040b4709b0a6be57188b9d86 [id=c8fdb0f9f9064f3eb8d19f4adf3c439f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.932276 14824 raft_consensus.cc:513] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.932467 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b41d9a854bac49599d3390c15f31e0fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.932591 14824 leader_election.cc:290] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:11.932802 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_45b6c80c56f74125bc0ce3b968e2012b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.932821 14171 catalog_manager.cc:5526] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:11.933079 14171 catalog_manager.cc:5526] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } }
W20250115 03:06:11.933158 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_45b6c80c56f74125bc0ce3b968e2012b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.933228 14171 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet f64a668e2f4548f3bd4f2b0be32e8682 (table default.Table_a3860130ddd44dd1bf2eff20f30d374b [id=488a9a6268f64669ae17e1a67caa73a5]). Expected version 1 got 0
I20250115 03:06:11.933323 14171 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet c6cd63ac1c084a0d8f166e63b0ed85bc (table default.Table_8f3bf985bedd43918313ea7504253e48 [id=05f9fd2ebec34d2299ea59c47a74c828]). Expected version 2 got 0
I20250115 03:06:11.933485 14812 tablet_bootstrap.cc:492] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.933552 14812 ts_tablet_manager.cc:1397] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:11.933935 14812 raft_consensus.cc:357] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.934022 14812 raft_consensus.cc:383] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.934051 14812 raft_consensus.cc:738] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.934154 14812 consensus_queue.cc:260] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.934321 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "026bf50b4721424aa06c59455fe7d07e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:11.934392 14812 ts_tablet_manager.cc:1428] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.934412 14421 raft_consensus.cc:2463] T 026bf50b4721424aa06c59455fe7d07e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:11.934796 14628 leader_election.cc:304] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.934957 14858 raft_consensus.cc:491] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.934991 14824 raft_consensus.cc:2798] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:11.935030 14824 raft_consensus.cc:491] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.935062 14824 raft_consensus.cc:3054] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.935030 14858 raft_consensus.cc:513] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.935331 14858 leader_election.cc:290] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.935477 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "026bf50b4721424aa06c59455fe7d07e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:11.935563 14557 raft_consensus.cc:2463] T 026bf50b4721424aa06c59455fe7d07e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:11.935786 14824 raft_consensus.cc:513] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.936067 14824 leader_election.cc:290] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:11.936168 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a786573d49e45669a3a2ef2385afc19" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:11.936201 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a786573d49e45669a3a2ef2385afc19" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:11.936283 14421 raft_consensus.cc:2463] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:11.936283 14689 raft_consensus.cc:2463] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:11.936481 14400 tablet_service.cc:1467] Processing CreateTablet for tablet b41d9a854bac49599d3390c15f31e0fa (DEFAULT_TABLE table=default.Table_0bdf66c6040b4709b0a6be57188b9d86 [id=c8fdb0f9f9064f3eb8d19f4adf3c439f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.936532 14493 leader_election.cc:304] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.936707 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b41d9a854bac49599d3390c15f31e0fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.937050 14858 raft_consensus.cc:2798] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:11.937100 14858 raft_consensus.cc:491] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.937132 14858 raft_consensus.cc:3054] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.937330 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "026bf50b4721424aa06c59455fe7d07e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:11.937412 14421 raft_consensus.cc:3054] T 026bf50b4721424aa06c59455fe7d07e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.937950 14858 raft_consensus.cc:513] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.937953 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0d7244964d874f5eb1f508e5c0dabf1a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.938004 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_baf019e5af52496087d420b56a0f894a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.938128 14421 raft_consensus.cc:2463] T 026bf50b4721424aa06c59455fe7d07e P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
W20250115 03:06:11.938243 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0d7244964d874f5eb1f508e5c0dabf1a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.938241 14858 leader_election.cc:290] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:11.938279 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_baf019e5af52496087d420b56a0f894a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.938597 14628 leader_election.cc:304] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.938796 14824 raft_consensus.cc:2798] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:11.938872 14824 raft_consensus.cc:695] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:11.938988 14824 consensus_queue.cc:237] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.938988 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_dd9da62650c241fb9b1b74438090cf87"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.939200 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "026bf50b4721424aa06c59455fe7d07e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:11.939234 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dd9da62650c241fb9b1b74438090cf87 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.939306 14557 raft_consensus.cc:3054] T 026bf50b4721424aa06c59455fe7d07e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.939817 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a786573d49e45669a3a2ef2385afc19" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:11.939904 14421 raft_consensus.cc:3054] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.940102 14557 raft_consensus.cc:2463] T 026bf50b4721424aa06c59455fe7d07e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:11.940485 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d7dd29730eb547fd9d8de4c5cb0269f3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.940608 14421 raft_consensus.cc:2463] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
W20250115 03:06:11.940754 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d7dd29730eb547fd9d8de4c5cb0269f3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.940909 14493 leader_election.cc:304] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.941071 14858 raft_consensus.cc:2798] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:11.941133 14858 raft_consensus.cc:695] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:11.941140 14672 tablet_service.cc:1467] Processing CreateTablet for tablet b41d9a854bac49599d3390c15f31e0fa (DEFAULT_TABLE table=default.Table_0bdf66c6040b4709b0a6be57188b9d86 [id=c8fdb0f9f9064f3eb8d19f4adf3c439f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.941234 14858 consensus_queue.cc:237] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.941323 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b41d9a854bac49599d3390c15f31e0fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.941448 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_75c469057273407dbfa671e136972f30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:11.941705 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_75c469057273407dbfa671e136972f30 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.942751 14811 tablet_bootstrap.cc:492] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.942821 14811 ts_tablet_manager.cc:1397] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20250115 03:06:11.943063 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_3f71dfd17bb74691b7956682ae89fb38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.943193 14811 raft_consensus.cc:357] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.943267 14811 raft_consensus.cc:383] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.943291 14811 raft_consensus.cc:738] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
W20250115 03:06:11.943321 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3f71dfd17bb74691b7956682ae89fb38 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.943382 14811 consensus_queue.cc:260] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.943609 14811 ts_tablet_manager.cc:1428] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.943748 14811 tablet_bootstrap.cc:492] T b41d9a854bac49599d3390c15f31e0fa P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.943959 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a786573d49e45669a3a2ef2385afc19" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:11.944049 14691 raft_consensus.cc:3054] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.944547 14811 tablet_bootstrap.cc:654] T b41d9a854bac49599d3390c15f31e0fa P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.944797 14691 raft_consensus.cc:2463] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:11.945052 14812 tablet_bootstrap.cc:492] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.945837 14812 tablet_bootstrap.cc:654] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.947674 14812 tablet_bootstrap.cc:492] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.947743 14812 ts_tablet_manager.cc:1397] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:11.948084 14812 raft_consensus.cc:357] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.948155 14812 raft_consensus.cc:383] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.948179 14812 raft_consensus.cc:738] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.948266 14812 consensus_queue.cc:260] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.948472 14812 ts_tablet_manager.cc:1428] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.948823 14809 tablet_bootstrap.cc:492] T b41d9a854bac49599d3390c15f31e0fa P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.949676 14809 tablet_bootstrap.cc:654] T b41d9a854bac49599d3390c15f31e0fa P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.951418 14811 tablet_bootstrap.cc:492] T b41d9a854bac49599d3390c15f31e0fa P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.951494 14811 ts_tablet_manager.cc:1397] T b41d9a854bac49599d3390c15f31e0fa P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20250115 03:06:11.951819 14421 raft_consensus.cc:1270] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 5c0dae7aff4b44c699275b95ba639770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:11.951853 14811 raft_consensus.cc:357] T b41d9a854bac49599d3390c15f31e0fa P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.951931 14811 raft_consensus.cc:383] T b41d9a854bac49599d3390c15f31e0fa P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.951956 14811 raft_consensus.cc:738] T b41d9a854bac49599d3390c15f31e0fa P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.952044 14811 consensus_queue.cc:260] T b41d9a854bac49599d3390c15f31e0fa P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.952170 14824 consensus_queue.cc:1035] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:11.952258 14811 ts_tablet_manager.cc:1428] T b41d9a854bac49599d3390c15f31e0fa P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.954140 14557 raft_consensus.cc:1270] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer 5c0dae7aff4b44c699275b95ba639770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:11.954388 14809 tablet_bootstrap.cc:492] T b41d9a854bac49599d3390c15f31e0fa P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:11.954455 14809 ts_tablet_manager.cc:1397] T b41d9a854bac49599d3390c15f31e0fa P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:11.954811 14809 raft_consensus.cc:357] T b41d9a854bac49599d3390c15f31e0fa P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.954885 14809 raft_consensus.cc:383] T b41d9a854bac49599d3390c15f31e0fa P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.954908 14809 raft_consensus.cc:738] T b41d9a854bac49599d3390c15f31e0fa P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.955010 14809 consensus_queue.cc:260] T b41d9a854bac49599d3390c15f31e0fa P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.955226 14809 ts_tablet_manager.cc:1428] T b41d9a854bac49599d3390c15f31e0fa P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:11.955543 14824 consensus_queue.cc:1035] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:11.956086 14167 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet c6cd63ac1c084a0d8f166e63b0ed85bc (table default.Table_8f3bf985bedd43918313ea7504253e48 [id=05f9fd2ebec34d2299ea59c47a74c828]). Expected version 2 got 0
I20250115 03:06:11.958376 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_9a1065b73c014b4c9c88a1f234952379"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:11.958670 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9a1065b73c014b4c9c88a1f234952379 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.962409 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_dee96a49a9754f88b4298601381f34c5" } new_table_name: "default.Table_d68e408815964b6e93e2ed412c1f82e4" modify_external_catalogs: true
W20250115 03:06:11.963963 14167 catalog_manager.cc:5718] Failed to send AlterTable RPC for tablet c6cd63ac1c084a0d8f166e63b0ed85bc (table default.Table_8f3bf985bedd43918313ea7504253e48 [id=05f9fd2ebec34d2299ea59c47a74c828], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet c6cd63ac1c084a0d8f166e63b0ed85bc consensus state has no leader
I20250115 03:06:11.964322 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet c6cd63ac1c084a0d8f166e63b0ed85bc (table default.Table_8f3bf985bedd43918313ea7504253e48 [id=05f9fd2ebec34d2299ea59c47a74c828]). Expected version 2 got 0
I20250115 03:06:11.964385 14891 tablet.cc:1721] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:11.965374 14883 tablet.cc:1721] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:11.970817 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 35df9fe895414d4e87c7923da9703e59 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:11.970988 14893 tablet.cc:1721] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20250115 03:06:11.971772 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 35df9fe895414d4e87c7923da9703e59 because the tablet was not found. No further retry: Not found: Tablet not found: 35df9fe895414d4e87c7923da9703e59
I20250115 03:06:11.974006 14853 raft_consensus.cc:491] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.974086 14853 raft_consensus.cc:513] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.974393 14853 leader_election.cc:290] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:11.974887 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ae1c21e6b564e08b6e7eec75290b117" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:11.974992 14689 raft_consensus.cc:2463] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:11.975239 14493 leader_election.cc:304] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:11.975564 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ae1c21e6b564e08b6e7eec75290b117" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:11.975672 14421 raft_consensus.cc:2463] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:11.976054 14853 raft_consensus.cc:2798] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:11.976109 14853 raft_consensus.cc:491] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.976145 14853 raft_consensus.cc:3054] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.977095 14853 raft_consensus.cc:513] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.977389 14853 leader_election.cc:290] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
W20250115 03:06:11.977421 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3ae1c21e6b564e08b6e7eec75290b117 consensus state has no leader
I20250115 03:06:11.977965 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ae1c21e6b564e08b6e7eec75290b117" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:11.978077 14689 raft_consensus.cc:3054] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.978941 14689 raft_consensus.cc:2463] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:11.979301 14493 leader_election.cc:304] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:11.979552 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ae1c21e6b564e08b6e7eec75290b117" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:11.979651 14421 raft_consensus.cc:3054] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.980268 14163 catalog_manager.cc:5526] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:11.980510 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 108aafc0c034498bbd300be9c6b1f8c4 (table default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd]). Expected version 2 got 0
I20250115 03:06:11.980526 14421 raft_consensus.cc:2463] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:11.980636 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 4f32d3e059e44722af83c55b71f14c26 (table default.Table_c1020680eef344a49998eddac921fea5 [id=649880f1ff0d4cfda2358b408110d8dc]). Expected version 1 got 0
I20250115 03:06:11.981797 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_075e1f04509641e989ddbef0880de5d8" } new_table_name: "default.Table_44e943822f744731aaee3411219a0765" modify_external_catalogs: true
I20250115 03:06:11.981916 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_baf019e5af52496087d420b56a0f894a" } new_table_name: "default.Table_e1a261c278c04d8ea57ffe2181c5fd0d" modify_external_catalogs: true
I20250115 03:06:11.982563 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4d3670427bb242829ca73ccc85e5a887" } new_table_name: "default.Table_fa83f65f02b04ad491cee3b2a617927f" modify_external_catalogs: true
I20250115 03:06:11.984560 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 5e6367c9f60348469d8442b686b3ff3a (DEFAULT_TABLE table=default.Table_45b6c80c56f74125bc0ce3b968e2012b [id=dc4f809fea9d4bd1a8150a561088c80a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.984802 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e6367c9f60348469d8442b686b3ff3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.985011 14853 raft_consensus.cc:2798] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:11.985087 14853 raft_consensus.cc:695] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:11.985193 14853 consensus_queue.cc:237] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.986735 14811 tablet_bootstrap.cc:492] T 5e6367c9f60348469d8442b686b3ff3a P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:11.987097 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 5e6367c9f60348469d8442b686b3ff3a (DEFAULT_TABLE table=default.Table_45b6c80c56f74125bc0ce3b968e2012b [id=dc4f809fea9d4bd1a8150a561088c80a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.987337 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e6367c9f60348469d8442b686b3ff3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.987522 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 5e6367c9f60348469d8442b686b3ff3a (DEFAULT_TABLE table=default.Table_45b6c80c56f74125bc0ce3b968e2012b [id=dc4f809fea9d4bd1a8150a561088c80a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:11.987646 14811 tablet_bootstrap.cc:654] T 5e6367c9f60348469d8442b686b3ff3a P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.987746 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e6367c9f60348469d8442b686b3ff3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:11.989320 14812 tablet_bootstrap.cc:492] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:11.989562 14809 tablet_bootstrap.cc:492] T 5e6367c9f60348469d8442b686b3ff3a P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:11.989884 14853 raft_consensus.cc:491] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.989969 14853 raft_consensus.cc:513] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.990082 14812 tablet_bootstrap.cc:654] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.990490 14809 tablet_bootstrap.cc:654] T 5e6367c9f60348469d8442b686b3ff3a P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:11.990576 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35b90c4b0ef341c8a7c1bc517a1231ed" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:11.990676 14421 raft_consensus.cc:2463] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:11.991050 14493 leader_election.cc:304] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.991194 14858 raft_consensus.cc:2798] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:11.991236 14858 raft_consensus.cc:491] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.991271 14858 raft_consensus.cc:3054] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.991365 14853 leader_election.cc:290] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.991926 14811 tablet_bootstrap.cc:492] T 5e6367c9f60348469d8442b686b3ff3a P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:11.991997 14811 ts_tablet_manager.cc:1397] T 5e6367c9f60348469d8442b686b3ff3a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:11.992096 14858 raft_consensus.cc:513] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.992390 14811 raft_consensus.cc:357] T 5e6367c9f60348469d8442b686b3ff3a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.992493 14811 raft_consensus.cc:383] T 5e6367c9f60348469d8442b686b3ff3a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.992522 14811 raft_consensus.cc:738] T 5e6367c9f60348469d8442b686b3ff3a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.992611 14811 consensus_queue.cc:260] T 5e6367c9f60348469d8442b686b3ff3a P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.992836 14861 raft_consensus.cc:491] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.992892 14811 ts_tablet_manager.cc:1428] T 5e6367c9f60348469d8442b686b3ff3a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:11.993099 14861 raft_consensus.cc:513] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.993456 14861 leader_election.cc:290] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.993463 14171 catalog_manager.cc:5526] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:11.993683 14171 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 108aafc0c034498bbd300be9c6b1f8c4 (table default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd]). Expected version 2 got 0
I20250115 03:06:11.993782 14171 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 4f32d3e059e44722af83c55b71f14c26 (table default.Table_c1020680eef344a49998eddac921fea5 [id=649880f1ff0d4cfda2358b408110d8dc]). Expected version 1 got 0
W20250115 03:06:11.995255 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3ed567d482aa43ee9e24f992c346ebc3 has no consensus state
I20250115 03:06:11.995594 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c19a41bd4d149bebdd81cba3fa87d01" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:11.995694 14557 raft_consensus.cc:2463] T 3c19a41bd4d149bebdd81cba3fa87d01 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:11.995973 14861 raft_consensus.cc:491] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.996058 14812 tablet_bootstrap.cc:492] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:11.996039 14861 raft_consensus.cc:513] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.996129 14812 ts_tablet_manager.cc:1397] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:11.996301 14861 leader_election.cc:290] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.996538 14812 raft_consensus.cc:357] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:11.996632 14812 raft_consensus.cc:383] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:11.996608 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108aafc0c034498bbd300be9c6b1f8c4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:11.996644 14356 leader_election.cc:304] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.996762 14691 raft_consensus.cc:2463] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:11.996908 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108aafc0c034498bbd300be9c6b1f8c4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:11.997030 14557 raft_consensus.cc:2463] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:11.997040 14358 leader_election.cc:304] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:11.997189 14885 raft_consensus.cc:2798] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:11.997231 14885 raft_consensus.cc:491] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.997262 14885 raft_consensus.cc:3054] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.997370 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d3b0da332ade44a98e9e45c9cb294874" } modify_external_catalogs: true
I20250115 03:06:11.997462 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d3b0da332ade44a98e9e45c9cb294874" } modify_external_catalogs: true
I20250115 03:06:11.997514 14824 raft_consensus.cc:491] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:11.997578 14824 raft_consensus.cc:513] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.997870 14824 leader_election.cc:290] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:11.998100 14885 raft_consensus.cc:513] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.998200 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b41d9a854bac49599d3390c15f31e0fa" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:11.998282 14557 raft_consensus.cc:2463] T b41d9a854bac49599d3390c15f31e0fa P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:11.998387 14885 leader_election.cc:290] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.998472 14885 raft_consensus.cc:2798] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:11.998513 14885 raft_consensus.cc:491] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.998530 14626 leader_election.cc:304] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:11.998566 14885 raft_consensus.cc:3054] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.998549 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_a565af7122644be3bb6f89259eaefa36"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:11.998667 14824 raft_consensus.cc:2798] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:11.998704 14824 raft_consensus.cc:491] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:11.998737 14824 raft_consensus.cc:3054] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:11.998867 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a565af7122644be3bb6f89259eaefa36 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:11.999274 14885 raft_consensus.cc:513] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:11.999475 14163 catalog_manager.cc:5718] Failed to send AlterTable RPC for tablet 108aafc0c034498bbd300be9c6b1f8c4 (table default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 108aafc0c034498bbd300be9c6b1f8c4 consensus state has no leader
I20250115 03:06:11.999477 14824 raft_consensus.cc:513] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:11.999563 14885 leader_election.cc:290] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:11.999761 14824 leader_election.cc:290] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:11.996660 14812 raft_consensus.cc:738] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:11.999922 14812 consensus_queue.cc:260] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:12.000169 14812 ts_tablet_manager.cc:1428] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:12.000483 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b41d9a854bac49599d3390c15f31e0fa" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:12.000573 14557 raft_consensus.cc:3054] T b41d9a854bac49599d3390c15f31e0fa P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.001657 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_45b6c80c56f74125bc0ce3b968e2012b" } new_table_name: "default.Table_70d5068545204e038bfbcfe3a23acb51" modify_external_catalogs: true
I20250115 03:06:12.002251 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108aafc0c034498bbd300be9c6b1f8c4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:12.002342 14555 raft_consensus.cc:3054] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.002425 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c19a41bd4d149bebdd81cba3fa87d01" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:12.002499 14556 raft_consensus.cc:3054] T 3c19a41bd4d149bebdd81cba3fa87d01 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.003263 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "108aafc0c034498bbd300be9c6b1f8c4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:12.003360 14690 raft_consensus.cc:3054] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:11.992401 14858 leader_election.cc:290] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.004148 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35b90c4b0ef341c8a7c1bc517a1231ed" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:12.004235 14690 raft_consensus.cc:2463] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:12.004246 14692 raft_consensus.cc:2463] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:12.004428 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c19a41bd4d149bebdd81cba3fa87d01" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:12.004451 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35b90c4b0ef341c8a7c1bc517a1231ed" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:12.004509 14691 raft_consensus.cc:3054] T 3c19a41bd4d149bebdd81cba3fa87d01 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.004659 14687 raft_consensus.cc:1270] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.005311 14691 raft_consensus.cc:2463] T 3c19a41bd4d149bebdd81cba3fa87d01 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:12.005391 14557 raft_consensus.cc:2463] T b41d9a854bac49599d3390c15f31e0fa P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:12.004511 14688 raft_consensus.cc:3054] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.005673 14626 leader_election.cc:304] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:12.005882 14828 raft_consensus.cc:2798] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.005940 14828 raft_consensus.cc:695] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:12.006042 14828 consensus_queue.cc:237] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.006292 14688 raft_consensus.cc:2463] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:12.007068 14556 raft_consensus.cc:2463] T 3c19a41bd4d149bebdd81cba3fa87d01 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:12.007850 14555 raft_consensus.cc:2463] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:12.007884 14422 raft_consensus.cc:1270] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.008358 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b41d9a854bac49599d3390c15f31e0fa" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:12.008450 14420 raft_consensus.cc:3054] T b41d9a854bac49599d3390c15f31e0fa P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.009238 14493 leader_election.cc:304] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:12.009300 14420 raft_consensus.cc:2463] T b41d9a854bac49599d3390c15f31e0fa P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:12.009449 14817 raft_consensus.cc:2798] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.009413 14858 consensus_queue.cc:1035] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.009521 14817 raft_consensus.cc:695] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:12.009619 14817 consensus_queue.cc:237] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.010075 14356 leader_election.cc:304] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.010214 14356 leader_election.cc:304] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.010243 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b41d9a854bac49599d3390c15f31e0fa" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:12.010322 14422 raft_consensus.cc:2371] T b41d9a854bac49599d3390c15f31e0fa P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1. Re-sending same reply.
I20250115 03:06:12.010393 14861 raft_consensus.cc:2798] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.010473 14861 raft_consensus.cc:695] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:12.010582 14861 consensus_queue.cc:237] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.011163 14861 raft_consensus.cc:2798] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.011229 14861 raft_consensus.cc:695] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:12.011348 14861 consensus_queue.cc:237] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.014030 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 108aafc0c034498bbd300be9c6b1f8c4 (table default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd]). Expected version 2 got 0
I20250115 03:06:12.014153 14167 catalog_manager.cc:5526] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:12.014318 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 026bf50b4721424aa06c59455fe7d07e (table default.Table_6098a167e4524987bbc24d8dfa47f5e4 [id=eb34ed3c30694c39bf957a6af751a9b7]). Expected version 1 got 0
I20250115 03:06:12.015000 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c19a41bd4d149bebdd81cba3fa87d01" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:12.015112 14689 raft_consensus.cc:2371] T 3c19a41bd4d149bebdd81cba3fa87d01 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1. Re-sending same reply.
I20250115 03:06:12.017305 14853 consensus_queue.cc:1035] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.019451 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 3c19a41bd4d149bebdd81cba3fa87d01
I20250115 03:06:12.020606 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c19a41bd4d149bebdd81cba3fa87d01 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.020761 14823 tablet_replica.cc:331] T 3c19a41bd4d149bebdd81cba3fa87d01 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.020874 14823 raft_consensus.cc:2238] T 3c19a41bd4d149bebdd81cba3fa87d01 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.020984 14823 raft_consensus.cc:2267] T 3c19a41bd4d149bebdd81cba3fa87d01 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.021518 14823 ts_tablet_manager.cc:1905] T 3c19a41bd4d149bebdd81cba3fa87d01 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.021634 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c19a41bd4d149bebdd81cba3fa87d01 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.021781 14819 tablet_replica.cc:331] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.021890 14819 raft_consensus.cc:2238] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:12.022007 14819 pending_rounds.cc:62] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75: Trying to abort 1 pending ops.
I20250115 03:06:12.022037 14819 pending_rounds.cc:69] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384883756470272 op_type: NO_OP noop_request { }
I20250115 03:06:12.022082 14819 raft_consensus.cc:2883] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:12.022125 14819 raft_consensus.cc:2267] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.022159 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e376c0b790ce48278ea203fdb6bda08d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.022476 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e376c0b790ce48278ea203fdb6bda08d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.023322 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c19a41bd4d149bebdd81cba3fa87d01 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.023797 14823 ts_tablet_manager.cc:1918] T 3c19a41bd4d149bebdd81cba3fa87d01 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.023829 14820 tablet_replica.cc:331] T 3c19a41bd4d149bebdd81cba3fa87d01 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.023873 14823 log.cc:1198] T 3c19a41bd4d149bebdd81cba3fa87d01 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/3c19a41bd4d149bebdd81cba3fa87d01
I20250115 03:06:12.023950 14820 raft_consensus.cc:2238] T 3c19a41bd4d149bebdd81cba3fa87d01 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.024046 14820 raft_consensus.cc:2267] T 3c19a41bd4d149bebdd81cba3fa87d01 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.024076 14823 ts_tablet_manager.cc:1939] T 3c19a41bd4d149bebdd81cba3fa87d01 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.024482 14820 ts_tablet_manager.cc:1905] T 3c19a41bd4d149bebdd81cba3fa87d01 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.024544 14688 raft_consensus.cc:1270] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.024886 14813 consensus_queue.cc:1035] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.024950 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 3c19a41bd4d149bebdd81cba3fa87d01 (table default.Table_d3b0da332ade44a98e9e45c9cb294874 [id=d9fda51f2bed4c1ca720fe30ec97f440]) successfully deleted
I20250115 03:06:12.026262 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35b90c4b0ef341c8a7c1bc517a1231ed" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:12.026364 14421 raft_consensus.cc:3054] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.026471 14820 ts_tablet_manager.cc:1918] T 3c19a41bd4d149bebdd81cba3fa87d01 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.026523 14820 log.cc:1198] T 3c19a41bd4d149bebdd81cba3fa87d01 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/3c19a41bd4d149bebdd81cba3fa87d01
I20250115 03:06:12.026703 14820 ts_tablet_manager.cc:1939] T 3c19a41bd4d149bebdd81cba3fa87d01 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.027194 14421 raft_consensus.cc:2463] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:12.027506 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 3c19a41bd4d149bebdd81cba3fa87d01 (table default.Table_d3b0da332ade44a98e9e45c9cb294874 [id=d9fda51f2bed4c1ca720fe30ec97f440]) successfully deleted
I20250115 03:06:12.028277 14555 raft_consensus.cc:1270] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.029256 14813 consensus_queue.cc:1035] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.031949 14166 catalog_manager.cc:5526] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:12.032160 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 35b90c4b0ef341c8a7c1bc517a1231ed (table default.Table_4fc6e70f174043999eccc3ec2b20474d [id=1b4a3ef0246a4341869bf5163487cac6]). Expected version 3 got 0
I20250115 03:06:12.032279 14166 catalog_manager.cc:5526] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:12.032438 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 3ae1c21e6b564e08b6e7eec75290b117 (table default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52]). Expected version 1 got 0
I20250115 03:06:12.033489 14893 tablet.cc:1721] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20250115 03:06:12.033876 14163 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5e6367c9f60348469d8442b686b3ff3a consensus state has no leader
I20250115 03:06:12.035059 14883 tablet.cc:1721] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.037003 14891 tablet.cc:1721] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20250115 03:06:12.039826 14963 consensus_queue.cc:1159] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e8df7247ee9c43e0aa18216da2661f75" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20250115 03:06:12.040352 14819 ts_tablet_manager.cc:1905] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.042210 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_70d5068545204e038bfbcfe3a23acb51" } modify_external_catalogs: true
I20250115 03:06:12.042305 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_70d5068545204e038bfbcfe3a23acb51" } modify_external_catalogs: true
I20250115 03:06:12.042371 14819 ts_tablet_manager.cc:1918] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:12.042439 14819 log.cc:1198] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/3c19a41bd4d149bebdd81cba3fa87d01
I20250115 03:06:12.042690 14819 ts_tablet_manager.cc:1939] T 3c19a41bd4d149bebdd81cba3fa87d01 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.043319 14167 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_c1020680eef344a49998eddac921fea5 [id=649880f1ff0d4cfda2358b408110d8dc] alter complete (version 1)
I20250115 03:06:12.046904 14167 master_service.cc:946] ReplaceTablet: received request to replace tablet f64a668e2f4548f3bd4f2b0be32e8682 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.048266 14809 tablet_bootstrap.cc:492] T 5e6367c9f60348469d8442b686b3ff3a P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.048347 14809 ts_tablet_manager.cc:1397] T 5e6367c9f60348469d8442b686b3ff3a P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.059s user 0.002s sys 0.000s
I20250115 03:06:12.048840 14809 raft_consensus.cc:357] T 5e6367c9f60348469d8442b686b3ff3a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:12.048959 14809 raft_consensus.cc:383] T 5e6367c9f60348469d8442b686b3ff3a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.048985 14883 tablet.cc:1721] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.050114 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6a0bb9a04eb04a809c416bbd3f662565" } modify_external_catalogs: true
I20250115 03:06:12.050208 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6a0bb9a04eb04a809c416bbd3f662565" } modify_external_catalogs: true
I20250115 03:06:12.050287 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6098a167e4524987bbc24d8dfa47f5e4" } modify_external_catalogs: true
I20250115 03:06:12.050386 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6098a167e4524987bbc24d8dfa47f5e4" } modify_external_catalogs: true
I20250115 03:06:12.050526 14891 tablet.cc:1721] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.048992 14809 raft_consensus.cc:738] T 5e6367c9f60348469d8442b686b3ff3a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.052179 14809 consensus_queue.cc:260] T 5e6367c9f60348469d8442b686b3ff3a P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:12.052476 14809 ts_tablet_manager.cc:1428] T 5e6367c9f60348469d8442b686b3ff3a P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20250115 03:06:12.053068 14893 tablet.cc:1721] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.054595 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet 70a6241a383c461ca646a7d33911d654 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.055464 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet 3ae1c21e6b564e08b6e7eec75290b117 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.056262 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet f64a668e2f4548f3bd4f2b0be32e8682
I20250115 03:06:12.056555 14167 catalog_manager.cc:6596] ReplaceTablet: tablet f64a668e2f4548f3bd4f2b0be32e8682 deleted and replaced by tablet 99e3ff9f0b364b2f968e0faf3cace781
I20250115 03:06:12.057204 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet f64a668e2f4548f3bd4f2b0be32e8682 with delete_type TABLET_DATA_DELETED (replaced by tablet 99e3ff9f0b364b2f968e0faf3cace781) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.058104 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet f64a668e2f4548f3bd4f2b0be32e8682 with delete_type TABLET_DATA_DELETED (replaced by tablet 99e3ff9f0b364b2f968e0faf3cace781) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.059872 14399 tablet_service.cc:1467] Processing CreateTablet for tablet cc4b426be9da40a09b64f0e412536fa7 (DEFAULT_TABLE table=default.Table_d7dd29730eb547fd9d8de4c5cb0269f3 [id=37d965d8a5eb408ea3f33313bd22c152]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.060158 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc4b426be9da40a09b64f0e412536fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.060369 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 748097e809744c408f557561e5f35e85 (DEFAULT_TABLE table=default.Table_dd9da62650c241fb9b1b74438090cf87 [id=a4d048fc192e46ae9dee9635a45a1229]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.060580 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 748097e809744c408f557561e5f35e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.062456 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 66d755eaad304219a1235031b4841ba8 (DEFAULT_TABLE table=default.Table_0d7244964d874f5eb1f508e5c0dabf1a [id=15762a37e62f4e89852b2eb66a008dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.062665 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66d755eaad304219a1235031b4841ba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.062707 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 3ed567d482aa43ee9e24f992c346ebc3 (DEFAULT_TABLE table=default.Table_e1a261c278c04d8ea57ffe2181c5fd0d [id=e34781ee2864498bbc532144911630da]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.062908 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ed567d482aa43ee9e24f992c346ebc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.063742 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 12a06b7730eb40fa8e73722cf75e9dce
I20250115 03:06:12.064640 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 66a38c4ffeb44212906dbe1207f91005 (DEFAULT_TABLE table=default.Table_3f71dfd17bb74691b7956682ae89fb38 [id=f94088b0776f4be990ddcec04afbca43]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.064853 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66a38c4ffeb44212906dbe1207f91005. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.065433 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet f64a668e2f4548f3bd4f2b0be32e8682 with delete_type TABLET_DATA_DELETED (replaced by tablet 99e3ff9f0b364b2f968e0faf3cace781) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.065613 14534 tablet_service.cc:1467] Processing CreateTablet for tablet 66a38c4ffeb44212906dbe1207f91005 (DEFAULT_TABLE table=default.Table_3f71dfd17bb74691b7956682ae89fb38 [id=f94088b0776f4be990ddcec04afbca43]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.065721 14537 tablet_service.cc:1467] Processing CreateTablet for tablet cc4b426be9da40a09b64f0e412536fa7 (DEFAULT_TABLE table=default.Table_d7dd29730eb547fd9d8de4c5cb0269f3 [id=37d965d8a5eb408ea3f33313bd22c152]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.065830 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66a38c4ffeb44212906dbe1207f91005. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.065969 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc4b426be9da40a09b64f0e412536fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.066002 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 66d755eaad304219a1235031b4841ba8 (DEFAULT_TABLE table=default.Table_0d7244964d874f5eb1f508e5c0dabf1a [id=15762a37e62f4e89852b2eb66a008dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.066246 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66d755eaad304219a1235031b4841ba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.066797 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 83ac7b09047849da8f6770126c8dc033 (DEFAULT_TABLE table=default.Table_9a1065b73c014b4c9c88a1f234952379 [id=ff4681cab8ce44ceb45ed76c70c1132e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.067010 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83ac7b09047849da8f6770126c8dc033. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.067778 14811 tablet_bootstrap.cc:492] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.068210 14532 tablet_service.cc:1467] Processing CreateTablet for tablet a39beac89ce94b26a40dd6539637e6b3 (DEFAULT_TABLE table=default.Table_75c469057273407dbfa671e136972f30 [id=dc009faaff3f4763ac300bd408e78323]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.068444 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a39beac89ce94b26a40dd6539637e6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.068600 14811 tablet_bootstrap.cc:654] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.068769 14820 tablet_replica.cc:331] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.068903 14820 raft_consensus.cc:2238] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:12.069080 14820 pending_rounds.cc:62] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1: Trying to abort 1 pending ops.
I20250115 03:06:12.069114 14820 pending_rounds.cc:69] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384883238748160 op_type: NO_OP noop_request { }
I20250115 03:06:12.069137 14823 tablet_replica.cc:331] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.069168 14820 raft_consensus.cc:2883] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:12.069221 14820 raft_consensus.cc:2267] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.069257 14823 raft_consensus.cc:2238] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.069353 14823 raft_consensus.cc:2267] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.069484 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 748097e809744c408f557561e5f35e85 (DEFAULT_TABLE table=default.Table_dd9da62650c241fb9b1b74438090cf87 [id=a4d048fc192e46ae9dee9635a45a1229]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.069693 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 748097e809744c408f557561e5f35e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.069814 14531 tablet_service.cc:1467] Processing CreateTablet for tablet 3ed567d482aa43ee9e24f992c346ebc3 (DEFAULT_TABLE table=default.Table_e1a261c278c04d8ea57ffe2181c5fd0d [id=e34781ee2864498bbc532144911630da]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.070031 14531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ed567d482aa43ee9e24f992c346ebc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.070276 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 12a06b7730eb40fa8e73722cf75e9dce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.070946 14809 tablet_bootstrap.cc:492] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.071895 14824 raft_consensus.cc:491] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.071935 14809 tablet_bootstrap.cc:654] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.071941 14823 ts_tablet_manager.cc:1905] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.071970 14824 raft_consensus.cc:513] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:12.072299 14824 leader_election.cc:290] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:12.073570 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6367c9f60348469d8442b686b3ff3a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:12.073696 14422 raft_consensus.cc:2463] T 5e6367c9f60348469d8442b686b3ff3a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:12.074023 14823 ts_tablet_manager.cc:1918] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.074100 14628 leader_election.cc:304] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.074127 14819 tablet_replica.cc:331] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.074239 14819 raft_consensus.cc:2238] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.074261 14824 raft_consensus.cc:2798] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:12.074308 14824 raft_consensus.cc:491] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:12.074317 14819 raft_consensus.cc:2267] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.074348 14824 raft_consensus.cc:3054] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.075111 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 70a6241a383c461ca646a7d33911d654
I20250115 03:06:12.075178 14824 raft_consensus.cc:513] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:12.075403 14168 catalog_manager.cc:6596] ReplaceTablet: tablet 70a6241a383c461ca646a7d33911d654 deleted and replaced by tablet c6771c98e569468ea932882858981d23
I20250115 03:06:12.075510 14824 leader_election.cc:290] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:12.075865 14531 tablet_service.cc:1514] Processing DeleteTablet for tablet 70a6241a383c461ca646a7d33911d654 with delete_type TABLET_DATA_DELETED (replaced by tablet c6771c98e569468ea932882858981d23) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.075878 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 83ac7b09047849da8f6770126c8dc033 (DEFAULT_TABLE table=default.Table_9a1065b73c014b4c9c88a1f234952379 [id=ff4681cab8ce44ceb45ed76c70c1132e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.076097 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83ac7b09047849da8f6770126c8dc033. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.076159 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6367c9f60348469d8442b686b3ff3a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:12.076246 14422 raft_consensus.cc:3054] T 5e6367c9f60348469d8442b686b3ff3a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.076699 14811 tablet_bootstrap.cc:492] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.076771 14811 ts_tablet_manager.cc:1397] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:12.077036 14422 raft_consensus.cc:2463] T 5e6367c9f60348469d8442b686b3ff3a P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:12.077175 14811 raft_consensus.cc:357] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.077265 14811 raft_consensus.cc:383] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.077292 14811 raft_consensus.cc:738] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.077322 14628 leader_election.cc:304] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.077397 14811 consensus_queue.cc:260] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.077486 14824 raft_consensus.cc:2798] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.077559 14824 raft_consensus.cc:695] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:12.077653 14811 ts_tablet_manager.cc:1428] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.077688 14824 consensus_queue.cc:237] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:12.077811 14811 tablet_bootstrap.cc:492] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.078006 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6367c9f60348469d8442b686b3ff3a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:12.078109 14556 raft_consensus.cc:3054] T 5e6367c9f60348469d8442b686b3ff3a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.078673 14811 tablet_bootstrap.cc:654] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.078908 14556 raft_consensus.cc:2463] T 5e6367c9f60348469d8442b686b3ff3a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:12.062433 14398 tablet_service.cc:1467] Processing CreateTablet for tablet a39beac89ce94b26a40dd6539637e6b3 (DEFAULT_TABLE table=default.Table_75c469057273407dbfa671e136972f30 [id=dc009faaff3f4763ac300bd408e78323]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.074105 14823 log.cc:1198] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/f64a668e2f4548f3bd4f2b0be32e8682
I20250115 03:06:12.079231 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a39beac89ce94b26a40dd6539637e6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.079274 14823 ts_tablet_manager.cc:1939] T f64a668e2f4548f3bd4f2b0be32e8682 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.082722 14820 ts_tablet_manager.cc:1905] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.082767 14811 tablet_bootstrap.cc:492] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.082837 14811 ts_tablet_manager.cc:1397] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:12.083251 14811 raft_consensus.cc:357] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.083340 14811 raft_consensus.cc:383] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.083369 14811 raft_consensus.cc:738] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.083550 14811 consensus_queue.cc:260] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.084631 14820 ts_tablet_manager.cc:1918] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:12.084697 14820 log.cc:1198] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/f64a668e2f4548f3bd4f2b0be32e8682
I20250115 03:06:12.084729 14811 ts_tablet_manager.cc:1428] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:12.085029 14811 tablet_bootstrap.cc:492] T 66d755eaad304219a1235031b4841ba8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.085676 14671 tablet_service.cc:1467] Processing CreateTablet for tablet 66a38c4ffeb44212906dbe1207f91005 (DEFAULT_TABLE table=default.Table_3f71dfd17bb74691b7956682ae89fb38 [id=f94088b0776f4be990ddcec04afbca43]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.085790 14668 tablet_service.cc:1467] Processing CreateTablet for tablet 748097e809744c408f557561e5f35e85 (DEFAULT_TABLE table=default.Table_dd9da62650c241fb9b1b74438090cf87 [id=a4d048fc192e46ae9dee9635a45a1229]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.085906 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66a38c4ffeb44212906dbe1207f91005. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.086019 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 748097e809744c408f557561e5f35e85. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.086124 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 83ac7b09047849da8f6770126c8dc033 (DEFAULT_TABLE table=default.Table_9a1065b73c014b4c9c88a1f234952379 [id=ff4681cab8ce44ceb45ed76c70c1132e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.086143 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 66d755eaad304219a1235031b4841ba8 (DEFAULT_TABLE table=default.Table_0d7244964d874f5eb1f508e5c0dabf1a [id=15762a37e62f4e89852b2eb66a008dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.087747 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83ac7b09047849da8f6770126c8dc033. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.087921 14670 tablet_service.cc:1467] Processing CreateTablet for tablet cc4b426be9da40a09b64f0e412536fa7 (DEFAULT_TABLE table=default.Table_d7dd29730eb547fd9d8de4c5cb0269f3 [id=37d965d8a5eb408ea3f33313bd22c152]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.088111 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc4b426be9da40a09b64f0e412536fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.089411 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet 748097e809744c408f557561e5f35e85 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.090060 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66d755eaad304219a1235031b4841ba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.090193 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet 4f32d3e059e44722af83c55b71f14c26 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.090265 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 3ed567d482aa43ee9e24f992c346ebc3 (DEFAULT_TABLE table=default.Table_e1a261c278c04d8ea57ffe2181c5fd0d [id=e34781ee2864498bbc532144911630da]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.090492 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ed567d482aa43ee9e24f992c346ebc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.092298 14812 tablet_bootstrap.cc:492] T 748097e809744c408f557561e5f35e85 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.092445 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_75c469057273407dbfa671e136972f30" } modify_external_catalogs: true
I20250115 03:06:12.092548 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_75c469057273407dbfa671e136972f30" } modify_external_catalogs: true
I20250115 03:06:12.093030 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet f64a668e2f4548f3bd4f2b0be32e8682 (table default.Table_a3860130ddd44dd1bf2eff20f30d374b [id=488a9a6268f64669ae17e1a67caa73a5]) successfully deleted
I20250115 03:06:12.093187 14812 tablet_bootstrap.cc:654] T 748097e809744c408f557561e5f35e85 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.090027 14667 tablet_service.cc:1467] Processing CreateTablet for tablet a39beac89ce94b26a40dd6539637e6b3 (DEFAULT_TABLE table=default.Table_75c469057273407dbfa671e136972f30 [id=dc009faaff3f4763ac300bd408e78323]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.094231 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a39beac89ce94b26a40dd6539637e6b3. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:12.094206 14156 catalog_manager.cc:4945] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): alter failed for tablet f64a668e2f4548f3bd4f2b0be32e8682 (table default.Table_a3860130ddd44dd1bf2eff20f30d374b [id=488a9a6268f64669ae17e1a67caa73a5]),no further retry: Illegal state: Tablet not RUNNING: STOPPED
I20250115 03:06:12.094915 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 12a06b7730eb40fa8e73722cf75e9dce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.096606 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet 12a06b7730eb40fa8e73722cf75e9dce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.096746 14823 tablet_replica.cc:331] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.096836 14669 tablet_service.cc:1514] Processing DeleteTablet for tablet 70a6241a383c461ca646a7d33911d654 with delete_type TABLET_DATA_DELETED (replaced by tablet c6771c98e569468ea932882858981d23) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.096849 14823 raft_consensus.cc:2238] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:12.097210 14823 pending_rounds.cc:62] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 1 pending ops.
I20250115 03:06:12.097282 14823 pending_rounds.cc:69] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384883298897920 op_type: NO_OP noop_request { }
I20250115 03:06:12.097342 14823 raft_consensus.cc:2883] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:12.097397 14823 raft_consensus.cc:2267] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.097596 14809 tablet_bootstrap.cc:492] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.097666 14809 ts_tablet_manager.cc:1397] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.002s
I20250115 03:06:12.098028 14809 raft_consensus.cc:357] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.098115 14809 raft_consensus.cc:383] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.098142 14809 raft_consensus.cc:738] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.098249 14809 consensus_queue.cc:260] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.098495 14809 ts_tablet_manager.cc:1428] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:12.098474 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e6367c9f60348469d8442b686b3ff3a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.098589 14555 raft_consensus.cc:2371] T 5e6367c9f60348469d8442b686b3ff3a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1. Re-sending same reply.
I20250115 03:06:12.098654 14809 tablet_bootstrap.cc:492] T 748097e809744c408f557561e5f35e85 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.099501 14809 tablet_bootstrap.cc:654] T 748097e809744c408f557561e5f35e85 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.101099 14823 ts_tablet_manager.cc:1905] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.102797 14823 ts_tablet_manager.cc:1918] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:12.102851 14823 log.cc:1198] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/12a06b7730eb40fa8e73722cf75e9dce
I20250115 03:06:12.103073 14823 ts_tablet_manager.cc:1939] T 12a06b7730eb40fa8e73722cf75e9dce P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.103430 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 3ae1c21e6b564e08b6e7eec75290b117
I20250115 03:06:12.103535 14823 tablet_replica.cc:331] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.103638 14823 raft_consensus.cc:2238] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.103701 14169 catalog_manager.cc:6596] ReplaceTablet: tablet 3ae1c21e6b564e08b6e7eec75290b117 deleted and replaced by tablet 36bda89e64d44e708bccbcab847634a1
I20250115 03:06:12.103724 14823 raft_consensus.cc:2267] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.103936 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 026bf50b4721424aa06c59455fe7d07e
I20250115 03:06:12.104061 14531 tablet_service.cc:1467] Processing CreateTablet for tablet b4ea80ef8b3946129c8d5ba57d582504 (DEFAULT_TABLE table=default.Table_e376c0b790ce48278ea203fdb6bda08d [id=841fc23508e147aeb4f8b8f60f22c94b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.104271 14531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4ea80ef8b3946129c8d5ba57d582504. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.104296 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 99e3ff9f0b364b2f968e0faf3cace781 (DEFAULT_TABLE table=default.Table_a3860130ddd44dd1bf2eff20f30d374b [id=488a9a6268f64669ae17e1a67caa73a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.104586 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99e3ff9f0b364b2f968e0faf3cace781. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.104786 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 9c54a5df35e7422b84acc91e3a10aac3 (DEFAULT_TABLE table=default.Table_a565af7122644be3bb6f89259eaefa36 [id=fb3a764b51e74e08a281d128e21466e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.105047 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c54a5df35e7422b84acc91e3a10aac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.106119 14861 raft_consensus.cc:491] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.106194 14861 raft_consensus.cc:513] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.106510 14861 leader_election.cc:290] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:12.093647 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 3c19a41bd4d149bebdd81cba3fa87d01 (table default.Table_d3b0da332ade44a98e9e45c9cb294874 [id=d9fda51f2bed4c1ca720fe30ec97f440]) successfully deleted
I20250115 03:06:12.106853 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc4b426be9da40a09b64f0e412536fa7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:12.107400 14358 leader_election.cc:343] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:12.107705 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 12a06b7730eb40fa8e73722cf75e9dce (table default.Table_6a0bb9a04eb04a809c416bbd3f662565 [id=cf66734746ce4d72a20b286b824f69a2]) successfully deleted
I20250115 03:06:12.107815 14422 raft_consensus.cc:1270] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.107839 14688 raft_consensus.cc:1270] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.108223 14858 consensus_queue.cc:1035] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.108455 14858 consensus_queue.cc:1035] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.108556 14402 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ae1c21e6b564e08b6e7eec75290b117 with delete_type TABLET_DATA_DELETED (replaced by tablet 36bda89e64d44e708bccbcab847634a1) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.108799 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 9c54a5df35e7422b84acc91e3a10aac3 (DEFAULT_TABLE table=default.Table_a565af7122644be3bb6f89259eaefa36 [id=fb3a764b51e74e08a281d128e21466e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.108865 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 99e3ff9f0b364b2f968e0faf3cace781 (DEFAULT_TABLE table=default.Table_a3860130ddd44dd1bf2eff20f30d374b [id=488a9a6268f64669ae17e1a67caa73a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.109002 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 026bf50b4721424aa06c59455fe7d07e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.109566 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 9c54a5df35e7422b84acc91e3a10aac3 (DEFAULT_TABLE table=default.Table_a565af7122644be3bb6f89259eaefa36 [id=fb3a764b51e74e08a281d128e21466e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.109776 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c54a5df35e7422b84acc91e3a10aac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.110801 14670 tablet_service.cc:1514] Processing DeleteTablet for tablet 026bf50b4721424aa06c59455fe7d07e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.110929 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ae1c21e6b564e08b6e7eec75290b117 with delete_type TABLET_DATA_DELETED (replaced by tablet 36bda89e64d44e708bccbcab847634a1) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.111114 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 99e3ff9f0b364b2f968e0faf3cace781 (DEFAULT_TABLE table=default.Table_a3860130ddd44dd1bf2eff20f30d374b [id=488a9a6268f64669ae17e1a67caa73a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.111320 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99e3ff9f0b364b2f968e0faf3cace781. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.111658 14671 tablet_service.cc:1467] Processing CreateTablet for tablet b4ea80ef8b3946129c8d5ba57d582504 (DEFAULT_TABLE table=default.Table_e376c0b790ce48278ea203fdb6bda08d [id=841fc23508e147aeb4f8b8f60f22c94b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.111867 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4ea80ef8b3946129c8d5ba57d582504. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.112706 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet 026bf50b4721424aa06c59455fe7d07e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:12.112977 14941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20250115 03:06:12.113612 14823 ts_tablet_manager.cc:1905] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.113701 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc4b426be9da40a09b64f0e412536fa7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.113754 14812 tablet_bootstrap.cc:492] T 748097e809744c408f557561e5f35e85 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.113821 14812 ts_tablet_manager.cc:1397] T 748097e809744c408f557561e5f35e85 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20250115 03:06:12.113813 14555 raft_consensus.cc:2463] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:12.114080 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 70a6241a383c461ca646a7d33911d654 with delete_type TABLET_DATA_DELETED (replaced by tablet c6771c98e569468ea932882858981d23) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.114194 14812 raft_consensus.cc:357] T 748097e809744c408f557561e5f35e85 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.114284 14812 raft_consensus.cc:383] T 748097e809744c408f557561e5f35e85 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.114315 14812 raft_consensus.cc:738] T 748097e809744c408f557561e5f35e85 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.114406 14812 consensus_queue.cc:260] T 748097e809744c408f557561e5f35e85 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.114472 14356 leader_election.cc:304] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770
I20250115 03:06:12.114653 14812 ts_tablet_manager.cc:1428] T 748097e809744c408f557561e5f35e85 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:12.114706 14861 raft_consensus.cc:2798] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:12.114764 14861 raft_consensus.cc:491] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:12.114805 14861 raft_consensus.cc:3054] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.114822 14812 tablet_bootstrap.cc:492] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.114876 14820 ts_tablet_manager.cc:1939] T f64a668e2f4548f3bd4f2b0be32e8682 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.115291 14823 ts_tablet_manager.cc:1918] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.115345 14823 log.cc:1198] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/70a6241a383c461ca646a7d33911d654
I20250115 03:06:12.115489 14820 tablet_replica.cc:331] T 12a06b7730eb40fa8e73722cf75e9dce P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.115551 14823 ts_tablet_manager.cc:1939] T 70a6241a383c461ca646a7d33911d654 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.115630 14820 raft_consensus.cc:2238] T 12a06b7730eb40fa8e73722cf75e9dce P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.115669 14812 tablet_bootstrap.cc:654] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.115639 14861 raft_consensus.cc:513] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.115729 14820 raft_consensus.cc:2267] T 12a06b7730eb40fa8e73722cf75e9dce P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.115959 14811 tablet_bootstrap.cc:654] T 66d755eaad304219a1235031b4841ba8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.115983 14861 leader_election.cc:290] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:12.116127 14823 tablet_replica.cc:331] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.116228 14823 raft_consensus.cc:2238] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:12.116338 14823 pending_rounds.cc:62] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 1 pending ops.
I20250115 03:06:12.116365 14823 pending_rounds.cc:69] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384883462828032 op_type: NO_OP noop_request { }
I20250115 03:06:12.116379 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc4b426be9da40a09b64f0e412536fa7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:12.116660 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc4b426be9da40a09b64f0e412536fa7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:12.116705 14358 leader_election.cc:343] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:12.116410 14823 raft_consensus.cc:2883] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:12.116757 14555 raft_consensus.cc:3054] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.116814 14823 raft_consensus.cc:2267] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:12.117074 14941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250115 03:06:12.117146 14941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20250115 03:06:12.117362 14812 tablet_bootstrap.cc:492] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.117420 14812 ts_tablet_manager.cc:1397] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:12.117512 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet f64a668e2f4548f3bd4f2b0be32e8682 (table default.Table_a3860130ddd44dd1bf2eff20f30d374b [id=488a9a6268f64669ae17e1a67caa73a5]) successfully deleted
I20250115 03:06:12.117790 14812 raft_consensus.cc:357] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.117834 14555 raft_consensus.cc:2463] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:12.117879 14812 raft_consensus.cc:383] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.117905 14812 raft_consensus.cc:738] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.117995 14812 consensus_queue.cc:260] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.118197 14812 ts_tablet_manager.cc:1428] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:12.118330 14812 tablet_bootstrap.cc:492] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.118443 14356 leader_election.cc:304] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770
I20250115 03:06:12.118522 14823 ts_tablet_manager.cc:1905] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.118700 14861 raft_consensus.cc:2798] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.118782 14861 raft_consensus.cc:695] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:12.118901 14861 consensus_queue.cc:237] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.119167 14812 tablet_bootstrap.cc:654] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.119576 14819 ts_tablet_manager.cc:1905] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.120872 14812 tablet_bootstrap.cc:492] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.120980 14812 ts_tablet_manager.cc:1397] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:12.121333 14823 ts_tablet_manager.cc:1918] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:12.121393 14823 log.cc:1198] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/026bf50b4721424aa06c59455fe7d07e
I20250115 03:06:12.121358 14812 raft_consensus.cc:357] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.121439 14812 raft_consensus.cc:383] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.121466 14812 raft_consensus.cc:738] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.121570 14812 consensus_queue.cc:260] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.121791 14812 ts_tablet_manager.cc:1428] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.121841 14823 ts_tablet_manager.cc:1939] T 026bf50b4721424aa06c59455fe7d07e P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.121933 14812 tablet_bootstrap.cc:492] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.122409 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 70a6241a383c461ca646a7d33911d654 (table default.Table_44e943822f744731aaee3411219a0765 [id=f56a0de6e56441bfa620902f175feb4a]) successfully deleted
W20250115 03:06:12.122453 14941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250115 03:06:12.122505 14941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250115 03:06:12.122534 14941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250115 03:06:12.122558 14941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250115 03:06:12.122859 14820 ts_tablet_manager.cc:1905] T 12a06b7730eb40fa8e73722cf75e9dce P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.122881 14812 tablet_bootstrap.cc:654] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.123766 14885 raft_consensus.cc:491] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.123838 14885 raft_consensus.cc:513] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.124135 14885 leader_election.cc:290] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.124867 14421 raft_consensus.cc:1270] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.125135 14812 tablet_bootstrap.cc:492] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.125198 14812 ts_tablet_manager.cc:1397] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:12.125226 14820 ts_tablet_manager.cc:1918] T 12a06b7730eb40fa8e73722cf75e9dce P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.125293 14820 log.cc:1198] T 12a06b7730eb40fa8e73722cf75e9dce P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/12a06b7730eb40fa8e73722cf75e9dce
I20250115 03:06:12.125506 14820 ts_tablet_manager.cc:1939] T 12a06b7730eb40fa8e73722cf75e9dce P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.125557 14812 raft_consensus.cc:357] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.125634 14812 raft_consensus.cc:383] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.125659 14812 raft_consensus.cc:738] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.104305 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ae1c21e6b564e08b6e7eec75290b117 with delete_type TABLET_DATA_DELETED (replaced by tablet 36bda89e64d44e708bccbcab847634a1) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.125763 14812 consensus_queue.cc:260] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.125982 14812 ts_tablet_manager.cc:1428] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:12.126044 14820 tablet_replica.cc:331] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.126173 14820 raft_consensus.cc:2238] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:12.126341 14820 raft_consensus.cc:2267] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.126426 14858 consensus_queue.cc:1035] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.126557 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6cd63ac1c084a0d8f166e63b0ed85bc" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.126663 14555 raft_consensus.cc:2463] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:12.126958 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6cd63ac1c084a0d8f166e63b0ed85bc" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:12.127033 14356 leader_election.cc:304] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.127091 14689 raft_consensus.cc:2463] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:12.127174 14885 raft_consensus.cc:2798] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:12.127214 14885 raft_consensus.cc:491] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:12.127246 14885 raft_consensus.cc:3054] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.127668 14691 raft_consensus.cc:1270] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.128047 14885 raft_consensus.cc:513] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.128108 14858 consensus_queue.cc:1035] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.128346 14885 leader_election.cc:290] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.128427 14941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.186.212:37407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.13.186.254:39901,127.13.186.253:35411,127.13.186.252:41855
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.186.254:39901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.186.254
--webserver_port=37481
--never_fsync=true
--heap_profile_path=/tmp/kudu.14941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 75b75ca2f58f1a5aba69207830e3558011033d99
build type DEBUG
built by None at 15 Jan 2025 02:43:26 UTC on 24a791456cd2
build id 4181
I20250115 03:06:12.128906 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 12a06b7730eb40fa8e73722cf75e9dce (table default.Table_6a0bb9a04eb04a809c416bbd3f662565 [id=cf66734746ce4d72a20b286b824f69a2]) successfully deleted
I20250115 03:06:12.129693 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6cd63ac1c084a0d8f166e63b0ed85bc" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:12.129786 14689 raft_consensus.cc:3054] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.130292 14941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250115 03:06:12.130597 14689 raft_consensus.cc:2463] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:12.130767 14812 tablet_bootstrap.cc:492] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.130842 14358 leader_election.cc:304] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.131150 14885 raft_consensus.cc:2798] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.131227 14885 raft_consensus.cc:695] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:12.131347 14885 consensus_queue.cc:237] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.131666 14812 tablet_bootstrap.cc:654] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.132038 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6cd63ac1c084a0d8f166e63b0ed85bc" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:12.132030 14398 tablet_service.cc:1467] Processing CreateTablet for tablet b4ea80ef8b3946129c8d5ba57d582504 (DEFAULT_TABLE table=default.Table_e376c0b790ce48278ea203fdb6bda08d [id=841fc23508e147aeb4f8b8f60f22c94b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.132128 14555 raft_consensus.cc:3054] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.132299 14941 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250115 03:06:12.132416 14823 tablet_replica.cc:331] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.132519 14823 raft_consensus.cc:2238] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.132586 14823 pending_rounds.cc:62] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 2 pending ops.
I20250115 03:06:12.132604 14823 pending_rounds.cc:69] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7114384884149485568 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "3ae1c21e6b564e08b6e7eec75290b117" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_d68e408815964b6e93e2ed412c1f82e4" dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" new_extra_config { } }
I20250115 03:06:12.132777 14823 pending_rounds.cc:69] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384883652452352 op_type: NO_OP noop_request { }
I20250115 03:06:12.132822 14823 raft_consensus.cc:2883] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:12.132865 14823 raft_consensus.cc:2267] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.132895 14555 raft_consensus.cc:2463] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:12.133316 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 026bf50b4721424aa06c59455fe7d07e (table default.Table_6098a167e4524987bbc24d8dfa47f5e4 [id=eb34ed3c30694c39bf957a6af751a9b7]) successfully deleted
I20250115 03:06:12.109042 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c54a5df35e7422b84acc91e3a10aac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.135010 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99e3ff9f0b364b2f968e0faf3cace781. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.135273 14820 ts_tablet_manager.cc:1905] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.137054 14820 ts_tablet_manager.cc:1918] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.137133 14820 log.cc:1198] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/70a6241a383c461ca646a7d33911d654
I20250115 03:06:12.137253 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4ea80ef8b3946129c8d5ba57d582504. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.137346 14820 ts_tablet_manager.cc:1939] T 70a6241a383c461ca646a7d33911d654 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.137751 14820 tablet_replica.cc:331] T 026bf50b4721424aa06c59455fe7d07e P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.137866 14820 raft_consensus.cc:2238] T 026bf50b4721424aa06c59455fe7d07e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.137883 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 70a6241a383c461ca646a7d33911d654 (table default.Table_44e943822f744731aaee3411219a0765 [id=f56a0de6e56441bfa620902f175feb4a]) successfully deleted
I20250115 03:06:12.137951 14820 raft_consensus.cc:2267] T 026bf50b4721424aa06c59455fe7d07e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.138372 14820 ts_tablet_manager.cc:1905] T 026bf50b4721424aa06c59455fe7d07e P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.138876 14819 ts_tablet_manager.cc:1918] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.138935 14819 log.cc:1198] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/f64a668e2f4548f3bd4f2b0be32e8682
I20250115 03:06:12.139149 14819 ts_tablet_manager.cc:1939] T f64a668e2f4548f3bd4f2b0be32e8682 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.139448 14165 catalog_manager.cc:5526] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:12.139673 14819 tablet_replica.cc:331] T 12a06b7730eb40fa8e73722cf75e9dce P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.139691 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 108aafc0c034498bbd300be9c6b1f8c4 (table default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd]). Expected version 2 got 0
I20250115 03:06:12.139782 14819 raft_consensus.cc:2238] T 12a06b7730eb40fa8e73722cf75e9dce P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.139861 14819 raft_consensus.cc:2267] T 12a06b7730eb40fa8e73722cf75e9dce P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.140084 14820 ts_tablet_manager.cc:1918] T 026bf50b4721424aa06c59455fe7d07e P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.140139 14820 log.cc:1198] T 026bf50b4721424aa06c59455fe7d07e P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/026bf50b4721424aa06c59455fe7d07e
I20250115 03:06:12.140254 14819 ts_tablet_manager.cc:1905] T 12a06b7730eb40fa8e73722cf75e9dce P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.140306 14820 ts_tablet_manager.cc:1939] T 026bf50b4721424aa06c59455fe7d07e P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.140827 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 026bf50b4721424aa06c59455fe7d07e (table default.Table_6098a167e4524987bbc24d8dfa47f5e4 [id=eb34ed3c30694c39bf957a6af751a9b7]) successfully deleted
I20250115 03:06:12.141165 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet f64a668e2f4548f3bd4f2b0be32e8682 (table default.Table_a3860130ddd44dd1bf2eff20f30d374b [id=488a9a6268f64669ae17e1a67caa73a5]) successfully deleted
I20250115 03:06:12.141263 14820 tablet_replica.cc:331] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.141394 14820 raft_consensus.cc:2238] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:12.141513 14820 pending_rounds.cc:62] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1: Trying to abort 2 pending ops.
I20250115 03:06:12.141556 14820 pending_rounds.cc:69] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7114384884149485568 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "3ae1c21e6b564e08b6e7eec75290b117" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_d68e408815964b6e93e2ed412c1f82e4" dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" new_extra_config { } }
I20250115 03:06:12.141604 14671 tablet_service.cc:1467] Processing CreateTablet for tablet c6771c98e569468ea932882858981d23 (DEFAULT_TABLE table=default.Table_44e943822f744731aaee3411219a0765 [id=f56a0de6e56441bfa620902f175feb4a]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:12.141743 14820 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47222: Aborted: Op aborted
I20250115 03:06:12.141829 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6771c98e569468ea932882858981d23. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.141974 14819 ts_tablet_manager.cc:1918] T 12a06b7730eb40fa8e73722cf75e9dce P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.141958 14533 tablet_service.cc:1467] Processing CreateTablet for tablet c6771c98e569468ea932882858981d23 (DEFAULT_TABLE table=default.Table_44e943822f744731aaee3411219a0765 [id=f56a0de6e56441bfa620902f175feb4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.142031 14819 log.cc:1198] T 12a06b7730eb40fa8e73722cf75e9dce P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/12a06b7730eb40fa8e73722cf75e9dce
W20250115 03:06:12.142024 14156 catalog_manager.cc:4951] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): alter failed for tablet 3ae1c21e6b564e08b6e7eec75290b117 (table default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52]): Aborted: Op aborted
I20250115 03:06:12.142159 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6771c98e569468ea932882858981d23. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.142204 14819 ts_tablet_manager.cc:1939] T 12a06b7730eb40fa8e73722cf75e9dce P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.142189 14820 pending_rounds.cc:69] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384883652452352 op_type: NO_OP noop_request { }
I20250115 03:06:12.142251 14820 raft_consensus.cc:2883] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:12.142298 14820 raft_consensus.cc:2267] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.142804 14820 ts_tablet_manager.cc:1905] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.142990 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 12a06b7730eb40fa8e73722cf75e9dce (table default.Table_6a0bb9a04eb04a809c416bbd3f662565 [id=cf66734746ce4d72a20b286b824f69a2]) successfully deleted
I20250115 03:06:12.143167 14819 tablet_replica.cc:331] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.143262 14819 raft_consensus.cc:2238] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.143330 14819 pending_rounds.cc:62] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75: Trying to abort 2 pending ops.
I20250115 03:06:12.143350 14819 pending_rounds.cc:69] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7114384884149485568 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "3ae1c21e6b564e08b6e7eec75290b117" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_d68e408815964b6e93e2ed412c1f82e4" dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" new_extra_config { } }
I20250115 03:06:12.143553 14819 pending_rounds.cc:69] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384883652452352 op_type: NO_OP noop_request { }
I20250115 03:06:12.143610 14819 raft_consensus.cc:2883] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:12.143671 14819 raft_consensus.cc:2267] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.144225 14398 tablet_service.cc:1467] Processing CreateTablet for tablet c6771c98e569468ea932882858981d23 (DEFAULT_TABLE table=default.Table_44e943822f744731aaee3411219a0765 [id=f56a0de6e56441bfa620902f175feb4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.144510 14820 ts_tablet_manager.cc:1918] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.144630 14820 log.cc:1198] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/3ae1c21e6b564e08b6e7eec75290b117
I20250115 03:06:12.144883 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6771c98e569468ea932882858981d23. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.144913 14820 ts_tablet_manager.cc:1939] T 3ae1c21e6b564e08b6e7eec75290b117 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.146323 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 3ae1c21e6b564e08b6e7eec75290b117 (table default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52]) successfully deleted
W20250115 03:06:12.146679 14983 consensus_queue.cc:1159] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5c0dae7aff4b44c699275b95ba639770" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20250115 03:06:12.147256 14823 ts_tablet_manager.cc:1905] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.147239 14893 tablet.cc:1721] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:12.148597 14891 tablet.cc:1721] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:12.149091 14823 ts_tablet_manager.cc:1918] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.149166 14823 log.cc:1198] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/3ae1c21e6b564e08b6e7eec75290b117
I20250115 03:06:12.149425 14823 ts_tablet_manager.cc:1939] T 3ae1c21e6b564e08b6e7eec75290b117 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.150465 14969 tablet.cc:1721] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:12.150791 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 3ae1c21e6b564e08b6e7eec75290b117 (table default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52]) successfully deleted
W20250115 03:06:12.152873 14982 consensus_queue.cc:1159] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e8df7247ee9c43e0aa18216da2661f75" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20250115 03:06:12.153472 14819 ts_tablet_manager.cc:1905] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.153792 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_59ca04e73d1747c684c81f7563a3e183"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.154122 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_59ca04e73d1747c684c81f7563a3e183 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.155043 14819 ts_tablet_manager.cc:1918] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.155129 14819 log.cc:1198] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/3ae1c21e6b564e08b6e7eec75290b117
I20250115 03:06:12.155432 14819 ts_tablet_manager.cc:1939] T 3ae1c21e6b564e08b6e7eec75290b117 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.156112 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 3ae1c21e6b564e08b6e7eec75290b117 (table default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52]) successfully deleted
I20250115 03:06:12.156316 14819 tablet_replica.cc:331] T 026bf50b4721424aa06c59455fe7d07e P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.156419 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 748097e809744c408f557561e5f35e85
I20250115 03:06:12.156502 14819 raft_consensus.cc:2238] T 026bf50b4721424aa06c59455fe7d07e P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.156591 14819 raft_consensus.cc:2267] T 026bf50b4721424aa06c59455fe7d07e P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.156742 14168 catalog_manager.cc:6596] ReplaceTablet: tablet 748097e809744c408f557561e5f35e85 deleted and replaced by tablet 594f755feed947afba1ba8b0853d5788
W20250115 03:06:12.156837 14997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250115 03:06:12.157344 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 748097e809744c408f557561e5f35e85 with delete_type TABLET_DATA_DELETED (replaced by tablet 594f755feed947afba1ba8b0853d5788) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.157953 14819 ts_tablet_manager.cc:1905] T 026bf50b4721424aa06c59455fe7d07e P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.159117 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet a39beac89ce94b26a40dd6539637e6b3
I20250115 03:06:12.159984 14819 ts_tablet_manager.cc:1918] T 026bf50b4721424aa06c59455fe7d07e P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.160038 14819 log.cc:1198] T 026bf50b4721424aa06c59455fe7d07e P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/026bf50b4721424aa06c59455fe7d07e
I20250115 03:06:12.160142 14164 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet aa88f604011741f2b3eb2699b9328eae (table default.Table_fa83f65f02b04ad491cee3b2a617927f [id=46f096547b3a4de9a46522b978ac4b8c]). Expected version 1 got 0
I20250115 03:06:12.160199 14819 ts_tablet_manager.cc:1939] T 026bf50b4721424aa06c59455fe7d07e P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.160259 14164 catalog_manager.cc:5526] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:12.160625 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 5e6367c9f60348469d8442b686b3ff3a
I20250115 03:06:12.160774 14819 tablet_replica.cc:331] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.160869 14819 raft_consensus.cc:2238] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.160871 14812 tablet_bootstrap.cc:492] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.161000 14819 raft_consensus.cc:2267] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.161420 14812 ts_tablet_manager.cc:1397] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.031s user 0.001s sys 0.001s
I20250115 03:06:12.161484 14858 raft_consensus.cc:491] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.161559 14858 raft_consensus.cc:513] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.161633 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 026bf50b4721424aa06c59455fe7d07e (table default.Table_6098a167e4524987bbc24d8dfa47f5e4 [id=eb34ed3c30694c39bf957a6af751a9b7]) successfully deleted
I20250115 03:06:12.161895 14858 leader_election.cc:290] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.161928 14812 raft_consensus.cc:357] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.162037 14812 raft_consensus.cc:383] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.162074 14812 raft_consensus.cc:738] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.162178 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet a39beac89ce94b26a40dd6539637e6b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.162196 14812 consensus_queue.cc:260] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.162477 14812 ts_tablet_manager.cc:1428] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.162474 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46e799d700db4803b79ed46273b99df6" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:12.162623 14688 raft_consensus.cc:2463] T 46e799d700db4803b79ed46273b99df6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:12.162719 14812 tablet_bootstrap.cc:492] T a39beac89ce94b26a40dd6539637e6b3 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.162777 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e6367c9f60348469d8442b686b3ff3a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.162879 14493 leader_election.cc:304] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:12.163005 14669 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e6367c9f60348469d8442b686b3ff3a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.163033 14858 raft_consensus.cc:2798] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:12.163072 14858 raft_consensus.cc:491] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:12.163126 14858 raft_consensus.cc:3054] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.163700 14812 tablet_bootstrap.cc:654] T a39beac89ce94b26a40dd6539637e6b3 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.163940 14858 raft_consensus.cc:513] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.164255 14858 leader_election.cc:290] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.164340 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet a39beac89ce94b26a40dd6539637e6b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.164845 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e8e668fb5b3e4d0bb3a1665209c0fcad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:12.165045 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46e799d700db4803b79ed46273b99df6" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:12.165155 14422 raft_consensus.cc:2463] T 46e799d700db4803b79ed46273b99df6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
W20250115 03:06:12.165179 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e8e668fb5b3e4d0bb3a1665209c0fcad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.165342 14823 tablet_replica.cc:331] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.165452 14823 raft_consensus.cc:2238] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:12.165576 14823 pending_rounds.cc:62] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 1 pending ops.
I20250115 03:06:12.165580 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e6367c9f60348469d8442b686b3ff3a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.165599 14823 pending_rounds.cc:69] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384884031131648 op_type: NO_OP noop_request { }
I20250115 03:06:12.165649 14823 raft_consensus.cc:2883] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:12.165704 14823 raft_consensus.cc:2267] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.165961 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46e799d700db4803b79ed46273b99df6" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:12.166059 14422 raft_consensus.cc:3054] T 46e799d700db4803b79ed46273b99df6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.166004 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_94ee8224b6cd468bb2a1c15b2c2af35a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:12.166081 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46e799d700db4803b79ed46273b99df6" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:12.166163 14688 raft_consensus.cc:3054] T 46e799d700db4803b79ed46273b99df6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:12.166287 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_94ee8224b6cd468bb2a1c15b2c2af35a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.166918 14422 raft_consensus.cc:2463] T 46e799d700db4803b79ed46273b99df6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:12.167054 14688 raft_consensus.cc:2463] T 46e799d700db4803b79ed46273b99df6 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:12.167462 14823 ts_tablet_manager.cc:1905] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.167590 14493 leader_election.cc:304] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.167585 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_dd57412817d54552a349868025274901"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.167851 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dd57412817d54552a349868025274901 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.167876 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet a39beac89ce94b26a40dd6539637e6b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.167985 14858 raft_consensus.cc:2798] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.168047 14858 raft_consensus.cc:695] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:12.168155 14858 consensus_queue.cc:237] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.168980 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 748097e809744c408f557561e5f35e85 with delete_type TABLET_DATA_DELETED (replaced by tablet 594f755feed947afba1ba8b0853d5788) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.169464 14671 tablet_service.cc:1514] Processing DeleteTablet for tablet 748097e809744c408f557561e5f35e85 with delete_type TABLET_DATA_DELETED (replaced by tablet 594f755feed947afba1ba8b0853d5788) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.169616 14823 ts_tablet_manager.cc:1918] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:12.169749 14823 log.cc:1198] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/5e6367c9f60348469d8442b686b3ff3a
I20250115 03:06:12.169870 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_7a59f65551fc4aff908198db770fc111"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:12.169924 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_507ab8820cba47a0bb879411fd28f671"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:12.170046 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_37fe34c9dcd44645bce254874d61cbb5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.170147 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a59f65551fc4aff908198db770fc111 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.170220 14823 ts_tablet_manager.cc:1939] T 5e6367c9f60348469d8442b686b3ff3a P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
W20250115 03:06:12.170248 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_507ab8820cba47a0bb879411fd28f671 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:12.170292 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_37fe34c9dcd44645bce254874d61cbb5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.170850 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 5e6367c9f60348469d8442b686b3ff3a (table default.Table_70d5068545204e038bfbcfe3a23acb51 [id=dc4f809fea9d4bd1a8150a561088c80a]) successfully deleted
W20250115 03:06:12.171278 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet a39beac89ce94b26a40dd6539637e6b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a39beac89ce94b26a40dd6539637e6b3 already in progress: creating tablet
I20250115 03:06:12.171501 14823 tablet_replica.cc:331] T 748097e809744c408f557561e5f35e85 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.173235 14823 raft_consensus.cc:2238] T 748097e809744c408f557561e5f35e85 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.173380 14823 raft_consensus.cc:2267] T 748097e809744c408f557561e5f35e85 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.173856 14823 ts_tablet_manager.cc:1905] T 748097e809744c408f557561e5f35e85 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.176550 14165 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_fa83f65f02b04ad491cee3b2a617927f [id=46f096547b3a4de9a46522b978ac4b8c] alter complete (version 1)
I20250115 03:06:12.177378 14823 ts_tablet_manager.cc:1918] T 748097e809744c408f557561e5f35e85 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.177439 14823 log.cc:1198] T 748097e809744c408f557561e5f35e85 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/748097e809744c408f557561e5f35e85
I20250115 03:06:12.177623 14823 ts_tablet_manager.cc:1939] T 748097e809744c408f557561e5f35e85 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.178354 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 748097e809744c408f557561e5f35e85 (table default.Table_dd9da62650c241fb9b1b74438090cf87 [id=a4d048fc192e46ae9dee9635a45a1229]) successfully deleted
I20250115 03:06:12.178732 14811 tablet_bootstrap.cc:492] T 66d755eaad304219a1235031b4841ba8 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.178823 14811 ts_tablet_manager.cc:1397] T 66d755eaad304219a1235031b4841ba8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.094s user 0.002s sys 0.000s
I20250115 03:06:12.178901 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c19a41bd4d149bebdd81cba3fa87d01 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:11 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.179301 14811 raft_consensus.cc:357] T 66d755eaad304219a1235031b4841ba8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.179404 14811 raft_consensus.cc:383] T 66d755eaad304219a1235031b4841ba8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.179450 14811 raft_consensus.cc:738] T 66d755eaad304219a1235031b4841ba8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.179569 14811 consensus_queue.cc:260] T 66d755eaad304219a1235031b4841ba8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.179862 14811 ts_tablet_manager.cc:1428] T 66d755eaad304219a1235031b4841ba8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.179982 14820 tablet_replica.cc:331] T 5e6367c9f60348469d8442b686b3ff3a P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.180038 14811 tablet_bootstrap.cc:492] T 83ac7b09047849da8f6770126c8dc033 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.180431 14671 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e6367c9f60348469d8442b686b3ff3a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.180446 14819 ts_tablet_manager.cc:1905] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.181658 14820 raft_consensus.cc:2238] T 5e6367c9f60348469d8442b686b3ff3a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.181849 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet 70a6241a383c461ca646a7d33911d654 with delete_type TABLET_DATA_DELETED (replaced by tablet c6771c98e569468ea932882858981d23) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:12.182065 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 748097e809744c408f557561e5f35e85 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 748097e809744c408f557561e5f35e85 already in progress: creating tablet
I20250115 03:06:12.182535 14688 raft_consensus.cc:1270] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.182535 14555 raft_consensus.cc:1270] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.182888 14885 consensus_queue.cc:1035] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.183308 14811 tablet_bootstrap.cc:654] T 83ac7b09047849da8f6770126c8dc033 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.183357 14885 consensus_queue.cc:1035] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.183635 14819 ts_tablet_manager.cc:1918] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.183699 14819 log.cc:1198] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/70a6241a383c461ca646a7d33911d654
I20250115 03:06:12.183931 14819 ts_tablet_manager.cc:1939] T 70a6241a383c461ca646a7d33911d654 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.181777 14820 raft_consensus.cc:2267] T 5e6367c9f60348469d8442b686b3ff3a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.184549 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 70a6241a383c461ca646a7d33911d654 (table default.Table_44e943822f744731aaee3411219a0765 [id=f56a0de6e56441bfa620902f175feb4a]) successfully deleted
I20250115 03:06:12.185207 14809 tablet_bootstrap.cc:492] T 748097e809744c408f557561e5f35e85 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.185294 14809 ts_tablet_manager.cc:1397] T 748097e809744c408f557561e5f35e85 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.087s user 0.001s sys 0.000s
I20250115 03:06:12.185771 14809 raft_consensus.cc:357] T 748097e809744c408f557561e5f35e85 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.185873 14809 raft_consensus.cc:383] T 748097e809744c408f557561e5f35e85 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.185910 14809 raft_consensus.cc:738] T 748097e809744c408f557561e5f35e85 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.185943 14820 ts_tablet_manager.cc:1905] T 5e6367c9f60348469d8442b686b3ff3a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.186039 14809 consensus_queue.cc:260] T 748097e809744c408f557561e5f35e85 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.186316 14809 ts_tablet_manager.cc:1428] T 748097e809744c408f557561e5f35e85 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.186517 14809 tablet_bootstrap.cc:492] T 66d755eaad304219a1235031b4841ba8 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.187477 14809 tablet_bootstrap.cc:654] T 66d755eaad304219a1235031b4841ba8 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.189692 14811 tablet_bootstrap.cc:492] T 83ac7b09047849da8f6770126c8dc033 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.189671 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_3e8022382eb84ab9bae6fa5bc222cd1d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:12.189782 14811 ts_tablet_manager.cc:1397] T 83ac7b09047849da8f6770126c8dc033 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.001s
W20250115 03:06:12.189989 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3e8022382eb84ab9bae6fa5bc222cd1d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.190264 14811 raft_consensus.cc:357] T 83ac7b09047849da8f6770126c8dc033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.190344 14820 ts_tablet_manager.cc:1918] T 5e6367c9f60348469d8442b686b3ff3a P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.190358 14811 raft_consensus.cc:383] T 83ac7b09047849da8f6770126c8dc033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.190392 14811 raft_consensus.cc:738] T 83ac7b09047849da8f6770126c8dc033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.190398 14820 log.cc:1198] T 5e6367c9f60348469d8442b686b3ff3a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/5e6367c9f60348469d8442b686b3ff3a
I20250115 03:06:12.190529 14811 consensus_queue.cc:260] T 83ac7b09047849da8f6770126c8dc033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.190762 14820 ts_tablet_manager.cc:1939] T 5e6367c9f60348469d8442b686b3ff3a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.190807 14811 ts_tablet_manager.cc:1428] T 83ac7b09047849da8f6770126c8dc033 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.190989 14811 tablet_bootstrap.cc:492] T a39beac89ce94b26a40dd6539637e6b3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.191900 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_9a1065b73c014b4c9c88a1f234952379" } new_table_name: "default.Table_ee5faa8cd90f40199372322d90c5a204" modify_external_catalogs: true
I20250115 03:06:12.191972 14811 tablet_bootstrap.cc:654] T a39beac89ce94b26a40dd6539637e6b3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:12.192226 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet a39beac89ce94b26a40dd6539637e6b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a39beac89ce94b26a40dd6539637e6b3 already in progress: creating tablet
I20250115 03:06:12.192642 14819 tablet_replica.cc:331] T 5e6367c9f60348469d8442b686b3ff3a P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
W20250115 03:06:12.193395 14158 catalog_manager.cc:4839] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 5e6367c9f60348469d8442b686b3ff3a because the tablet was not found. No further retry: Not found: Tablet not found: 5e6367c9f60348469d8442b686b3ff3a
I20250115 03:06:12.193603 14812 tablet_bootstrap.cc:492] T a39beac89ce94b26a40dd6539637e6b3 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.193698 14812 ts_tablet_manager.cc:1397] T a39beac89ce94b26a40dd6539637e6b3 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.031s user 0.001s sys 0.000s
I20250115 03:06:12.194167 14812 raft_consensus.cc:357] T a39beac89ce94b26a40dd6539637e6b3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.194270 14812 raft_consensus.cc:383] T a39beac89ce94b26a40dd6539637e6b3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.194319 14812 raft_consensus.cc:738] T a39beac89ce94b26a40dd6539637e6b3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.194449 14812 consensus_queue.cc:260] T a39beac89ce94b26a40dd6539637e6b3 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.194770 14812 ts_tablet_manager.cc:1428] T a39beac89ce94b26a40dd6539637e6b3 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20250115 03:06:12.194802 14158 catalog_manager.cc:4839] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 70a6241a383c461ca646a7d33911d654 because the tablet was not found. No further retry: Not found: Tablet not found: 70a6241a383c461ca646a7d33911d654
I20250115 03:06:12.194943 14812 tablet_bootstrap.cc:492] T 66a38c4ffeb44212906dbe1207f91005 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.195851 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e8e668fb5b3e4d0bb3a1665209c0fcad" } new_table_name: "default.Table_3c64fb94bd9c4890989ce6f63525a733" modify_external_catalogs: true
I20250115 03:06:12.195996 14812 tablet_bootstrap.cc:654] T 66a38c4ffeb44212906dbe1207f91005 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.196273 14891 tablet.cc:1721] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:12.197047 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 5e6367c9f60348469d8442b686b3ff3a (table default.Table_70d5068545204e038bfbcfe3a23acb51 [id=dc4f809fea9d4bd1a8150a561088c80a]) successfully deleted
W20250115 03:06:12.197190 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet a39beac89ce94b26a40dd6539637e6b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a39beac89ce94b26a40dd6539637e6b3 already in progress: creating tablet
I20250115 03:06:12.197360 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_37fe34c9dcd44645bce254874d61cbb5" } new_table_name: "default.Table_66cd278959e849dfaf7c9fc9e604d46a" modify_external_catalogs: true
I20250115 03:06:12.198794 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a3860130ddd44dd1bf2eff20f30d374b" } new_table_name: "default.Table_0dd17013f6334efe97618f0b520512c5" modify_external_catalogs: true
I20250115 03:06:12.199425 14812 tablet_bootstrap.cc:492] T 66a38c4ffeb44212906dbe1207f91005 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.199501 14812 ts_tablet_manager.cc:1397] T 66a38c4ffeb44212906dbe1207f91005 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:12.195339 14969 tablet.cc:1721] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:12.199779 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 4f32d3e059e44722af83c55b71f14c26
I20250115 03:06:12.199941 14812 raft_consensus.cc:357] T 66a38c4ffeb44212906dbe1207f91005 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.200022 14163 catalog_manager.cc:6596] ReplaceTablet: tablet 4f32d3e059e44722af83c55b71f14c26 deleted and replaced by tablet fa7043f411374111b4c81467260fc6cd
I20250115 03:06:12.200034 14812 raft_consensus.cc:383] T 66a38c4ffeb44212906dbe1207f91005 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.200065 14812 raft_consensus.cc:738] T 66a38c4ffeb44212906dbe1207f91005 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.200177 14812 consensus_queue.cc:260] T 66a38c4ffeb44212906dbe1207f91005 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.200431 14812 ts_tablet_manager.cc:1428] T 66a38c4ffeb44212906dbe1207f91005 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.200599 14812 tablet_bootstrap.cc:492] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.200798 14824 raft_consensus.cc:491] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.200867 14824 raft_consensus.cc:513] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.201188 14824 leader_election.cc:290] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:12.201625 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f32d3e059e44722af83c55b71f14c26 with delete_type TABLET_DATA_DELETED (replaced by tablet fa7043f411374111b4c81467260fc6cd) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.201659 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d755eaad304219a1235031b4841ba8" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.201750 14556 raft_consensus.cc:2463] T 66d755eaad304219a1235031b4841ba8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:12.201771 14820 tablet_replica.cc:331] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.201864 14820 raft_consensus.cc:2238] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.201957 14820 raft_consensus.cc:2267] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.202189 14819 raft_consensus.cc:2238] T 5e6367c9f60348469d8442b686b3ff3a P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.202308 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d755eaad304219a1235031b4841ba8" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:12.202946 14626 leader_election.cc:304] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:12.203095 14828 raft_consensus.cc:2798] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:12.203147 14828 raft_consensus.cc:491] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:12.203183 14828 raft_consensus.cc:3054] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.204069 14828 raft_consensus.cc:513] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.204383 14828 leader_election.cc:290] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:12.204439 14812 tablet_bootstrap.cc:654] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:12.203104 14628 leader_election.cc:343] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:12.205036 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d755eaad304219a1235031b4841ba8" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:12.205147 14164 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 3ed567d482aa43ee9e24f992c346ebc3 (table default.Table_e1a261c278c04d8ea57ffe2181c5fd0d [id=e34781ee2864498bbc532144911630da]). Expected version 1 got 0
W20250115 03:06:12.205451 14628 leader_election.cc:343] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:12.205775 14171 catalog_manager.cc:5526] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:12.206048 14171 catalog_manager.cc:5526] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:12.206202 14171 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet c6cd63ac1c084a0d8f166e63b0ed85bc (table default.Table_8f3bf985bedd43918313ea7504253e48 [id=05f9fd2ebec34d2299ea59c47a74c828]). Expected version 2 got 0
I20250115 03:06:12.206593 14170 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_4fc6e70f174043999eccc3ec2b20474d [id=1b4a3ef0246a4341869bf5163487cac6] alter complete (version 3)
I20250115 03:06:12.206957 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66d755eaad304219a1235031b4841ba8" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:12.207055 14556 raft_consensus.cc:3054] T 66d755eaad304219a1235031b4841ba8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.207878 14556 raft_consensus.cc:2463] T 66d755eaad304219a1235031b4841ba8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:12.208230 14626 leader_election.cc:304] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:12.208802 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f32d3e059e44722af83c55b71f14c26 with delete_type TABLET_DATA_DELETED (replaced by tablet fa7043f411374111b4c81467260fc6cd) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.208977 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f32d3e059e44722af83c55b71f14c26 with delete_type TABLET_DATA_DELETED (replaced by tablet fa7043f411374111b4c81467260fc6cd) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.208956 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 36bda89e64d44e708bccbcab847634a1 (DEFAULT_TABLE table=default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.209187 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36bda89e64d44e708bccbcab847634a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.209336 14671 tablet_service.cc:1467] Processing CreateTablet for tablet 36bda89e64d44e708bccbcab847634a1 (DEFAULT_TABLE table=default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.209543 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36bda89e64d44e708bccbcab847634a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.210383 14809 tablet_bootstrap.cc:492] T 66d755eaad304219a1235031b4841ba8 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.210484 14809 ts_tablet_manager.cc:1397] T 66d755eaad304219a1235031b4841ba8 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20250115 03:06:12.210970 14811 tablet_bootstrap.cc:492] T a39beac89ce94b26a40dd6539637e6b3 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.210953 14809 raft_consensus.cc:357] T 66d755eaad304219a1235031b4841ba8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.211041 14811 ts_tablet_manager.cc:1397] T a39beac89ce94b26a40dd6539637e6b3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.000s
I20250115 03:06:12.211050 14809 raft_consensus.cc:383] T 66d755eaad304219a1235031b4841ba8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.211084 14809 raft_consensus.cc:738] T 66d755eaad304219a1235031b4841ba8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.211205 14809 consensus_queue.cc:260] T 66d755eaad304219a1235031b4841ba8 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.211265 14828 raft_consensus.cc:2798] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.211361 14828 raft_consensus.cc:695] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:12.211494 14809 ts_tablet_manager.cc:1428] T 66d755eaad304219a1235031b4841ba8 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.211480 14828 consensus_queue.cc:237] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.211504 14811 raft_consensus.cc:357] T a39beac89ce94b26a40dd6539637e6b3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.211591 14811 raft_consensus.cc:383] T a39beac89ce94b26a40dd6539637e6b3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.211622 14811 raft_consensus.cc:738] T a39beac89ce94b26a40dd6539637e6b3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.211683 14809 tablet_bootstrap.cc:492] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.211745 14811 consensus_queue.cc:260] T a39beac89ce94b26a40dd6539637e6b3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.211808 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 36bda89e64d44e708bccbcab847634a1 (DEFAULT_TABLE table=default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.212008 14811 ts_tablet_manager.cc:1428] T a39beac89ce94b26a40dd6539637e6b3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.212020 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36bda89e64d44e708bccbcab847634a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.212194 14811 tablet_bootstrap.cc:492] T 748097e809744c408f557561e5f35e85 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.202296 14819 raft_consensus.cc:2267] T 5e6367c9f60348469d8442b686b3ff3a P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.212679 14809 tablet_bootstrap.cc:654] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.212706 14823 tablet_replica.cc:331] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.212822 14823 raft_consensus.cc:2238] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:12.213001 14823 raft_consensus.cc:2267] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.213274 14811 tablet_bootstrap.cc:654] T 748097e809744c408f557561e5f35e85 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.213464 14823 ts_tablet_manager.cc:1905] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.213933 14893 tablet.cc:1721] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:12.215715 14812 tablet_bootstrap.cc:492] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.215795 14812 ts_tablet_manager.cc:1397] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
W20250115 03:06:12.216296 14994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250115 03:06:12.216265 14812 raft_consensus.cc:357] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.216358 14812 raft_consensus.cc:383] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.216390 14812 raft_consensus.cc:738] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.216511 14812 consensus_queue.cc:260] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.216758 14823 ts_tablet_manager.cc:1918] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.216796 14812 ts_tablet_manager.cc:1428] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.217010 14812 tablet_bootstrap.cc:492] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:12.217861 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 83ac7b09047849da8f6770126c8dc033 consensus state has no leader
I20250115 03:06:12.217890 14812 tablet_bootstrap.cc:654] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.218704 14823 log.cc:1198] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/4f32d3e059e44722af83c55b71f14c26
I20250115 03:06:12.219130 14823 ts_tablet_manager.cc:1939] T 4f32d3e059e44722af83c55b71f14c26 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.219117 14819 ts_tablet_manager.cc:1905] T 5e6367c9f60348469d8442b686b3ff3a P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.219966 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 4f32d3e059e44722af83c55b71f14c26 (table default.Table_c1020680eef344a49998eddac921fea5 [id=649880f1ff0d4cfda2358b408110d8dc]) successfully deleted
W20250115 03:06:12.221021 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c has no consensus state
I20250115 03:06:12.221066 14824 raft_consensus.cc:491] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.221143 14824 raft_consensus.cc:513] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.221115 14819 ts_tablet_manager.cc:1918] T 5e6367c9f60348469d8442b686b3ff3a P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.221501 14824 leader_election.cc:290] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:12.221547 14819 log.cc:1198] T 5e6367c9f60348469d8442b686b3ff3a P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/5e6367c9f60348469d8442b686b3ff3a
I20250115 03:06:12.221774 14819 ts_tablet_manager.cc:1939] T 5e6367c9f60348469d8442b686b3ff3a P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.222079 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc4b426be9da40a09b64f0e412536fa7" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.222225 14555 raft_consensus.cc:2388] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c0dae7aff4b44c699275b95ba639770 in current term 1: Already voted for candidate e8df7247ee9c43e0aa18216da2661f75 in this term.
I20250115 03:06:12.222343 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc4b426be9da40a09b64f0e412536fa7" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:12.222398 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 5e6367c9f60348469d8442b686b3ff3a (table default.Table_70d5068545204e038bfbcfe3a23acb51 [id=dc4f809fea9d4bd1a8150a561088c80a]) successfully deleted
W20250115 03:06:12.222700 14162 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 99e3ff9f0b364b2f968e0faf3cace781 consensus state has no leader
W20250115 03:06:12.222745 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7a78c3eca7d542d7a71f9a33e8585c18 has no consensus state
I20250115 03:06:12.222883 14628 leader_election.cc:304] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:12.223100 14824 raft_consensus.cc:3054] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.223717 14812 tablet_bootstrap.cc:492] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.223805 14812 ts_tablet_manager.cc:1397] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:12.223829 14820 ts_tablet_manager.cc:1905] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.223994 14824 raft_consensus.cc:2743] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:12.224264 14812 raft_consensus.cc:357] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.224366 14812 raft_consensus.cc:383] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.224411 14812 raft_consensus.cc:738] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.224486 14809 tablet_bootstrap.cc:492] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.224570 14809 ts_tablet_manager.cc:1397] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20250115 03:06:12.225031 14809 raft_consensus.cc:357] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.224692 14812 consensus_queue.cc:260] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.225138 14809 raft_consensus.cc:383] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.225172 14809 raft_consensus.cc:738] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.225284 14809 consensus_queue.cc:260] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.225371 14812 ts_tablet_manager.cc:1428] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:12.225577 14809 ts_tablet_manager.cc:1428] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.225590 14820 ts_tablet_manager.cc:1918] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.225647 14820 log.cc:1198] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/4f32d3e059e44722af83c55b71f14c26
I20250115 03:06:12.225783 14809 tablet_bootstrap.cc:492] T 66a38c4ffeb44212906dbe1207f91005 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.225880 14820 ts_tablet_manager.cc:1939] T 4f32d3e059e44722af83c55b71f14c26 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.225559 14812 tablet_bootstrap.cc:492] T b4ea80ef8b3946129c8d5ba57d582504 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.227016 14809 tablet_bootstrap.cc:654] T 66a38c4ffeb44212906dbe1207f91005 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.227092 14858 raft_consensus.cc:491] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.227159 14858 raft_consensus.cc:513] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.227453 14858 leader_election.cc:290] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:12.227845 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a38c4ffeb44212906dbe1207f91005" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:12.227942 14688 raft_consensus.cc:2463] T 66a38c4ffeb44212906dbe1207f91005 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:12.228192 14493 leader_election.cc:304] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:12.228277 14812 tablet_bootstrap.cc:654] T b4ea80ef8b3946129c8d5ba57d582504 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.229019 14858 raft_consensus.cc:2798] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:12.229068 14858 raft_consensus.cc:491] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:12.229102 14858 raft_consensus.cc:3054] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.229076 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 4f32d3e059e44722af83c55b71f14c26 (table default.Table_c1020680eef344a49998eddac921fea5 [id=649880f1ff0d4cfda2358b408110d8dc]) successfully deleted
I20250115 03:06:12.229539 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b51a11d524dc4f7489148985667c7bcc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.229899 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b51a11d524dc4f7489148985667c7bcc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.229868 14858 raft_consensus.cc:513] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.230166 14858 leader_election.cc:290] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:12.230330 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_7a59f65551fc4aff908198db770fc111" } modify_external_catalogs: true
I20250115 03:06:12.230413 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_7a59f65551fc4aff908198db770fc111" } modify_external_catalogs: true
I20250115 03:06:12.230691 14812 tablet_bootstrap.cc:492] T b4ea80ef8b3946129c8d5ba57d582504 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.230764 14812 ts_tablet_manager.cc:1397] T b4ea80ef8b3946129c8d5ba57d582504 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
W20250115 03:06:12.230903 14995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250115 03:06:12.230962 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a38c4ffeb44212906dbe1207f91005" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:12.231050 14688 raft_consensus.cc:3054] T 66a38c4ffeb44212906dbe1207f91005 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.231230 14812 raft_consensus.cc:357] T b4ea80ef8b3946129c8d5ba57d582504 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.231344 14812 raft_consensus.cc:383] T b4ea80ef8b3946129c8d5ba57d582504 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.231379 14812 raft_consensus.cc:738] T b4ea80ef8b3946129c8d5ba57d582504 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.231552 14811 tablet_bootstrap.cc:492] T 748097e809744c408f557561e5f35e85 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.231506 14812 consensus_queue.cc:260] T b4ea80ef8b3946129c8d5ba57d582504 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.231626 14811 ts_tablet_manager.cc:1397] T 748097e809744c408f557561e5f35e85 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20250115 03:06:12.231743 14688 raft_consensus.cc:2463] T 66a38c4ffeb44212906dbe1207f91005 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:12.231846 14812 ts_tablet_manager.cc:1428] T b4ea80ef8b3946129c8d5ba57d582504 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.232003 14812 tablet_bootstrap.cc:492] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.232106 14811 raft_consensus.cc:357] T 748097e809744c408f557561e5f35e85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.232203 14811 raft_consensus.cc:383] T 748097e809744c408f557561e5f35e85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.232234 14811 raft_consensus.cc:738] T 748097e809744c408f557561e5f35e85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.232344 14811 consensus_queue.cc:260] T 748097e809744c408f557561e5f35e85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.232369 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a38c4ffeb44212906dbe1207f91005" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:12.232589 14811 ts_tablet_manager.cc:1428] T 748097e809744c408f557561e5f35e85 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:12.232759 14811 tablet_bootstrap.cc:492] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.232872 14812 tablet_bootstrap.cc:654] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:12.232861 14493 leader_election.cc:343] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:12.233632 14811 tablet_bootstrap.cc:654] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.234047 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a38c4ffeb44212906dbe1207f91005" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:12.234413 14493 leader_election.cc:343] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:12.235304 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e6367c9f60348469d8442b686b3ff3a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.235417 14493 leader_election.cc:304] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:12.235587 14858 raft_consensus.cc:2798] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.235667 14858 raft_consensus.cc:695] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:12.235785 14858 consensus_queue.cc:237] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
W20250115 03:06:12.236126 14156 catalog_manager.cc:4945] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): alter failed for tablet 3ae1c21e6b564e08b6e7eec75290b117 (table default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52]),no further retry: Not found: Tablet not found: 3ae1c21e6b564e08b6e7eec75290b117
I20250115 03:06:12.237483 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_1132d16f9e04448689aab85d1952013b" } new_table_name: "default.Table_a86904006b724cb980fe7575e6b6c13a" modify_external_catalogs: true
I20250115 03:06:12.236687 14819 tablet_replica.cc:331] T 748097e809744c408f557561e5f35e85 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.238044 14819 raft_consensus.cc:2238] T 748097e809744c408f557561e5f35e85 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.238620 14819 raft_consensus.cc:2267] T 748097e809744c408f557561e5f35e85 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.238596 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_fadcbb64abcd4a1b812defb8ed9d99f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:12.238798 14811 tablet_bootstrap.cc:492] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.238883 14811 ts_tablet_manager.cc:1397] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
W20250115 03:06:12.238895 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fadcbb64abcd4a1b812defb8ed9d99f5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.239068 14819 ts_tablet_manager.cc:1905] T 748097e809744c408f557561e5f35e85 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.239331 14811 raft_consensus.cc:357] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.239408 14811 raft_consensus.cc:383] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.239431 14811 raft_consensus.cc:738] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.239552 14811 consensus_queue.cc:260] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.239801 14811 ts_tablet_manager.cc:1428] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.239960 14811 tablet_bootstrap.cc:492] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.240763 14811 tablet_bootstrap.cc:654] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.241684 14811 tablet_bootstrap.cc:492] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.241755 14811 ts_tablet_manager.cc:1397] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:12.242266 14811 raft_consensus.cc:357] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.242350 14811 raft_consensus.cc:383] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.242380 14811 raft_consensus.cc:738] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.242518 14811 consensus_queue.cc:260] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.242749 14941 server_base.cc:1034] running on GCE node
I20250115 03:06:12.242795 14811 ts_tablet_manager.cc:1428] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:12.242949 14811 tablet_bootstrap.cc:492] T 99e3ff9f0b364b2f968e0faf3cace781 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.243779 14811 tablet_bootstrap.cc:654] T 99e3ff9f0b364b2f968e0faf3cace781 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.244048 14941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250115 03:06:12.244915 14811 tablet_bootstrap.cc:492] T 99e3ff9f0b364b2f968e0faf3cace781 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.245090 14811 ts_tablet_manager.cc:1397] T 99e3ff9f0b364b2f968e0faf3cace781 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20250115 03:06:12.245182 14812 tablet_bootstrap.cc:492] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.245249 14941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250115 03:06:12.245271 14812 ts_tablet_manager.cc:1397] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20250115 03:06:12.245692 14812 raft_consensus.cc:357] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.245795 14812 raft_consensus.cc:383] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.245831 14812 raft_consensus.cc:738] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.245848 14811 raft_consensus.cc:357] T 99e3ff9f0b364b2f968e0faf3cace781 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.245941 14812 consensus_queue.cc:260] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.246017 14811 raft_consensus.cc:383] T 99e3ff9f0b364b2f968e0faf3cace781 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.246059 14811 raft_consensus.cc:738] T 99e3ff9f0b364b2f968e0faf3cace781 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.246204 14812 ts_tablet_manager.cc:1428] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.246377 14812 tablet_bootstrap.cc:492] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.246433 14811 consensus_queue.cc:260] T 99e3ff9f0b364b2f968e0faf3cace781 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.247242 14812 tablet_bootstrap.cc:654] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.247681 14809 tablet_bootstrap.cc:492] T 66a38c4ffeb44212906dbe1207f91005 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.247740 14824 raft_consensus.cc:491] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.247773 14809 ts_tablet_manager.cc:1397] T 66a38c4ffeb44212906dbe1207f91005 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20250115 03:06:12.247802 14824 raft_consensus.cc:513] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.248106 14824 leader_election.cc:290] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:12.248173 14811 ts_tablet_manager.cc:1428] T 99e3ff9f0b364b2f968e0faf3cace781 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20250115 03:06:12.248183 14809 raft_consensus.cc:357] T 66a38c4ffeb44212906dbe1207f91005 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.248267 14809 raft_consensus.cc:383] T 66a38c4ffeb44212906dbe1207f91005 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.248298 14809 raft_consensus.cc:738] T 66a38c4ffeb44212906dbe1207f91005 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.248312 14811 tablet_bootstrap.cc:492] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.248405 14809 consensus_queue.cc:260] T 66a38c4ffeb44212906dbe1207f91005 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.248555 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83ac7b09047849da8f6770126c8dc033" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.248661 14555 raft_consensus.cc:2463] T 83ac7b09047849da8f6770126c8dc033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:12.249374 14809 ts_tablet_manager.cc:1428] T 66a38c4ffeb44212906dbe1207f91005 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:12.249671 14811 tablet_bootstrap.cc:654] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.250303 14809 tablet_bootstrap.cc:492] T 83ac7b09047849da8f6770126c8dc033 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.250612 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83ac7b09047849da8f6770126c8dc033" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:12.250934 14628 leader_election.cc:343] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:12.251080 14626 leader_election.cc:304] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:12.251215 14828 raft_consensus.cc:2798] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:12.251259 14828 raft_consensus.cc:491] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:12.251298 14828 raft_consensus.cc:3054] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.251645 14809 tablet_bootstrap.cc:654] T 83ac7b09047849da8f6770126c8dc033 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.252110 14828 raft_consensus.cc:513] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.252429 14828 leader_election.cc:290] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:12.252714 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83ac7b09047849da8f6770126c8dc033" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:12.252737 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83ac7b09047849da8f6770126c8dc033" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:12.252827 14555 raft_consensus.cc:3054] T 83ac7b09047849da8f6770126c8dc033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.252995 14819 ts_tablet_manager.cc:1918] T 748097e809744c408f557561e5f35e85 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.253057 14819 log.cc:1198] T 748097e809744c408f557561e5f35e85 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/748097e809744c408f557561e5f35e85
W20250115 03:06:12.253136 14628 leader_election.cc:343] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:12.253240 14819 ts_tablet_manager.cc:1939] T 748097e809744c408f557561e5f35e85 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.253721 14555 raft_consensus.cc:2463] T 83ac7b09047849da8f6770126c8dc033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:12.253790 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 748097e809744c408f557561e5f35e85 (table default.Table_dd9da62650c241fb9b1b74438090cf87 [id=a4d048fc192e46ae9dee9635a45a1229]) successfully deleted
I20250115 03:06:12.254037 14626 leader_election.cc:304] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:12.254738 14812 tablet_bootstrap.cc:492] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.254810 14812 ts_tablet_manager.cc:1397] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
W20250115 03:06:12.254802 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 3c19a41bd4d149bebdd81cba3fa87d01 because the tablet was not found. No further retry: Not found: Tablet not found: 3c19a41bd4d149bebdd81cba3fa87d01
I20250115 03:06:12.254983 14819 tablet_replica.cc:331] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.255146 14819 raft_consensus.cc:2238] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.255208 14812 raft_consensus.cc:357] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.255297 14812 raft_consensus.cc:383] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.255331 14812 raft_consensus.cc:738] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.255404 14819 raft_consensus.cc:2267] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.255432 14812 consensus_queue.cc:260] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.255697 14812 ts_tablet_manager.cc:1428] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.256008 14819 ts_tablet_manager.cc:1905] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.256812 14688 raft_consensus.cc:1270] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.257226 14851 consensus_queue.cc:1035] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.257663 14166 catalog_manager.cc:5526] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:12.257905 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 83ac7b09047849da8f6770126c8dc033 (table default.Table_ee5faa8cd90f40199372322d90c5a204 [id=ff4681cab8ce44ceb45ed76c70c1132e]). Expected version 1 got 0
I20250115 03:06:12.257997 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 108aafc0c034498bbd300be9c6b1f8c4 (table default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd]). Expected version 2 got 0
I20250115 03:06:12.258978 14809 tablet_bootstrap.cc:492] T 83ac7b09047849da8f6770126c8dc033 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.259066 14809 ts_tablet_manager.cc:1397] T 83ac7b09047849da8f6770126c8dc033 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20250115 03:06:12.259472 14809 raft_consensus.cc:357] T 83ac7b09047849da8f6770126c8dc033 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.259567 14809 raft_consensus.cc:383] T 83ac7b09047849da8f6770126c8dc033 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.259599 14809 raft_consensus.cc:738] T 83ac7b09047849da8f6770126c8dc033 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.259712 14809 consensus_queue.cc:260] T 83ac7b09047849da8f6770126c8dc033 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.260031 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 21b146e38bb345fca7a9a159983db68f (DEFAULT_TABLE table=default.Table_7a59f65551fc4aff908198db770fc111 [id=7a9994216f8d437491e6b24c201e9422]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.260068 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 7a78c3eca7d542d7a71f9a33e8585c18 (DEFAULT_TABLE table=default.Table_66cd278959e849dfaf7c9fc9e604d46a [id=6ecde174f5dc422387c42ec4e2f3a053]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.260210 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c (DEFAULT_TABLE table=default.Table_3c64fb94bd9c4890989ce6f63525a733 [id=f8ea91c4841b470db1b9f05fa47244fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.260260 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21b146e38bb345fca7a9a159983db68f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.260280 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a78c3eca7d542d7a71f9a33e8585c18. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.260421 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.260612 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 0c8dc386a6b4468d920dbfab906e1366 (DEFAULT_TABLE table=default.Table_59ca04e73d1747c684c81f7563a3e183 [id=6d1418f7a3b64aebace43d4478f10915]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.260672 14893 tablet.cc:1721] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.260828 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c8dc386a6b4468d920dbfab906e1366. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.261615 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 21b146e38bb345fca7a9a159983db68f (DEFAULT_TABLE table=default.Table_7a59f65551fc4aff908198db770fc111 [id=7a9994216f8d437491e6b24c201e9422]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.261824 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21b146e38bb345fca7a9a159983db68f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.261873 14671 tablet_service.cc:1467] Processing CreateTablet for tablet 7a78c3eca7d542d7a71f9a33e8585c18 (DEFAULT_TABLE table=default.Table_66cd278959e849dfaf7c9fc9e604d46a [id=6ecde174f5dc422387c42ec4e2f3a053]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.262009 14809 ts_tablet_manager.cc:1428] T 83ac7b09047849da8f6770126c8dc033 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:12.262063 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a78c3eca7d542d7a71f9a33e8585c18. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.262161 14809 tablet_bootstrap.cc:492] T a39beac89ce94b26a40dd6539637e6b3 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.262588 14422 raft_consensus.cc:1270] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.263106 14891 tablet.cc:1721] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.263197 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c (DEFAULT_TABLE table=default.Table_3c64fb94bd9c4890989ce6f63525a733 [id=f8ea91c4841b470db1b9f05fa47244fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.263209 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 0c8dc386a6b4468d920dbfab906e1366 (DEFAULT_TABLE table=default.Table_59ca04e73d1747c684c81f7563a3e183 [id=6d1418f7a3b64aebace43d4478f10915]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.263803 14824 raft_consensus.cc:491] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.263813 14851 consensus_queue.cc:1035] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.263877 14824 raft_consensus.cc:513] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.264178 14824 leader_election.cc:290] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:12.264552 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c54a5df35e7422b84acc91e3a10aac3" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:12.264911 14532 tablet_service.cc:1467] Processing CreateTablet for tablet 21b146e38bb345fca7a9a159983db68f (DEFAULT_TABLE table=default.Table_7a59f65551fc4aff908198db770fc111 [id=7a9994216f8d437491e6b24c201e9422]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:12.264981 14628 leader_election.cc:343] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:12.265208 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c (DEFAULT_TABLE table=default.Table_3c64fb94bd9c4890989ce6f63525a733 [id=f8ea91c4841b470db1b9f05fa47244fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.265650 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 0c8dc386a6b4468d920dbfab906e1366 (DEFAULT_TABLE table=default.Table_59ca04e73d1747c684c81f7563a3e183 [id=6d1418f7a3b64aebace43d4478f10915]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.265799 14809 tablet_bootstrap.cc:654] T a39beac89ce94b26a40dd6539637e6b3 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.265923 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c8dc386a6b4468d920dbfab906e1366. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.266193 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 7a78c3eca7d542d7a71f9a33e8585c18 (DEFAULT_TABLE table=default.Table_66cd278959e849dfaf7c9fc9e604d46a [id=6ecde174f5dc422387c42ec4e2f3a053]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.257807 14819 ts_tablet_manager.cc:1918] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.266294 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.266299 14819 log.cc:1198] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/4f32d3e059e44722af83c55b71f14c26
I20250115 03:06:12.266528 14819 ts_tablet_manager.cc:1939] T 4f32d3e059e44722af83c55b71f14c26 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.266638 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21b146e38bb345fca7a9a159983db68f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.255447 14828 raft_consensus.cc:2798] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.267611 14828 raft_consensus.cc:695] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:12.267717 14828 consensus_queue.cc:237] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.268307 14809 tablet_bootstrap.cc:492] T a39beac89ce94b26a40dd6539637e6b3 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.268386 14809 ts_tablet_manager.cc:1397] T a39beac89ce94b26a40dd6539637e6b3 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:12.268419 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 4f32d3e059e44722af83c55b71f14c26 (table default.Table_c1020680eef344a49998eddac921fea5 [id=649880f1ff0d4cfda2358b408110d8dc]) successfully deleted
W20250115 03:06:12.268563 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 5e6367c9f60348469d8442b686b3ff3a because the tablet was not found. No further retry: Not found: Tablet not found: 5e6367c9f60348469d8442b686b3ff3a
I20250115 03:06:12.268797 14809 raft_consensus.cc:357] T a39beac89ce94b26a40dd6539637e6b3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.268883 14809 raft_consensus.cc:383] T a39beac89ce94b26a40dd6539637e6b3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.268911 14809 raft_consensus.cc:738] T a39beac89ce94b26a40dd6539637e6b3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.269069 14809 consensus_queue.cc:260] T a39beac89ce94b26a40dd6539637e6b3 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.269326 14809 ts_tablet_manager.cc:1428] T a39beac89ce94b26a40dd6539637e6b3 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.269469 14809 tablet_bootstrap.cc:492] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.269611 14969 tablet.cc:1721] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.270289 14809 tablet_bootstrap.cc:654] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.270774 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b51a11d524dc4f7489148985667c7bcc" } modify_external_catalogs: true
I20250115 03:06:12.270872 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b51a11d524dc4f7489148985667c7bcc" } modify_external_catalogs: true
I20250115 03:06:12.271229 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a78c3eca7d542d7a71f9a33e8585c18. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.272215 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c8dc386a6b4468d920dbfab906e1366. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.272500 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.272815 14809 tablet_bootstrap.cc:492] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.272886 14809 ts_tablet_manager.cc:1397] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:12.273310 14809 raft_consensus.cc:357] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.273391 14809 raft_consensus.cc:383] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.273420 14809 raft_consensus.cc:738] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.273520 14809 consensus_queue.cc:260] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.273726 14809 ts_tablet_manager.cc:1428] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.273871 14809 tablet_bootstrap.cc:492] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.274600 14809 tablet_bootstrap.cc:654] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.275297 14812 tablet_bootstrap.cc:492] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.275393 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c54a5df35e7422b84acc91e3a10aac3" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.276304 14812 tablet_bootstrap.cc:654] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.276860 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 21b146e38bb345fca7a9a159983db68f
I20250115 03:06:12.276878 14809 tablet_bootstrap.cc:492] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.276969 14809 ts_tablet_manager.cc:1397] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
W20250115 03:06:12.277091 14626 leader_election.cc:343] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:12.277145 14626 leader_election.cc:304] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:12.277354 14809 raft_consensus.cc:357] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.277441 14809 raft_consensus.cc:383] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.277472 14809 raft_consensus.cc:738] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.277570 14824 raft_consensus.cc:2743] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:12.277557 14809 consensus_queue.cc:260] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.277730 14164 catalog_manager.cc:5526] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:12.277853 14809 ts_tablet_manager.cc:1428] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.277985 14809 tablet_bootstrap.cc:492] T b4ea80ef8b3946129c8d5ba57d582504 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.278790 14809 tablet_bootstrap.cc:654] T b4ea80ef8b3946129c8d5ba57d582504 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.279116 14941 hybrid_clock.cc:648] HybridClock initialized: now 1736910372277468 us; error 1667 us; skew 500 ppm
I20250115 03:06:12.281003 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 21b146e38bb345fca7a9a159983db68f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:12.281666 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 21b146e38bb345fca7a9a159983db68f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 21b146e38bb345fca7a9a159983db68f already in progress: creating tablet
I20250115 03:06:12.281752 14941 webserver.cc:458] Webserver started at http://127.13.186.254:37481/ using document root <none> and password file <none>
I20250115 03:06:12.281764 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet 83ac7b09047849da8f6770126c8dc033 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.282402 14941 fs_manager.cc:362] Metadata directory not provided
I20250115 03:06:12.282483 14941 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250115 03:06:12.282655 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 21b146e38bb345fca7a9a159983db68f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.283037 14168 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet 59ec4f67382248afab557f81047c5003
I20250115 03:06:12.286008 14672 tablet_service.cc:1467] Processing CreateTablet for tablet fa7043f411374111b4c81467260fc6cd (DEFAULT_TABLE table=default.Table_c1020680eef344a49998eddac921fea5 [id=649880f1ff0d4cfda2358b408110d8dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.286213 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa7043f411374111b4c81467260fc6cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.286593 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 99fccdbe3a924305af7794e1ce8b092a (DEFAULT_TABLE table=default.Table_94ee8224b6cd468bb2a1c15b2c2af35a [id=63bd4ca58023436eb2b453f6fa0ea0c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.286816 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99fccdbe3a924305af7794e1ce8b092a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.287092 14400 tablet_service.cc:1467] Processing CreateTablet for tablet fa7043f411374111b4c81467260fc6cd (DEFAULT_TABLE table=default.Table_c1020680eef344a49998eddac921fea5 [id=649880f1ff0d4cfda2358b408110d8dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.287283 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa7043f411374111b4c81467260fc6cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.288002 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.288864 14669 tablet_service.cc:1514] Processing DeleteTablet for tablet 21b146e38bb345fca7a9a159983db68f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.289067 14532 tablet_service.cc:1467] Processing CreateTablet for tablet 3633e83ec7144230b08e36f6846dbc8e (DEFAULT_TABLE table=default.Table_dd57412817d54552a349868025274901 [id=8144a2c71b2d475597cd54b3ff2ba720]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.289139 14530 tablet_service.cc:1467] Processing CreateTablet for tablet fa7043f411374111b4c81467260fc6cd (DEFAULT_TABLE table=default.Table_c1020680eef344a49998eddac921fea5 [id=649880f1ff0d4cfda2358b408110d8dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.289266 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3633e83ec7144230b08e36f6846dbc8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.289947 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 99fccdbe3a924305af7794e1ce8b092a (DEFAULT_TABLE table=default.Table_94ee8224b6cd468bb2a1c15b2c2af35a [id=63bd4ca58023436eb2b453f6fa0ea0c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.290164 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99fccdbe3a924305af7794e1ce8b092a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.290935 14171 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 3ed567d482aa43ee9e24f992c346ebc3 (table default.Table_e1a261c278c04d8ea57ffe2181c5fd0d [id=e34781ee2864498bbc532144911630da]). Expected version 1 got 0
I20250115 03:06:12.291584 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 99fccdbe3a924305af7794e1ce8b092a (DEFAULT_TABLE table=default.Table_94ee8224b6cd468bb2a1c15b2c2af35a [id=63bd4ca58023436eb2b453f6fa0ea0c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.291795 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99fccdbe3a924305af7794e1ce8b092a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.292475 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 3ed567d482aa43ee9e24f992c346ebc3 (table default.Table_e1a261c278c04d8ea57ffe2181c5fd0d [id=e34781ee2864498bbc532144911630da]). Expected version 1 got 0
I20250115 03:06:12.292574 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 99e3ff9f0b364b2f968e0faf3cace781 (table default.Table_0dd17013f6334efe97618f0b520512c5 [id=488a9a6268f64669ae17e1a67caa73a5]). Expected version 2 got 0
I20250115 03:06:12.292732 14166 catalog_manager.cc:5526] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } }
W20250115 03:06:12.292734 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 21b146e38bb345fca7a9a159983db68f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 21b146e38bb345fca7a9a159983db68f already in progress: creating tablet
I20250115 03:06:12.293509 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa7043f411374111b4c81467260fc6cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.293885 14941 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:12.294755 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 748097e809744c408f557561e5f35e85 with delete_type TABLET_DATA_DELETED (replaced by tablet 594f755feed947afba1ba8b0853d5788) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.295351 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet a39beac89ce94b26a40dd6539637e6b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.295899 14823 tablet_replica.cc:331] T a39beac89ce94b26a40dd6539637e6b3 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
W20250115 03:06:12.295910 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 21b146e38bb345fca7a9a159983db68f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 21b146e38bb345fca7a9a159983db68f already in progress: creating tablet
I20250115 03:06:12.296010 14812 tablet_bootstrap.cc:492] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.296085 14812 ts_tablet_manager.cc:1397] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20250115 03:06:12.296161 14823 raft_consensus.cc:2238] T a39beac89ce94b26a40dd6539637e6b3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.296284 14823 raft_consensus.cc:2267] T a39beac89ce94b26a40dd6539637e6b3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.297721 14811 tablet_bootstrap.cc:492] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.297812 14811 ts_tablet_manager.cc:1397] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.050s user 0.000s sys 0.001s
I20250115 03:06:12.298489 14811 raft_consensus.cc:357] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.298589 14811 raft_consensus.cc:383] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.298620 14811 raft_consensus.cc:738] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.298707 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 3633e83ec7144230b08e36f6846dbc8e (DEFAULT_TABLE table=default.Table_dd57412817d54552a349868025274901 [id=8144a2c71b2d475597cd54b3ff2ba720]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.298727 14811 consensus_queue.cc:260] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.298913 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3633e83ec7144230b08e36f6846dbc8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.298944 15034 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250115 03:06:12.298969 14811 ts_tablet_manager.cc:1428] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20250115 03:06:12.299120 14811 tablet_bootstrap.cc:492] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.299541 14809 tablet_bootstrap.cc:492] T b4ea80ef8b3946129c8d5ba57d582504 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.299633 14809 ts_tablet_manager.cc:1397] T b4ea80ef8b3946129c8d5ba57d582504 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20250115 03:06:12.299952 14811 tablet_bootstrap.cc:654] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.300022 14809 raft_consensus.cc:357] T b4ea80ef8b3946129c8d5ba57d582504 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.300108 14809 raft_consensus.cc:383] T b4ea80ef8b3946129c8d5ba57d582504 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.300138 14809 raft_consensus.cc:738] T b4ea80ef8b3946129c8d5ba57d582504 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.300240 14809 consensus_queue.cc:260] T b4ea80ef8b3946129c8d5ba57d582504 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.300489 14809 ts_tablet_manager.cc:1428] T b4ea80ef8b3946129c8d5ba57d582504 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.300645 14809 tablet_bootstrap.cc:492] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.300665 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 594f755feed947afba1ba8b0853d5788 (DEFAULT_TABLE table=default.Table_dd9da62650c241fb9b1b74438090cf87 [id=a4d048fc192e46ae9dee9635a45a1229]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.300860 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 594f755feed947afba1ba8b0853d5788. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.300904 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c
I20250115 03:06:12.301162 14168 catalog_manager.cc:6596] ReplaceTablet: tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c deleted and replaced by tablet 7479bfc96d634011ba15069b5cf6794c
I20250115 03:06:12.301483 14809 tablet_bootstrap.cc:654] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.301726 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 83ac7b09047849da8f6770126c8dc033
I20250115 03:06:12.301945 14169 catalog_manager.cc:6596] ReplaceTablet: tablet 83ac7b09047849da8f6770126c8dc033 deleted and replaced by tablet 338592cc70ad4e61ba90905a424a3871
I20250115 03:06:12.302386 14809 tablet_bootstrap.cc:492] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.303565 14941 fs_manager.cc:730] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20250115 03:06:12.303987 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c with delete_type TABLET_DATA_DELETED (replaced by tablet 7479bfc96d634011ba15069b5cf6794c) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.304116 14532 tablet_service.cc:1467] Processing CreateTablet for tablet 0ecd4741aacb45e2ae6404675f885125 (DEFAULT_TABLE table=default.Table_3e8022382eb84ab9bae6fa5bc222cd1d [id=6fa83879f044445482db1ae6fde9f37f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.304004 14531 tablet_service.cc:1514] Processing DeleteTablet for tablet 83ac7b09047849da8f6770126c8dc033 with delete_type TABLET_DATA_DELETED (replaced by tablet 338592cc70ad4e61ba90905a424a3871) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.304299 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ecd4741aacb45e2ae6404675f885125. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.304369 14820 tablet_replica.cc:331] T 83ac7b09047849da8f6770126c8dc033 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.304461 14820 raft_consensus.cc:2238] T 83ac7b09047849da8f6770126c8dc033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.304541 14820 raft_consensus.cc:2267] T 83ac7b09047849da8f6770126c8dc033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.303665 14941 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data,/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/wal
uuid: "4225fe2fbfa84937bb41fed702ca66e4"
format_stamp: "Formatted at 2025-01-15 03:06:09 on dist-test-slave-c74p"
I20250115 03:06:12.304656 14941 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250115 03:06:12.304986 14820 ts_tablet_manager.cc:1905] T 83ac7b09047849da8f6770126c8dc033 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.306070 14396 tablet_service.cc:1467] Processing CreateTablet for tablet a2f815e6fe4e4936823d51fcd16b453b (DEFAULT_TABLE table=default.Table_507ab8820cba47a0bb879411fd28f671 [id=38bfb0375c334d16b97c7251d6ddd76b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.306249 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2f815e6fe4e4936823d51fcd16b453b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.306777 14820 ts_tablet_manager.cc:1918] T 83ac7b09047849da8f6770126c8dc033 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.306833 14820 log.cc:1198] T 83ac7b09047849da8f6770126c8dc033 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/83ac7b09047849da8f6770126c8dc033
I20250115 03:06:12.307003 14820 ts_tablet_manager.cc:1939] T 83ac7b09047849da8f6770126c8dc033 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
W20250115 03:06:12.308702 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 748097e809744c408f557561e5f35e85 because the tablet was not found. No further retry: Not found: Tablet not found: 748097e809744c408f557561e5f35e85
I20250115 03:06:12.302464 14809 ts_tablet_manager.cc:1397] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:12.309386 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 83ac7b09047849da8f6770126c8dc033 with delete_type TABLET_DATA_DELETED (replaced by tablet 338592cc70ad4e61ba90905a424a3871) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.309542 14819 tablet_replica.cc:331] T 83ac7b09047849da8f6770126c8dc033 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.309603 14809 raft_consensus.cc:357] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.309687 14809 raft_consensus.cc:383] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.309643 14819 raft_consensus.cc:2238] T 83ac7b09047849da8f6770126c8dc033 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.309725 14809 raft_consensus.cc:738] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.309787 14819 raft_consensus.cc:2267] T 83ac7b09047849da8f6770126c8dc033 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:12.309772 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c already in progress: creating tablet
I20250115 03:06:12.309821 14809 consensus_queue.cc:260] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.309959 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 83ac7b09047849da8f6770126c8dc033 (table default.Table_ee5faa8cd90f40199372322d90c5a204 [id=ff4681cab8ce44ceb45ed76c70c1132e]) successfully deleted
I20250115 03:06:12.310056 14809 ts_tablet_manager.cc:1428] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:12.310138 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_c1020680eef344a49998eddac921fea5" } modify_external_catalogs: true
I20250115 03:06:12.310205 14809 tablet_bootstrap.cc:492] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.310232 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_c1020680eef344a49998eddac921fea5" } modify_external_catalogs: true
I20250115 03:06:12.310688 14819 ts_tablet_manager.cc:1905] T 83ac7b09047849da8f6770126c8dc033 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.311515 14809 tablet_bootstrap.cc:654] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.312355 14171 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 36bda89e64d44e708bccbcab847634a1 (table default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52]). Expected version 1 got 0
I20250115 03:06:12.312463 14171 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 99e3ff9f0b364b2f968e0faf3cace781 (table default.Table_0dd17013f6334efe97618f0b520512c5 [id=488a9a6268f64669ae17e1a67caa73a5]). Expected version 2 got 0
I20250115 03:06:12.312479 14819 ts_tablet_manager.cc:1918] T 83ac7b09047849da8f6770126c8dc033 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.312537 14819 log.cc:1198] T 83ac7b09047849da8f6770126c8dc033 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/83ac7b09047849da8f6770126c8dc033
I20250115 03:06:12.312556 14171 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet c6771c98e569468ea932882858981d23 (table default.Table_44e943822f744731aaee3411219a0765 [id=f56a0de6e56441bfa620902f175feb4a]). Expected version 1 got 0
I20250115 03:06:12.312726 14819 ts_tablet_manager.cc:1939] T 83ac7b09047849da8f6770126c8dc033 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.313741 14164 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 99e3ff9f0b364b2f968e0faf3cace781 (table default.Table_0dd17013f6334efe97618f0b520512c5 [id=488a9a6268f64669ae17e1a67caa73a5]). Expected version 2 got 0
I20250115 03:06:12.314589 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet fa7043f411374111b4c81467260fc6cd
I20250115 03:06:12.314616 14555 raft_consensus.cc:1270] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.315021 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 594f755feed947afba1ba8b0853d5788 (DEFAULT_TABLE table=default.Table_dd9da62650c241fb9b1b74438090cf87 [id=a4d048fc192e46ae9dee9635a45a1229]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.315050 14813 consensus_queue.cc:1035] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.315199 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 0ecd4741aacb45e2ae6404675f885125 (DEFAULT_TABLE table=default.Table_3e8022382eb84ab9bae6fa5bc222cd1d [id=6fa83879f044445482db1ae6fde9f37f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.315405 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ecd4741aacb45e2ae6404675f885125. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.315588 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet fa7043f411374111b4c81467260fc6cd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.315824 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 83ac7b09047849da8f6770126c8dc033 (table default.Table_ee5faa8cd90f40199372322d90c5a204 [id=ff4681cab8ce44ceb45ed76c70c1132e]) successfully deleted
I20250115 03:06:12.316062 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 594f755feed947afba1ba8b0853d5788. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:12.316111 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet fa7043f411374111b4c81467260fc6cd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa7043f411374111b4c81467260fc6cd already in progress: creating tablet
I20250115 03:06:12.315696 14533 tablet_service.cc:1467] Processing CreateTablet for tablet a2f815e6fe4e4936823d51fcd16b453b (DEFAULT_TABLE table=default.Table_507ab8820cba47a0bb879411fd28f671 [id=38bfb0375c334d16b97c7251d6ddd76b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.296520 14812 raft_consensus.cc:357] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.298228 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 3633e83ec7144230b08e36f6846dbc8e (DEFAULT_TABLE table=default.Table_dd57412817d54552a349868025274901 [id=8144a2c71b2d475597cd54b3ff2ba720]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.317611 14812 raft_consensus.cc:383] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.317656 14812 raft_consensus.cc:738] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.298503 14688 raft_consensus.cc:1270] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.317776 14812 consensus_queue.cc:260] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.298542 14671 tablet_service.cc:1467] Processing CreateTablet for tablet 594f755feed947afba1ba8b0853d5788 (DEFAULT_TABLE table=default.Table_dd9da62650c241fb9b1b74438090cf87 [id=a4d048fc192e46ae9dee9635a45a1229]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.317781 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3633e83ec7144230b08e36f6846dbc8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.318050 14812 ts_tablet_manager.cc:1428] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.022s user 0.001s sys 0.000s
I20250115 03:06:12.318204 14812 tablet_bootstrap.cc:492] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.318701 14813 consensus_queue.cc:1035] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.319068 14812 tablet_bootstrap.cc:654] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.319463 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 594f755feed947afba1ba8b0853d5788. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.319947 14812 tablet_bootstrap.cc:492] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.320012 14812 ts_tablet_manager.cc:1397] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:12.320396 14812 raft_consensus.cc:357] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.320492 14812 raft_consensus.cc:383] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.320520 14812 raft_consensus.cc:738] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.320655 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet 66a38c4ffeb44212906dbe1207f91005 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.321173 14171 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_a86904006b724cb980fe7575e6b6c13a [id=9f650455cb3440a49a11010f541084db] alter complete (version 1)
I20250115 03:06:12.299248 14669 tablet_service.cc:1467] Processing CreateTablet for tablet a2f815e6fe4e4936823d51fcd16b453b (DEFAULT_TABLE table=default.Table_507ab8820cba47a0bb879411fd28f671 [id=38bfb0375c334d16b97c7251d6ddd76b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.321390 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2f815e6fe4e4936823d51fcd16b453b. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:12.321458 14171 catalog_manager.cc:5718] Failed to send AlterTable RPC for tablet 36bda89e64d44e708bccbcab847634a1 (table default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 36bda89e64d44e708bccbcab847634a1 consensus state has no leader
W20250115 03:06:12.321556 14171 catalog_manager.cc:5718] Failed to send AlterTable RPC for tablet c6771c98e569468ea932882858981d23 (table default.Table_44e943822f744731aaee3411219a0765 [id=f56a0de6e56441bfa620902f175feb4a], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet c6771c98e569468ea932882858981d23 consensus state has no leader
I20250115 03:06:12.321717 14812 consensus_queue.cc:260] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.322125 14812 ts_tablet_manager.cc:1428] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:12.322274 14812 tablet_bootstrap.cc:492] T fa7043f411374111b4c81467260fc6cd P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.304193 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 0ecd4741aacb45e2ae6404675f885125 (DEFAULT_TABLE table=default.Table_3e8022382eb84ab9bae6fa5bc222cd1d [id=6fa83879f044445482db1ae6fde9f37f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.308490 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet 83ac7b09047849da8f6770126c8dc033 with delete_type TABLET_DATA_DELETED (replaced by tablet 338592cc70ad4e61ba90905a424a3871) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.309404 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c with delete_type TABLET_DATA_DELETED (replaced by tablet 7479bfc96d634011ba15069b5cf6794c) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.315766 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet fa7043f411374111b4c81467260fc6cd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.323299 14812 tablet_bootstrap.cc:654] T fa7043f411374111b4c81467260fc6cd P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.323575 14164 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet c6cd63ac1c084a0d8f166e63b0ed85bc (table default.Table_8f3bf985bedd43918313ea7504253e48 [id=05f9fd2ebec34d2299ea59c47a74c828]). Expected version 2 got 0
I20250115 03:06:12.324366 14812 tablet_bootstrap.cc:492] T fa7043f411374111b4c81467260fc6cd P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.324466 14812 ts_tablet_manager.cc:1397] T fa7043f411374111b4c81467260fc6cd P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:12.324862 14812 raft_consensus.cc:357] T fa7043f411374111b4c81467260fc6cd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.324977 14812 raft_consensus.cc:383] T fa7043f411374111b4c81467260fc6cd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.325013 14812 raft_consensus.cc:738] T fa7043f411374111b4c81467260fc6cd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.325111 14812 consensus_queue.cc:260] T fa7043f411374111b4c81467260fc6cd P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.325333 14812 ts_tablet_manager.cc:1428] T fa7043f411374111b4c81467260fc6cd P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.325881 14824 raft_consensus.cc:491] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.325949 14824 raft_consensus.cc:513] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.326205 14824 leader_election.cc:290] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:12.326496 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet fa7043f411374111b4c81467260fc6cd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.326821 14811 tablet_bootstrap.cc:492] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.326893 14811 ts_tablet_manager.cc:1397] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.028s user 0.001s sys 0.000s
I20250115 03:06:12.327122 14531 tablet_service.cc:1514] Processing DeleteTablet for tablet a39beac89ce94b26a40dd6539637e6b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.327142 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ed567d482aa43ee9e24f992c346ebc3" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.327239 14556 raft_consensus.cc:2463] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:12.327280 14811 raft_consensus.cc:357] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.327350 14811 raft_consensus.cc:383] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.327378 14811 raft_consensus.cc:738] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.327466 14811 consensus_queue.cc:260] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.327553 14626 leader_election.cc:304] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:12.327723 14828 raft_consensus.cc:2798] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:12.327730 14811 ts_tablet_manager.cc:1428] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.327767 14828 raft_consensus.cc:491] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:12.327819 14828 raft_consensus.cc:3054] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.327867 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet 748097e809744c408f557561e5f35e85 with delete_type TABLET_DATA_DELETED (replaced by tablet 594f755feed947afba1ba8b0853d5788) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.327978 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ed567d482aa43ee9e24f992c346ebc3" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:12.328091 14421 raft_consensus.cc:2463] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:12.327991 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c with delete_type TABLET_DATA_DELETED (replaced by tablet 7479bfc96d634011ba15069b5cf6794c) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.328616 14828 raft_consensus.cc:513] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.328706 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ecd4741aacb45e2ae6404675f885125. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.328992 14828 leader_election.cc:290] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:12.329190 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ed567d482aa43ee9e24f992c346ebc3" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:12.329273 14421 raft_consensus.cc:3054] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.329982 14421 raft_consensus.cc:2463] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
W20250115 03:06:12.330026 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet fa7043f411374111b4c81467260fc6cd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fa7043f411374111b4c81467260fc6cd already in progress: creating tablet
I20250115 03:06:12.330251 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ed567d482aa43ee9e24f992c346ebc3" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:12.330349 14556 raft_consensus.cc:3054] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.331048 14811 tablet_bootstrap.cc:492] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.331081 14556 raft_consensus.cc:2463] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:12.331311 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 66a38c4ffeb44212906dbe1207f91005
I20250115 03:06:12.331509 14169 catalog_manager.cc:6596] ReplaceTablet: tablet 66a38c4ffeb44212906dbe1207f91005 deleted and replaced by tablet 99abc1c7a20a4475921597f93c2db56c
I20250115 03:06:12.331590 14820 tablet_replica.cc:331] T a39beac89ce94b26a40dd6539637e6b3 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.331730 14628 leader_election.cc:304] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.332114 14828 raft_consensus.cc:2798] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.332183 14828 raft_consensus.cc:695] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:12.332274 14828 consensus_queue.cc:237] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.331774 14820 raft_consensus.cc:2238] T a39beac89ce94b26a40dd6539637e6b3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.333362 14820 raft_consensus.cc:2267] T a39beac89ce94b26a40dd6539637e6b3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.333922 14820 ts_tablet_manager.cc:1905] T a39beac89ce94b26a40dd6539637e6b3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.334594 14169 catalog_manager.cc:5526] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:12.334796 14169 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 3ed567d482aa43ee9e24f992c346ebc3 (table default.Table_e1a261c278c04d8ea57ffe2181c5fd0d [id=e34781ee2864498bbc532144911630da]). Expected version 1 got 0
I20250115 03:06:12.335510 14812 tablet_bootstrap.cc:492] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.336335 14812 tablet_bootstrap.cc:654] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.337965 14812 tablet_bootstrap.cc:492] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.338035 14812 ts_tablet_manager.cc:1397] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:12.338371 14812 raft_consensus.cc:357] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.338464 14812 raft_consensus.cc:383] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.338490 14812 raft_consensus.cc:738] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.338578 14812 consensus_queue.cc:260] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.338796 14812 ts_tablet_manager.cc:1428] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.338937 14812 tablet_bootstrap.cc:492] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.339802 14812 tablet_bootstrap.cc:654] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.340569 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet c6cd63ac1c084a0d8f166e63b0ed85bc (table default.Table_8f3bf985bedd43918313ea7504253e48 [id=05f9fd2ebec34d2299ea59c47a74c828]). Expected version 2 got 0
I20250115 03:06:12.341300 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2f815e6fe4e4936823d51fcd16b453b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.342278 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet 66a38c4ffeb44212906dbe1207f91005 with delete_type TABLET_DATA_DELETED (replaced by tablet 99abc1c7a20a4475921597f93c2db56c) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.342432 14811 tablet_bootstrap.cc:654] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.343936 14969 tablet.cc:1721] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:12.344864 14812 tablet_bootstrap.cc:492] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.344964 14812 ts_tablet_manager.cc:1397] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:12.345309 14812 raft_consensus.cc:357] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.345386 14812 raft_consensus.cc:383] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.345412 14812 raft_consensus.cc:738] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.345515 14812 consensus_queue.cc:260] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.345732 14812 ts_tablet_manager.cc:1428] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.345862 14812 tablet_bootstrap.cc:492] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.346521 14823 ts_tablet_manager.cc:1905] T a39beac89ce94b26a40dd6539637e6b3 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.346720 14812 tablet_bootstrap.cc:654] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.346870 14891 tablet.cc:1721] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:12.348387 14823 ts_tablet_manager.cc:1918] T a39beac89ce94b26a40dd6539637e6b3 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.348469 14823 log.cc:1198] T a39beac89ce94b26a40dd6539637e6b3 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/a39beac89ce94b26a40dd6539637e6b3
I20250115 03:06:12.348659 14823 ts_tablet_manager.cc:1939] T a39beac89ce94b26a40dd6539637e6b3 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.350546 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet a39beac89ce94b26a40dd6539637e6b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.350728 14819 tablet_replica.cc:331] T a39beac89ce94b26a40dd6539637e6b3 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.350836 14819 raft_consensus.cc:2238] T a39beac89ce94b26a40dd6539637e6b3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.350849 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 83ac7b09047849da8f6770126c8dc033 with delete_type TABLET_DATA_DELETED (replaced by tablet 338592cc70ad4e61ba90905a424a3871) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.350929 14819 raft_consensus.cc:2267] T a39beac89ce94b26a40dd6539637e6b3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.351356 14819 ts_tablet_manager.cc:1905] T a39beac89ce94b26a40dd6539637e6b3 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.351506 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 66a38c4ffeb44212906dbe1207f91005 with delete_type TABLET_DATA_DELETED (replaced by tablet 99abc1c7a20a4475921597f93c2db56c) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:12.351890 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c already in progress: creating tablet
I20250115 03:06:12.352001 14670 tablet_service.cc:1514] Processing DeleteTablet for tablet 66a38c4ffeb44212906dbe1207f91005 with delete_type TABLET_DATA_DELETED (replaced by tablet 99abc1c7a20a4475921597f93c2db56c) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.352253 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet a39beac89ce94b26a40dd6539637e6b3 (table default.Table_75c469057273407dbfa671e136972f30 [id=dc009faaff3f4763ac300bd408e78323]) successfully deleted
I20250115 03:06:12.352438 14669 tablet_service.cc:1514] Processing DeleteTablet for tablet 21b146e38bb345fca7a9a159983db68f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.352893 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet c6771c98e569468ea932882858981d23 (table default.Table_44e943822f744731aaee3411219a0765 [id=f56a0de6e56441bfa620902f175feb4a]). Expected version 1 got 0
I20250115 03:06:12.352917 14819 ts_tablet_manager.cc:1918] T a39beac89ce94b26a40dd6539637e6b3 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.353005 14819 log.cc:1198] T a39beac89ce94b26a40dd6539637e6b3 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/a39beac89ce94b26a40dd6539637e6b3
I20250115 03:06:12.353137 14819 ts_tablet_manager.cc:1939] T a39beac89ce94b26a40dd6539637e6b3 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
W20250115 03:06:12.353252 14155 catalog_manager.cc:4670] Async tablet task AlterTable RPC for tablet 36bda89e64d44e708bccbcab847634a1 (table default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 36bda89e64d44e708bccbcab847634a1 consensus state has no leader
I20250115 03:06:12.353849 14819 tablet_replica.cc:331] T 66a38c4ffeb44212906dbe1207f91005 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.353976 14819 raft_consensus.cc:2238] T 66a38c4ffeb44212906dbe1207f91005 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.354068 14819 raft_consensus.cc:2267] T 66a38c4ffeb44212906dbe1207f91005 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.354281 14812 tablet_bootstrap.cc:492] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.354350 14812 ts_tablet_manager.cc:1397] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:12.354492 14809 tablet_bootstrap.cc:492] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.354578 14809 ts_tablet_manager.cc:1397] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.044s user 0.001s sys 0.000s
I20250115 03:06:12.354740 14812 raft_consensus.cc:357] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.354795 14819 ts_tablet_manager.cc:1905] T 66a38c4ffeb44212906dbe1207f91005 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.354820 14812 raft_consensus.cc:383] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.354851 14812 raft_consensus.cc:738] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.354957 14812 consensus_queue.cc:260] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.354995 14809 raft_consensus.cc:357] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.355087 14809 raft_consensus.cc:383] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.355113 14809 raft_consensus.cc:738] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.355198 14812 ts_tablet_manager.cc:1428] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:12.355218 14809 consensus_queue.cc:260] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.355468 14809 ts_tablet_manager.cc:1428] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.355629 14809 tablet_bootstrap.cc:492] T 21b146e38bb345fca7a9a159983db68f P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.356501 14819 ts_tablet_manager.cc:1918] T 66a38c4ffeb44212906dbe1207f91005 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.356570 14819 log.cc:1198] T 66a38c4ffeb44212906dbe1207f91005 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/66a38c4ffeb44212906dbe1207f91005
I20250115 03:06:12.356631 14809 tablet_bootstrap.cc:654] T 21b146e38bb345fca7a9a159983db68f P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.356745 14819 ts_tablet_manager.cc:1939] T 66a38c4ffeb44212906dbe1207f91005 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.358004 14941 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250115 03:06:12.358455 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet a39beac89ce94b26a40dd6539637e6b3 (table default.Table_75c469057273407dbfa671e136972f30 [id=dc009faaff3f4763ac300bd408e78323]) successfully deleted
W20250115 03:06:12.358589 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 83ac7b09047849da8f6770126c8dc033 because the tablet was not found. No further retry: Not found: Tablet not found: 83ac7b09047849da8f6770126c8dc033
I20250115 03:06:12.358711 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 66a38c4ffeb44212906dbe1207f91005 (table default.Table_3f71dfd17bb74691b7956682ae89fb38 [id=f94088b0776f4be990ddcec04afbca43]) successfully deleted
I20250115 03:06:12.358937 14941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250115 03:06:12.359144 14941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250115 03:06:12.360918 14824 raft_consensus.cc:491] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.361030 14824 raft_consensus.cc:513] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.361377 14824 leader_election.cc:290] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:12.361675 14820 ts_tablet_manager.cc:1918] T a39beac89ce94b26a40dd6539637e6b3 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.361773 14820 log.cc:1198] T a39beac89ce94b26a40dd6539637e6b3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/a39beac89ce94b26a40dd6539637e6b3
I20250115 03:06:12.361927 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99e3ff9f0b364b2f968e0faf3cace781" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:12.362044 14820 ts_tablet_manager.cc:1939] T a39beac89ce94b26a40dd6539637e6b3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.362044 14422 raft_consensus.cc:2463] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:12.362505 14628 leader_election.cc:304] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.362555 14820 tablet_replica.cc:331] T 748097e809744c408f557561e5f35e85 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.362671 14820 raft_consensus.cc:2238] T 748097e809744c408f557561e5f35e85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.362725 14824 raft_consensus.cc:2798] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:12.362767 14820 raft_consensus.cc:2267] T 748097e809744c408f557561e5f35e85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.362774 14824 raft_consensus.cc:491] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:12.362814 14824 raft_consensus.cc:3054] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.363195 14820 ts_tablet_manager.cc:1905] T 748097e809744c408f557561e5f35e85 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.363649 14824 raft_consensus.cc:513] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.363951 14824 leader_election.cc:290] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:12.364202 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99e3ff9f0b364b2f968e0faf3cace781" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:12.364255 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet a39beac89ce94b26a40dd6539637e6b3 (table default.Table_75c469057273407dbfa671e136972f30 [id=dc009faaff3f4763ac300bd408e78323]) successfully deleted
I20250115 03:06:12.364252 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99e3ff9f0b364b2f968e0faf3cace781" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.364315 14557 raft_consensus.cc:3054] T 99e3ff9f0b364b2f968e0faf3cace781 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.365082 14820 ts_tablet_manager.cc:1918] T 748097e809744c408f557561e5f35e85 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.365150 14820 log.cc:1198] T 748097e809744c408f557561e5f35e85 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/748097e809744c408f557561e5f35e85
I20250115 03:06:12.365218 14557 raft_consensus.cc:2463] T 99e3ff9f0b364b2f968e0faf3cace781 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:12.365350 14820 ts_tablet_manager.cc:1939] T 748097e809744c408f557561e5f35e85 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.365418 14811 tablet_bootstrap.cc:492] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.365501 14811 ts_tablet_manager.cc:1397] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.034s user 0.001s sys 0.001s
I20250115 03:06:12.365592 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet 99fccdbe3a924305af7794e1ce8b092a from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.365638 14626 leader_election.cc:304] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:12.365787 14824 raft_consensus.cc:2798] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.365857 14824 raft_consensus.cc:695] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:12.365939 14811 raft_consensus.cc:357] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.366027 14811 raft_consensus.cc:383] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.365978 14824 consensus_queue.cc:237] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.366053 14811 raft_consensus.cc:738] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.366165 14811 consensus_queue.cc:260] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.366432 14811 ts_tablet_manager.cc:1428] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.366578 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_dd9da62650c241fb9b1b74438090cf87" } modify_external_catalogs: true
I20250115 03:06:12.366629 14811 tablet_bootstrap.cc:492] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.366676 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_dd9da62650c241fb9b1b74438090cf87" } modify_external_catalogs: true
I20250115 03:06:12.366694 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_6d37c9d2d26e4b73ba4c230e471aeef7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:12.366940 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e376c0b790ce48278ea203fdb6bda08d" } modify_external_catalogs: true
I20250115 03:06:12.367008 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e376c0b790ce48278ea203fdb6bda08d" } modify_external_catalogs: true
W20250115 03:06:12.367025 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6d37c9d2d26e4b73ba4c230e471aeef7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.367559 14811 tablet_bootstrap.cc:654] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.369030 14861 raft_consensus.cc:491] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.369107 14861 raft_consensus.cc:513] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.369333 14820 tablet_replica.cc:331] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.369437 14820 raft_consensus.cc:2238] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:12.369426 14861 leader_election.cc:290] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:12.369563 14820 pending_rounds.cc:62] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1: Trying to abort 1 pending ops.
I20250115 03:06:12.369593 14820 pending_rounds.cc:69] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384884678991872 op_type: NO_OP noop_request { }
I20250115 03:06:12.369639 14820 raft_consensus.cc:2883] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:12.369688 14820 raft_consensus.cc:2267] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.369868 14858 raft_consensus.cc:491] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.369933 14858 raft_consensus.cc:513] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.370218 14858 leader_election.cc:290] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:12.370846 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ea80ef8b3946129c8d5ba57d582504" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:12.370925 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ea80ef8b3946129c8d5ba57d582504" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:12.370968 14688 raft_consensus.cc:2463] T b4ea80ef8b3946129c8d5ba57d582504 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:12.371034 14422 raft_consensus.cc:2463] T b4ea80ef8b3946129c8d5ba57d582504 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:12.371234 14493 leader_election.cc:304] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:12.371538 14858 raft_consensus.cc:2798] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:12.371590 14858 raft_consensus.cc:491] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:12.371625 14858 raft_consensus.cc:3054] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.371716 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c54a5df35e7422b84acc91e3a10aac3" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:12.371771 14941 rpc_server.cc:307] RPC server started. Bound to: 127.13.186.254:39901
I20250115 03:06:12.371817 14688 raft_consensus.cc:2463] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:12.372411 14858 raft_consensus.cc:513] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.372730 14858 leader_election.cc:290] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:12.372941 14828 raft_consensus.cc:491] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.372972 14358 leader_election.cc:304] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.373014 14828 raft_consensus.cc:513] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.373067 14941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data/info.pb
I20250115 03:06:12.373126 14861 raft_consensus.cc:2798] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:12.373163 14861 raft_consensus.cc:491] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:12.373194 14861 raft_consensus.cc:3054] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.373292 14828 leader_election.cc:290] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:12.373843 15101 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.186.254:39901 every 8 connection(s)
I20250115 03:06:12.374023 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ea80ef8b3946129c8d5ba57d582504" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:12.374126 14420 raft_consensus.cc:3054] T b4ea80ef8b3946129c8d5ba57d582504 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.374239 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21b146e38bb345fca7a9a159983db68f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:12.374315 14820 ts_tablet_manager.cc:1905] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.374631 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21b146e38bb345fca7a9a159983db68f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.374944 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99e3ff9f0b364b2f968e0faf3cace781" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:12.374974 14420 raft_consensus.cc:2463] T b4ea80ef8b3946129c8d5ba57d582504 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:12.375031 14422 raft_consensus.cc:3054] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:12.375228 14626 leader_election.cc:343] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:12.375283 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ea80ef8b3946129c8d5ba57d582504" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:12.375380 14688 raft_consensus.cc:3054] T b4ea80ef8b3946129c8d5ba57d582504 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.375800 14422 raft_consensus.cc:2463] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
W20250115 03:06:12.376147 14628 leader_election.cc:343] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:12.376199 14688 raft_consensus.cc:2463] T b4ea80ef8b3946129c8d5ba57d582504 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:12.376207 14628 leader_election.cc:304] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:12.376544 14493 leader_election.cc:304] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:12.376564 14828 raft_consensus.cc:2743] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:12.376747 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c54a5df35e7422b84acc91e3a10aac3" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.376847 14557 raft_consensus.cc:2463] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:12.376911 14820 ts_tablet_manager.cc:1918] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:12.376983 14820 log.cc:1198] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/66a38c4ffeb44212906dbe1207f91005
I20250115 03:06:12.377185 14820 ts_tablet_manager.cc:1939] T 66a38c4ffeb44212906dbe1207f91005 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.374089 14861 raft_consensus.cc:513] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.377749 14861 leader_election.cc:290] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:12.378177 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c54a5df35e7422b84acc91e3a10aac3" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:12.378247 14809 tablet_bootstrap.cc:492] T 21b146e38bb345fca7a9a159983db68f P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.378268 14557 raft_consensus.cc:3054] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.378319 14809 ts_tablet_manager.cc:1397] T 21b146e38bb345fca7a9a159983db68f P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.002s
I20250115 03:06:12.378640 14811 tablet_bootstrap.cc:492] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.378710 14811 ts_tablet_manager.cc:1397] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20250115 03:06:12.378726 14809 raft_consensus.cc:357] T 21b146e38bb345fca7a9a159983db68f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.378818 14809 raft_consensus.cc:383] T 21b146e38bb345fca7a9a159983db68f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.378846 14809 raft_consensus.cc:738] T 21b146e38bb345fca7a9a159983db68f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.378850 15102 sys_catalog.cc:263] Verifying existing consensus state
I20250115 03:06:12.378953 14809 consensus_queue.cc:260] T 21b146e38bb345fca7a9a159983db68f P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.379027 14557 raft_consensus.cc:2463] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:12.379102 14811 raft_consensus.cc:357] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.379185 14811 raft_consensus.cc:383] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.379220 14811 raft_consensus.cc:738] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.379217 14809 ts_tablet_manager.cc:1428] T 21b146e38bb345fca7a9a159983db68f P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:12.379316 14811 consensus_queue.cc:260] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.379384 14809 tablet_bootstrap.cc:492] T 7a78c3eca7d542d7a71f9a33e8585c18 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.379552 14811 ts_tablet_manager.cc:1428] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.379720 14811 tablet_bootstrap.cc:492] T 21b146e38bb345fca7a9a159983db68f P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.380165 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c54a5df35e7422b84acc91e3a10aac3" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:12.380260 14688 raft_consensus.cc:3054] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.380292 14809 tablet_bootstrap.cc:654] T 7a78c3eca7d542d7a71f9a33e8585c18 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.380577 14811 tablet_bootstrap.cc:654] T 21b146e38bb345fca7a9a159983db68f P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.380781 15102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4: Bootstrap starting.
I20250115 03:06:12.381060 14858 raft_consensus.cc:2798] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.381140 14858 raft_consensus.cc:695] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:12.381244 14858 consensus_queue.cc:237] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.381531 14356 leader_election.cc:304] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.382045 14059 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu as pid 14941
I20250115 03:06:12.382414 14885 raft_consensus.cc:2798] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.382494 14885 raft_consensus.cc:695] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:12.382488 14688 raft_consensus.cc:2463] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:12.382606 14885 consensus_queue.cc:237] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:12.382942 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 748097e809744c408f557561e5f35e85 (table default.Table_dd9da62650c241fb9b1b74438090cf87 [id=a4d048fc192e46ae9dee9635a45a1229]) successfully deleted
I20250115 03:06:12.383085 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 66a38c4ffeb44212906dbe1207f91005 (table default.Table_3f71dfd17bb74691b7956682ae89fb38 [id=f94088b0776f4be990ddcec04afbca43]) successfully deleted
I20250115 03:06:12.383823 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 36bda89e64d44e708bccbcab847634a1 (table default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52]). Expected version 1 got 0
I20250115 03:06:12.385691 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 594f755feed947afba1ba8b0853d5788
I20250115 03:06:12.386781 15102 log.cc:826] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4: Log is configured to *not* fsync() on all Append() calls
I20250115 03:06:12.388305 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet c6771c98e569468ea932882858981d23 (table default.Table_44e943822f744731aaee3411219a0765 [id=f56a0de6e56441bfa620902f175feb4a]). Expected version 1 got 0
I20250115 03:06:12.388476 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 594f755feed947afba1ba8b0853d5788 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.388957 14669 tablet_service.cc:1514] Processing DeleteTablet for tablet 594f755feed947afba1ba8b0853d5788 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.389211 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 99fccdbe3a924305af7794e1ce8b092a
I20250115 03:06:12.389446 14164 catalog_manager.cc:6596] ReplaceTablet: tablet 99fccdbe3a924305af7794e1ce8b092a deleted and replaced by tablet 876e28105fbb47bdb722e5d3f8467341
I20250115 03:06:12.391980 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_5088472ecce645e2935b8b4cb18aa458"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:12.392067 14811 tablet_bootstrap.cc:492] T 21b146e38bb345fca7a9a159983db68f P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.392140 14811 ts_tablet_manager.cc:1397] T 21b146e38bb345fca7a9a159983db68f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
W20250115 03:06:12.392329 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5088472ecce645e2935b8b4cb18aa458 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.392550 14811 raft_consensus.cc:357] T 21b146e38bb345fca7a9a159983db68f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.392645 14811 raft_consensus.cc:383] T 21b146e38bb345fca7a9a159983db68f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.392676 14811 raft_consensus.cc:738] T 21b146e38bb345fca7a9a159983db68f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.392784 14811 consensus_queue.cc:260] T 21b146e38bb345fca7a9a159983db68f P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.393051 14811 ts_tablet_manager.cc:1428] T 21b146e38bb345fca7a9a159983db68f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.393213 14811 tablet_bootstrap.cc:492] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.393745 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_228cef4dbc0c49bc9c06dffd88d1074c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:12.394104 14811 tablet_bootstrap.cc:654] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:12.394130 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_228cef4dbc0c49bc9c06dffd88d1074c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.395331 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_64c4e8668b494feca761d65318286040"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:12.395495 14811 tablet_bootstrap.cc:492] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.395562 14811 ts_tablet_manager.cc:1397] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
W20250115 03:06:12.395627 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_64c4e8668b494feca761d65318286040 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.395956 14811 raft_consensus.cc:357] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.396044 14811 raft_consensus.cc:383] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.396072 14811 raft_consensus.cc:738] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.396592 14809 tablet_bootstrap.cc:492] T 7a78c3eca7d542d7a71f9a33e8585c18 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.396557 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d0dde0a873df401c9502d832495fb2b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:12.396582 14811 consensus_queue.cc:260] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.396675 14809 ts_tablet_manager.cc:1397] T 7a78c3eca7d542d7a71f9a33e8585c18 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.001s
I20250115 03:06:12.396829 14811 ts_tablet_manager.cc:1428] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20250115 03:06:12.396819 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d0dde0a873df401c9502d832495fb2b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.396998 14811 tablet_bootstrap.cc:492] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.397146 14809 raft_consensus.cc:357] T 7a78c3eca7d542d7a71f9a33e8585c18 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.397243 14809 raft_consensus.cc:383] T 7a78c3eca7d542d7a71f9a33e8585c18 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.397271 14809 raft_consensus.cc:738] T 7a78c3eca7d542d7a71f9a33e8585c18 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.397361 14809 consensus_queue.cc:260] T 7a78c3eca7d542d7a71f9a33e8585c18 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.397603 14809 ts_tablet_manager.cc:1428] T 7a78c3eca7d542d7a71f9a33e8585c18 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.397768 14809 tablet_bootstrap.cc:492] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.397969 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2e115074bf7049bfbb1239b4d9c1ca2e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.398231 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e115074bf7049bfbb1239b4d9c1ca2e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.398762 14809 tablet_bootstrap.cc:654] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.398938 15102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250115 03:06:12.399257 15102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4: Bootstrap complete.
I20250115 03:06:12.399330 14811 tablet_bootstrap.cc:654] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.399417 14861 raft_consensus.cc:491] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.399497 14861 raft_consensus.cc:513] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.399811 14861 leader_election.cc:290] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.401057 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36bda89e64d44e708bccbcab847634a1" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.401129 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36bda89e64d44e708bccbcab847634a1" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:12.401163 14557 raft_consensus.cc:2463] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:12.401232 14688 raft_consensus.cc:2463] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:12.401495 14356 leader_election.cc:304] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.401695 14813 raft_consensus.cc:2798] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:12.401748 14813 raft_consensus.cc:491] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:12.401785 14813 raft_consensus.cc:3054] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.402727 14813 raft_consensus.cc:513] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.402907 14166 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 7a78c3eca7d542d7a71f9a33e8585c18 (table default.Table_66cd278959e849dfaf7c9fc9e604d46a [id=6ecde174f5dc422387c42ec4e2f3a053]). Expected version 1 got 0
I20250115 03:06:12.403040 14813 leader_election.cc:290] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.404727 14828 raft_consensus.cc:491] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.404796 14828 raft_consensus.cc:513] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.405108 14828 leader_election.cc:290] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:12.405164 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet b4ea80ef8b3946129c8d5ba57d582504
I20250115 03:06:12.406013 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99fccdbe3a924305af7794e1ce8b092a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.405768 15102 raft_consensus.cc:357] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } }
I20250115 03:06:12.406137 15102 raft_consensus.cc:738] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4225fe2fbfa84937bb41fed702ca66e4, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.406630 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_6c1f266c05474fa4a4ff61f5078bcfbc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:12.406625 15102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } }
W20250115 03:06:12.406898 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6c1f266c05474fa4a4ff61f5078bcfbc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.406991 14469 heartbeater.cc:346] Connected to a master server at 127.13.186.254:39901
W20250115 03:06:12.407043 14626 leader_election.cc:343] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:12.407075 14469 heartbeater.cc:501] Master 127.13.186.254:39901 was elected leader, sending a full tablet report...
I20250115 03:06:12.407702 15102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [sys.catalog]: configured and running, proceeding with master startup.
I20250115 03:06:12.408489 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet 66d755eaad304219a1235031b4841ba8 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.408860 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36bda89e64d44e708bccbcab847634a1" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:12.408987 14557 raft_consensus.cc:3054] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.409848 14557 raft_consensus.cc:2463] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:12.410573 14356 leader_election.cc:304] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.410727 14813 raft_consensus.cc:2798] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.410800 14813 raft_consensus.cc:695] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:12.410919 14813 consensus_queue.cc:237] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.411017 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36bda89e64d44e708bccbcab847634a1" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:12.411106 14688 raft_consensus.cc:3054] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.411944 14688 raft_consensus.cc:2463] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:12.412199 14824 consensus_queue.cc:1035] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.413322 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet b4ea80ef8b3946129c8d5ba57d582504 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.413465 14820 tablet_replica.cc:331] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.413565 14820 raft_consensus.cc:2238] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:12.413681 14820 pending_rounds.cc:62] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1: Trying to abort 1 pending ops.
I20250115 03:06:12.413700 14820 pending_rounds.cc:69] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384885274783744 op_type: NO_OP noop_request { }
I20250115 03:06:12.413750 14820 raft_consensus.cc:2883] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:12.413789 14820 raft_consensus.cc:2267] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.414000 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c with delete_type TABLET_DATA_DELETED (replaced by tablet 7479bfc96d634011ba15069b5cf6794c) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.414485 15126 catalog_manager.cc:1260] Loaded cluster ID: 6933a79146ea47b6911c92899078d551
I20250115 03:06:12.414477 14469 heartbeater.cc:501] Master 127.13.186.254:39901 was elected leader, sending a full tablet report...
I20250115 03:06:12.414525 15126 catalog_manager.cc:1553] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4: loading cluster ID for follower catalog manager: success
I20250115 03:06:12.415716 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99fccdbe3a924305af7794e1ce8b092a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:12.416018 14628 leader_election.cc:343] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:12.416077 14628 leader_election.cc:304] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:12.416236 14824 raft_consensus.cc:2743] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:12.416728 15126 catalog_manager.cc:1575] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4: acquiring CA information for follower catalog manager: success
I20250115 03:06:12.417596 15055 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" instance_seqno: 1736910370552147) as {username='slave'} at 127.13.186.193:52235; Asking this server to re-register.
I20250115 03:06:12.418256 14469 heartbeater.cc:463] Registering TS with master...
I20250115 03:06:12.418440 14166 catalog_manager.cc:5526] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:12.418584 15126 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250115 03:06:12.418879 14166 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 99e3ff9f0b364b2f968e0faf3cace781 (table default.Table_0dd17013f6334efe97618f0b520512c5 [id=488a9a6268f64669ae17e1a67caa73a5]). Expected version 2 got 0
I20250115 03:06:12.419385 14820 ts_tablet_manager.cc:1905] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.419701 15055 ts_manager.cc:194] Registered new tserver with Master: e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:12.420825 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8f3bf985bedd43918313ea7504253e48" } new_table_name: "default.Table_5b29b7e1e58f49c09b26040407bf49ce" modify_external_catalogs: true
I20250115 03:06:12.420907 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b947c3886a464283ad54fe14793e5618"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.421216 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b947c3886a464283ad54fe14793e5618 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.421572 14820 ts_tablet_manager.cc:1918] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:12.421643 14820 log.cc:1198] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/b4ea80ef8b3946129c8d5ba57d582504
I20250115 03:06:12.421844 14820 ts_tablet_manager.cc:1939] T b4ea80ef8b3946129c8d5ba57d582504 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.422329 14820 tablet_replica.cc:331] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.422428 14820 raft_consensus.cc:2238] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.422514 14820 raft_consensus.cc:2267] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.422792 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet b4ea80ef8b3946129c8d5ba57d582504 (table default.Table_e376c0b790ce48278ea203fdb6bda08d [id=841fc23508e147aeb4f8b8f60f22c94b]) successfully deleted
I20250115 03:06:12.422857 14820 ts_tablet_manager.cc:1905] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.423034 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_59ca04e73d1747c684c81f7563a3e183" } new_table_name: "default.Table_fefc309a5c684cc7a466bc67cba0f109" modify_external_catalogs: true
I20250115 03:06:12.424496 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 594f755feed947afba1ba8b0853d5788 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.424842 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 99fccdbe3a924305af7794e1ce8b092a with delete_type TABLET_DATA_DELETED (replaced by tablet 876e28105fbb47bdb722e5d3f8467341) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.425172 14669 tablet_service.cc:1514] Processing DeleteTablet for tablet 99fccdbe3a924305af7794e1ce8b092a with delete_type TABLET_DATA_DELETED (replaced by tablet 876e28105fbb47bdb722e5d3f8467341) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.425257 15075 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [term 1 FOLLOWER]: Advancing to term 2
I20250115 03:06:12.425293 14670 tablet_service.cc:1514] Processing DeleteTablet for tablet b4ea80ef8b3946129c8d5ba57d582504 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.425573 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet b4ea80ef8b3946129c8d5ba57d582504 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:12.425581 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 594f755feed947afba1ba8b0853d5788 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 594f755feed947afba1ba8b0853d5788 already in progress: creating tablet
I20250115 03:06:12.425582 14820 ts_tablet_manager.cc:1918] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.425642 14820 log.cc:1198] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/81bf6bbe775e42d2a8b0dc61a12a8f5c
W20250115 03:06:12.425695 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 99fccdbe3a924305af7794e1ce8b092a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 99fccdbe3a924305af7794e1ce8b092a already in progress: creating tablet
I20250115 03:06:12.425716 14819 tablet_replica.cc:331] T b4ea80ef8b3946129c8d5ba57d582504 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.425812 14820 ts_tablet_manager.cc:1939] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.425820 14819 raft_consensus.cc:2238] T b4ea80ef8b3946129c8d5ba57d582504 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.425853 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 99fccdbe3a924305af7794e1ce8b092a with delete_type TABLET_DATA_DELETED (replaced by tablet 876e28105fbb47bdb722e5d3f8467341) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.425901 14819 raft_consensus.cc:2267] T b4ea80ef8b3946129c8d5ba57d582504 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.426407 14819 ts_tablet_manager.cc:1905] T b4ea80ef8b3946129c8d5ba57d582504 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.427175 14824 consensus_queue.cc:1035] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.427502 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 66d755eaad304219a1235031b4841ba8
I20250115 03:06:12.427724 14169 catalog_manager.cc:6596] ReplaceTablet: tablet 66d755eaad304219a1235031b4841ba8 deleted and replaced by tablet feefb2be7f1440b98d7893dbbf9db46c
I20250115 03:06:12.428369 14819 ts_tablet_manager.cc:1918] T b4ea80ef8b3946129c8d5ba57d582504 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.428426 14819 log.cc:1198] T b4ea80ef8b3946129c8d5ba57d582504 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/b4ea80ef8b3946129c8d5ba57d582504
I20250115 03:06:12.429215 14809 tablet_bootstrap.cc:492] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.429306 14809 ts_tablet_manager.cc:1397] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20250115 03:06:12.429327 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 66d755eaad304219a1235031b4841ba8 with delete_type TABLET_DATA_DELETED (replaced by tablet feefb2be7f1440b98d7893dbbf9db46c) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.429697 14670 tablet_service.cc:1514] Processing DeleteTablet for tablet 66d755eaad304219a1235031b4841ba8 with delete_type TABLET_DATA_DELETED (replaced by tablet feefb2be7f1440b98d7893dbbf9db46c) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.429731 14809 raft_consensus.cc:357] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.429843 14809 raft_consensus.cc:383] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.429877 14809 raft_consensus.cc:738] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.429931 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 66d755eaad304219a1235031b4841ba8 with delete_type TABLET_DATA_DELETED (replaced by tablet feefb2be7f1440b98d7893dbbf9db46c) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.429982 14809 consensus_queue.cc:260] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.430235 14809 ts_tablet_manager.cc:1428] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.430385 14809 tablet_bootstrap.cc:492] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.430622 14165 catalog_manager.cc:5526] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:12.430847 14165 catalog_manager.cc:5526] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:12.430999 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 36bda89e64d44e708bccbcab847634a1 (table default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52]). Expected version 1 got 0
I20250115 03:06:12.431123 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 7a78c3eca7d542d7a71f9a33e8585c18 (table default.Table_66cd278959e849dfaf7c9fc9e604d46a [id=6ecde174f5dc422387c42ec4e2f3a053]). Expected version 1 got 0
I20250115 03:06:12.431245 14809 tablet_bootstrap.cc:654] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.428606 14819 ts_tablet_manager.cc:1939] T b4ea80ef8b3946129c8d5ba57d582504 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.432492 14824 raft_consensus.cc:491] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.432570 14824 raft_consensus.cc:513] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.432709 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e1a261c278c04d8ea57ffe2181c5fd0d" } new_table_name: "default.Table_2635fdd75c304cb1a3fd33bf84b66d27" modify_external_catalogs: true
I20250115 03:06:12.432901 14824 leader_election.cc:290] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:12.433250 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a78c3eca7d542d7a71f9a33e8585c18" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:12.433343 14420 raft_consensus.cc:2463] T 7a78c3eca7d542d7a71f9a33e8585c18 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:12.433565 14824 consensus_queue.cc:1035] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.433591 14628 leader_election.cc:304] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.433748 14828 raft_consensus.cc:2798] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:12.433791 14828 raft_consensus.cc:491] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:12.433823 14828 raft_consensus.cc:3054] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.434151 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a78c3eca7d542d7a71f9a33e8585c18" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:12.434672 14626 leader_election.cc:343] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:12.434691 14828 raft_consensus.cc:513] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.435006 14828 leader_election.cc:290] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:12.435392 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a78c3eca7d542d7a71f9a33e8585c18" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:12.435515 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a78c3eca7d542d7a71f9a33e8585c18" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:12.435596 14421 raft_consensus.cc:3054] T 7a78c3eca7d542d7a71f9a33e8585c18 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.435653 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c (table default.Table_3c64fb94bd9c4890989ce6f63525a733 [id=f8ea91c4841b470db1b9f05fa47244fc]) successfully deleted
I20250115 03:06:12.436096 14820 tablet_replica.cc:331] T 66d755eaad304219a1235031b4841ba8 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.436203 14820 raft_consensus.cc:2238] T 66d755eaad304219a1235031b4841ba8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.436287 14820 raft_consensus.cc:2267] T 66d755eaad304219a1235031b4841ba8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.436376 14421 raft_consensus.cc:2463] T 7a78c3eca7d542d7a71f9a33e8585c18 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:12.436480 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d7dd29730eb547fd9d8de4c5cb0269f3" } new_table_name: "default.Table_6bc8705ef8c145478d0ccd20fcb3ce34" modify_external_catalogs: true
I20250115 03:06:12.436795 14820 ts_tablet_manager.cc:1905] T 66d755eaad304219a1235031b4841ba8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:12.436795 14626 leader_election.cc:343] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:12.437022 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 594f755feed947afba1ba8b0853d5788 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 594f755feed947afba1ba8b0853d5788 already in progress: creating tablet
I20250115 03:06:12.437041 14628 leader_election.cc:304] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
W20250115 03:06:12.437172 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 99fccdbe3a924305af7794e1ce8b092a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 99fccdbe3a924305af7794e1ce8b092a already in progress: creating tablet
I20250115 03:06:12.437395 14828 raft_consensus.cc:2798] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.437475 14828 raft_consensus.cc:695] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:12.437582 14828 consensus_queue.cc:237] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.438853 14819 tablet_replica.cc:331] T 66d755eaad304219a1235031b4841ba8 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.439003 14819 raft_consensus.cc:2238] T 66d755eaad304219a1235031b4841ba8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.439162 14819 raft_consensus.cc:2267] T 66d755eaad304219a1235031b4841ba8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.439648 14819 ts_tablet_manager.cc:1905] T 66d755eaad304219a1235031b4841ba8 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.442703 14820 ts_tablet_manager.cc:1918] T 66d755eaad304219a1235031b4841ba8 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.442826 14820 log.cc:1198] T 66d755eaad304219a1235031b4841ba8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/66d755eaad304219a1235031b4841ba8
I20250115 03:06:12.443071 14820 ts_tablet_manager.cc:1939] T 66d755eaad304219a1235031b4841ba8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.443742 14819 ts_tablet_manager.cc:1918] T 66d755eaad304219a1235031b4841ba8 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.443815 14819 log.cc:1198] T 66d755eaad304219a1235031b4841ba8 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/66d755eaad304219a1235031b4841ba8
I20250115 03:06:12.444005 14819 ts_tablet_manager.cc:1939] T 66d755eaad304219a1235031b4841ba8 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.444221 14739 heartbeater.cc:346] Connected to a master server at 127.13.186.254:39901
I20250115 03:06:12.444319 14739 heartbeater.cc:501] Master 127.13.186.254:39901 was elected leader, sending a full tablet report...
I20250115 03:06:12.438084 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 4d6b5a30f27e47a3a515fb34691b9b15 (DEFAULT_TABLE table=default.Table_fadcbb64abcd4a1b812defb8ed9d99f5 [id=e3077355979a401a89f0558104eb53cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.444967 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d6b5a30f27e47a3a515fb34691b9b15. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.444180 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 66d755eaad304219a1235031b4841ba8 (table default.Table_0d7244964d874f5eb1f508e5c0dabf1a [id=15762a37e62f4e89852b2eb66a008dbb]) successfully deleted
I20250115 03:06:12.447149 14809 tablet_bootstrap.cc:492] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.447235 14809 ts_tablet_manager.cc:1397] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
W20250115 03:06:12.447595 14162 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0c8dc386a6b4468d920dbfab906e1366 consensus state has no leader
I20250115 03:06:12.447645 14809 raft_consensus.cc:357] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.447757 14809 raft_consensus.cc:383] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.447796 14809 raft_consensus.cc:738] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.447908 14809 consensus_queue.cc:260] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.448153 14809 ts_tablet_manager.cc:1428] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.448320 14809 tablet_bootstrap.cc:492] T fa7043f411374111b4c81467260fc6cd P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.449213 14809 tablet_bootstrap.cc:654] T fa7043f411374111b4c81467260fc6cd P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.452957 14169 catalog_manager.cc:5526] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:12.453137 14169 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 7a78c3eca7d542d7a71f9a33e8585c18 (table default.Table_66cd278959e849dfaf7c9fc9e604d46a [id=6ecde174f5dc422387c42ec4e2f3a053]). Expected version 1 got 0
I20250115 03:06:12.454353 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 21b146e38bb345fca7a9a159983db68f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.454528 14820 tablet_replica.cc:331] T 21b146e38bb345fca7a9a159983db68f P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.454628 14820 raft_consensus.cc:2238] T 21b146e38bb345fca7a9a159983db68f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.454735 14820 raft_consensus.cc:2267] T 21b146e38bb345fca7a9a159983db68f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.455338 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 4d6b5a30f27e47a3a515fb34691b9b15 (DEFAULT_TABLE table=default.Table_fadcbb64abcd4a1b812defb8ed9d99f5 [id=e3077355979a401a89f0558104eb53cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.455432 14820 ts_tablet_manager.cc:1905] T 21b146e38bb345fca7a9a159983db68f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.455560 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d6b5a30f27e47a3a515fb34691b9b15. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.457185 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4fc6e70f174043999eccc3ec2b20474d" } new_table_name: "default.Table_65b9d218ca9743d7a7eaf7cec1e16c80" modify_external_catalogs: true
I20250115 03:06:12.458184 15056 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" instance_seqno: 1736910370903500) as {username='slave'} at 127.13.186.195:37969; Asking this server to re-register.
I20250115 03:06:12.458832 14739 heartbeater.cc:463] Registering TS with master...
I20250115 03:06:12.459345 14820 ts_tablet_manager.cc:1918] T 21b146e38bb345fca7a9a159983db68f P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.459406 14820 log.cc:1198] T 21b146e38bb345fca7a9a159983db68f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/21b146e38bb345fca7a9a159983db68f
I20250115 03:06:12.459578 14820 ts_tablet_manager.cc:1939] T 21b146e38bb345fca7a9a159983db68f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.460055 15055 ts_manager.cc:194] Registered new tserver with Master: 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.460325 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f590b49e96644f96934046c984688857"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:12.460656 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f156dd7a2fb54e1c95bcf4039284acdf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.460958 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f156dd7a2fb54e1c95bcf4039284acdf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:12.461467 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f590b49e96644f96934046c984688857 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.462371 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 21b146e38bb345fca7a9a159983db68f (table default.Table_7a59f65551fc4aff908198db770fc111 [id=7a9994216f8d437491e6b24c201e9422]) successfully deleted
I20250115 03:06:12.462785 14823 tablet_replica.cc:331] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.462913 14823 raft_consensus.cc:2238] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:12.463099 14812 tablet_bootstrap.cc:492] T 3633e83ec7144230b08e36f6846dbc8e P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.463964 14823 pending_rounds.cc:62] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 1 pending ops.
I20250115 03:06:12.464013 14812 tablet_bootstrap.cc:654] T 3633e83ec7144230b08e36f6846dbc8e P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.464001 14823 pending_rounds.cc:69] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384884809109504 op_type: NO_OP noop_request { }
I20250115 03:06:12.464053 14823 raft_consensus.cc:2883] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:12.464099 14823 raft_consensus.cc:2267] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.464581 14813 raft_consensus.cc:491] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.464656 14813 raft_consensus.cc:513] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.465013 14813 leader_election.cc:290] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.465375 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6771c98e569468ea932882858981d23" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.465478 14556 raft_consensus.cc:2463] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:12.465672 14893 tablet.cc:1721] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:12.465771 14356 leader_election.cc:304] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.465909 14813 raft_consensus.cc:2798] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:12.465955 14813 raft_consensus.cc:491] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:12.465992 14813 raft_consensus.cc:3054] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.466825 14813 raft_consensus.cc:513] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.466969 14969 tablet.cc:1721] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:12.467133 14813 leader_election.cc:290] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.467234 14891 tablet.cc:1721] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:12.467478 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6771c98e569468ea932882858981d23" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:12.467571 14556 raft_consensus.cc:3054] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.468346 14556 raft_consensus.cc:2463] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:12.469409 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 0c8dc386a6b4468d920dbfab906e1366 (table default.Table_fefc309a5c684cc7a466bc67cba0f109 [id=6d1418f7a3b64aebace43d4478f10915]). Expected version 1 got 0
I20250115 03:06:12.469731 14356 leader_election.cc:304] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.469965 14813 raft_consensus.cc:2798] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.470041 14813 raft_consensus.cc:695] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:12.470144 14813 consensus_queue.cc:237] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.470474 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6771c98e569468ea932882858981d23" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:12.470572 14691 raft_consensus.cc:3054] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.471284 14691 raft_consensus.cc:2463] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:12.471809 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet b4ea80ef8b3946129c8d5ba57d582504 (table default.Table_e376c0b790ce48278ea203fdb6bda08d [id=841fc23508e147aeb4f8b8f60f22c94b]) successfully deleted
I20250115 03:06:12.471935 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 66d755eaad304219a1235031b4841ba8 (table default.Table_0d7244964d874f5eb1f508e5c0dabf1a [id=15762a37e62f4e89852b2eb66a008dbb]) successfully deleted
I20250115 03:06:12.472452 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 21b146e38bb345fca7a9a159983db68f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.472627 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6771c98e569468ea932882858981d23" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:12.472704 14688 raft_consensus.cc:2371] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1. Re-sending same reply.
I20250115 03:06:12.473157 14819 tablet_replica.cc:331] T 21b146e38bb345fca7a9a159983db68f P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.473274 14819 raft_consensus.cc:2238] T 21b146e38bb345fca7a9a159983db68f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.473356 14819 raft_consensus.cc:2267] T 21b146e38bb345fca7a9a159983db68f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.473811 14819 ts_tablet_manager.cc:1905] T 21b146e38bb345fca7a9a159983db68f P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.473939 14823 ts_tablet_manager.cc:1905] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.475103 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 4d6b5a30f27e47a3a515fb34691b9b15 (DEFAULT_TABLE table=default.Table_fadcbb64abcd4a1b812defb8ed9d99f5 [id=e3077355979a401a89f0558104eb53cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.475237 14819 ts_tablet_manager.cc:1918] T 21b146e38bb345fca7a9a159983db68f P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.475282 14819 log.cc:1198] T 21b146e38bb345fca7a9a159983db68f P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/21b146e38bb345fca7a9a159983db68f
I20250115 03:06:12.475407 14819 ts_tablet_manager.cc:1939] T 21b146e38bb345fca7a9a159983db68f P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.476284 14556 raft_consensus.cc:1270] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.476466 14691 raft_consensus.cc:1270] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.476469 14557 raft_consensus.cc:1270] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer 5c0dae7aff4b44c699275b95ba639770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.476509 14823 ts_tablet_manager.cc:1918] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:12.476560 14823 log.cc:1198] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/83ac7b09047849da8f6770126c8dc033
I20250115 03:06:12.476788 14823 ts_tablet_manager.cc:1939] T 83ac7b09047849da8f6770126c8dc033 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.476863 14420 raft_consensus.cc:1270] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 5c0dae7aff4b44c699275b95ba639770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.476861 14824 consensus_queue.cc:1035] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.477591 14861 consensus_queue.cc:1035] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.478386 14885 consensus_queue.cc:1035] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.478889 14969 tablet.cc:1721] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:12.479635 14891 tablet.cc:1721] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:12.482306 14824 consensus_queue.cc:1035] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.482916 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_94ee8224b6cd468bb2a1c15b2c2af35a" } modify_external_catalogs: true
I20250115 03:06:12.483007 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_94ee8224b6cd468bb2a1c15b2c2af35a" } modify_external_catalogs: true
I20250115 03:06:12.483865 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_fadcbb64abcd4a1b812defb8ed9d99f5" } modify_external_catalogs: true
I20250115 03:06:12.483947 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_fadcbb64abcd4a1b812defb8ed9d99f5" } modify_external_catalogs: true
I20250115 03:06:12.486034 14891 tablet.cc:1721] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:12.486640 15151 consensus_queue.cc:1035] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.487145 14969 tablet.cc:1721] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:12.487776 14164 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_65b9d218ca9743d7a7eaf7cec1e16c80 [id=1b4a3ef0246a4341869bf5163487cac6] alter complete (version 4)
I20250115 03:06:12.490192 14164 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 3ed567d482aa43ee9e24f992c346ebc3 (table default.Table_2635fdd75c304cb1a3fd33bf84b66d27 [id=e34781ee2864498bbc532144911630da]). Expected version 2 got 0
I20250115 03:06:12.490334 14164 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet cc4b426be9da40a09b64f0e412536fa7 (table default.Table_6bc8705ef8c145478d0ccd20fcb3ce34 [id=37d965d8a5eb408ea3f33313bd22c152]). Expected version 1 got 0
I20250115 03:06:12.492420 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d6b5a30f27e47a3a515fb34691b9b15. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.493234 14809 tablet_bootstrap.cc:492] T fa7043f411374111b4c81467260fc6cd P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.493311 14809 ts_tablet_manager.cc:1397] T fa7043f411374111b4c81467260fc6cd P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.045s user 0.002s sys 0.000s
I20250115 03:06:12.493755 14809 raft_consensus.cc:357] T fa7043f411374111b4c81467260fc6cd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.493849 14809 raft_consensus.cc:383] T fa7043f411374111b4c81467260fc6cd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.493880 14809 raft_consensus.cc:738] T fa7043f411374111b4c81467260fc6cd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.493975 14809 consensus_queue.cc:260] T fa7043f411374111b4c81467260fc6cd P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.494216 14809 ts_tablet_manager.cc:1428] T fa7043f411374111b4c81467260fc6cd P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.494220 14893 tablet.cc:1721] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:12.494355 14809 tablet_bootstrap.cc:492] T 99fccdbe3a924305af7794e1ce8b092a P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.495247 14809 tablet_bootstrap.cc:654] T 99fccdbe3a924305af7794e1ce8b092a P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.496258 14166 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet cc4b426be9da40a09b64f0e412536fa7 (table default.Table_6bc8705ef8c145478d0ccd20fcb3ce34 [id=37d965d8a5eb408ea3f33313bd22c152]). Expected version 1 got 0
I20250115 03:06:12.497241 14164 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet c6cd63ac1c084a0d8f166e63b0ed85bc (table default.Table_5b29b7e1e58f49c09b26040407bf49ce [id=05f9fd2ebec34d2299ea59c47a74c828]). Expected version 3 got 2
I20250115 03:06:12.499130 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 8c21c254cdef4015bfaba3fb5d61f554 (DEFAULT_TABLE table=default.Table_2e115074bf7049bfbb1239b4d9c1ca2e [id=6cf622de00cb490bb794837223aa7bcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.499254 15157 tablet.cc:1721] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:12.499347 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c21c254cdef4015bfaba3fb5d61f554. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.499969 14533 tablet_service.cc:1467] Processing CreateTablet for tablet e4994733ae904c99986cabe4c6efc503 (DEFAULT_TABLE table=default.Table_d0dde0a873df401c9502d832495fb2b1 [id=f7e4534c1f6f48f09e10b84c1d5c458c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.500216 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4994733ae904c99986cabe4c6efc503. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.503101 14823 tablet_replica.cc:331] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.503216 14823 raft_consensus.cc:2238] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.503299 14823 raft_consensus.cc:2267] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.503449 14893 tablet.cc:1721] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:12.503784 14398 tablet_service.cc:1467] Processing CreateTablet for tablet e4994733ae904c99986cabe4c6efc503 (DEFAULT_TABLE table=default.Table_d0dde0a873df401c9502d832495fb2b1 [id=f7e4534c1f6f48f09e10b84c1d5c458c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.503979 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4994733ae904c99986cabe4c6efc503. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.504755 14823 ts_tablet_manager.cc:1905] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.505882 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 876e28105fbb47bdb722e5d3f8467341
I20250115 03:06:12.506505 14823 ts_tablet_manager.cc:1918] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.506561 14823 log.cc:1198] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/81bf6bbe775e42d2a8b0dc61a12a8f5c
I20250115 03:06:12.506753 14823 ts_tablet_manager.cc:1939] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.507259 14823 tablet_replica.cc:331] T fa7043f411374111b4c81467260fc6cd P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.507196 14670 tablet_service.cc:1467] Processing CreateTablet for tablet e4994733ae904c99986cabe4c6efc503 (DEFAULT_TABLE table=default.Table_d0dde0a873df401c9502d832495fb2b1 [id=f7e4534c1f6f48f09e10b84c1d5c458c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.507458 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4994733ae904c99986cabe4c6efc503. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.507578 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 83ac7b09047849da8f6770126c8dc033 (table default.Table_ee5faa8cd90f40199372322d90c5a204 [id=ff4681cab8ce44ceb45ed76c70c1132e]) successfully deleted
I20250115 03:06:12.507669 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c (table default.Table_3c64fb94bd9c4890989ce6f63525a733 [id=f8ea91c4841b470db1b9f05fa47244fc]) successfully deleted
I20250115 03:06:12.507776 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 21b146e38bb345fca7a9a159983db68f (table default.Table_7a59f65551fc4aff908198db770fc111 [id=7a9994216f8d437491e6b24c201e9422]) successfully deleted
I20250115 03:06:12.508983 14162 catalog_manager.cc:5526] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:12.509238 14162 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet c6771c98e569468ea932882858981d23 (table default.Table_44e943822f744731aaee3411219a0765 [id=f56a0de6e56441bfa620902f175feb4a]). Expected version 1 got 0
I20250115 03:06:12.509290 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet 7a78c3eca7d542d7a71f9a33e8585c18 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.509359 14162 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 0c8dc386a6b4468d920dbfab906e1366 (table default.Table_fefc309a5c684cc7a466bc67cba0f109 [id=6d1418f7a3b64aebace43d4478f10915]). Expected version 1 got 0
I20250115 03:06:12.509723 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 876e28105fbb47bdb722e5d3f8467341 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.510207 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 876e28105fbb47bdb722e5d3f8467341 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.510944 14670 tablet_service.cc:1514] Processing DeleteTablet for tablet 876e28105fbb47bdb722e5d3f8467341 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:12.510978 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 876e28105fbb47bdb722e5d3f8467341 because the tablet was not found. No further retry: Not found: Tablet not found: 876e28105fbb47bdb722e5d3f8467341
I20250115 03:06:12.511346 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 4d6b5a30f27e47a3a515fb34691b9b15
W20250115 03:06:12.511873 14156 catalog_manager.cc:4839] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 876e28105fbb47bdb722e5d3f8467341 because the tablet was not found. No further retry: Not found: Tablet not found: 876e28105fbb47bdb722e5d3f8467341
I20250115 03:06:12.512404 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d6b5a30f27e47a3a515fb34691b9b15 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:12.512758 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 4d6b5a30f27e47a3a515fb34691b9b15 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d6b5a30f27e47a3a515fb34691b9b15 already in progress: creating tablet
I20250115 03:06:12.512810 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet 99e3ff9f0b364b2f968e0faf3cace781 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.511763 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_5b29b7e1e58f49c09b26040407bf49ce [id=05f9fd2ebec34d2299ea59c47a74c828] alter complete (version 3)
I20250115 03:06:12.513888 14977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "147c86b98f604d518e71f2ba3121fc21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:12.513979 14977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [sys.catalog]: This master's current role is: LEADER
I20250115 03:06:12.517210 14533 tablet_service.cc:1467] Processing CreateTablet for tablet e379cb7a61a8415994a7419266346d8e (DEFAULT_TABLE table=default.Table_5088472ecce645e2935b8b4cb18aa458 [id=295264d1a39945ee9ff50a0997a5becd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.517347 14532 tablet_service.cc:1467] Processing CreateTablet for tablet 7479bfc96d634011ba15069b5cf6794c (DEFAULT_TABLE table=default.Table_3c64fb94bd9c4890989ce6f63525a733 [id=f8ea91c4841b470db1b9f05fa47244fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.517482 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e379cb7a61a8415994a7419266346d8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.517347 14531 tablet_service.cc:1467] Processing CreateTablet for tablet 338592cc70ad4e61ba90905a424a3871 (DEFAULT_TABLE table=default.Table_ee5faa8cd90f40199372322d90c5a204 [id=ff4681cab8ce44ceb45ed76c70c1132e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.517844 14531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 338592cc70ad4e61ba90905a424a3871. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.518280 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 7a78c3eca7d542d7a71f9a33e8585c18
I20250115 03:06:12.518498 14168 catalog_manager.cc:6596] ReplaceTablet: tablet 7a78c3eca7d542d7a71f9a33e8585c18 deleted and replaced by tablet 03884812ed8d4bd390a8c547494c16e2
I20250115 03:06:12.518817 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 9adb11e56c394461a91c924dc1eb404b (DEFAULT_TABLE table=default.Table_228cef4dbc0c49bc9c06dffd88d1074c [id=d0fc70d8af9944099f8e7ee9a416399d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.518997 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9adb11e56c394461a91c924dc1eb404b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.519405 14969 tablet.cc:1721] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.520015 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7479bfc96d634011ba15069b5cf6794c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.521136 14891 tablet.cc:1721] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.521440 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a78c3eca7d542d7a71f9a33e8585c18 with delete_type TABLET_DATA_DELETED (replaced by tablet 03884812ed8d4bd390a8c547494c16e2) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:12.522362 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 7a78c3eca7d542d7a71f9a33e8585c18 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7a78c3eca7d542d7a71f9a33e8585c18 already in progress: creating tablet
I20250115 03:06:12.522432 14886 mvcc.cc:204] Tried to move back new op lower bound from 7114384885653073920 to 7114384884200407040. Current Snapshot: MvccSnapshot[applied={T|T < 7114384885653073920}]
I20250115 03:06:12.522585 14893 tablet.cc:1721] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.522706 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_66cd278959e849dfaf7c9fc9e604d46a" } modify_external_catalogs: true
I20250115 03:06:12.522790 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_66cd278959e849dfaf7c9fc9e604d46a" } modify_external_catalogs: true
I20250115 03:06:12.528337 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_2635fdd75c304cb1a3fd33bf84b66d27 [id=e34781ee2864498bbc532144911630da] alter complete (version 2)
I20250115 03:06:12.530547 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d6b5a30f27e47a3a515fb34691b9b15 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.531113 14670 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d6b5a30f27e47a3a515fb34691b9b15 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.531643 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 8c21c254cdef4015bfaba3fb5d61f554 (DEFAULT_TABLE table=default.Table_2e115074bf7049bfbb1239b4d9c1ca2e [id=6cf622de00cb490bb794837223aa7bcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.531855 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c21c254cdef4015bfaba3fb5d61f554. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.531873 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 8c21c254cdef4015bfaba3fb5d61f554 (DEFAULT_TABLE table=default.Table_2e115074bf7049bfbb1239b4d9c1ca2e [id=6cf622de00cb490bb794837223aa7bcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.532411 14811 tablet_bootstrap.cc:492] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.532483 14811 ts_tablet_manager.cc:1397] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.136s user 0.002s sys 0.000s
I20250115 03:06:12.532972 14811 raft_consensus.cc:357] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.533078 14811 raft_consensus.cc:383] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.533107 14811 raft_consensus.cc:738] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.533216 14811 consensus_queue.cc:260] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.533265 14823 raft_consensus.cc:2238] T fa7043f411374111b4c81467260fc6cd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.533363 14823 raft_consensus.cc:2267] T fa7043f411374111b4c81467260fc6cd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.533460 14811 ts_tablet_manager.cc:1428] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.533615 14811 tablet_bootstrap.cc:492] T 3633e83ec7144230b08e36f6846dbc8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.534508 14811 tablet_bootstrap.cc:654] T 3633e83ec7144230b08e36f6846dbc8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.534792 14823 ts_tablet_manager.cc:1905] T fa7043f411374111b4c81467260fc6cd P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.535009 14604 heartbeater.cc:346] Connected to a master server at 127.13.186.254:39901
I20250115 03:06:12.535110 14604 heartbeater.cc:501] Master 127.13.186.254:39901 was elected leader, sending a full tablet report...
I20250115 03:06:12.535756 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 99e3ff9f0b364b2f968e0faf3cace781
I20250115 03:06:12.535995 14164 catalog_manager.cc:6596] ReplaceTablet: tablet 99e3ff9f0b364b2f968e0faf3cace781 deleted and replaced by tablet d4f76d0b3a394a5f8aa33ad17b11da34
I20250115 03:06:12.537047 14823 ts_tablet_manager.cc:1918] T fa7043f411374111b4c81467260fc6cd P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.537127 14162 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet 03884812ed8d4bd390a8c547494c16e2
I20250115 03:06:12.537268 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 99e3ff9f0b364b2f968e0faf3cace781 with delete_type TABLET_DATA_DELETED (replaced by tablet d4f76d0b3a394a5f8aa33ad17b11da34) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.537479 14820 tablet_replica.cc:331] T 99e3ff9f0b364b2f968e0faf3cace781 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.537585 14820 raft_consensus.cc:2238] T 99e3ff9f0b364b2f968e0faf3cace781 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.537672 14820 raft_consensus.cc:2267] T 99e3ff9f0b364b2f968e0faf3cace781 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.538071 14820 ts_tablet_manager.cc:1905] T 99e3ff9f0b364b2f968e0faf3cace781 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.538213 14398 tablet_service.cc:1467] Processing CreateTablet for tablet e379cb7a61a8415994a7419266346d8e (DEFAULT_TABLE table=default.Table_5088472ecce645e2935b8b4cb18aa458 [id=295264d1a39945ee9ff50a0997a5becd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.538331 14809 tablet_bootstrap.cc:492] T 99fccdbe3a924305af7794e1ce8b092a P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.538407 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e379cb7a61a8415994a7419266346d8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.538408 14809 ts_tablet_manager.cc:1397] T 99fccdbe3a924305af7794e1ce8b092a P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.044s user 0.001s sys 0.000s
I20250115 03:06:12.538873 14809 raft_consensus.cc:357] T 99fccdbe3a924305af7794e1ce8b092a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.538966 14809 raft_consensus.cc:383] T 99fccdbe3a924305af7794e1ce8b092a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.539064 14809 raft_consensus.cc:738] T 99fccdbe3a924305af7794e1ce8b092a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.539196 14809 consensus_queue.cc:260] T 99fccdbe3a924305af7794e1ce8b092a P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.539927 14820 ts_tablet_manager.cc:1918] T 99e3ff9f0b364b2f968e0faf3cace781 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.539986 14820 log.cc:1198] T 99e3ff9f0b364b2f968e0faf3cace781 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/99e3ff9f0b364b2f968e0faf3cace781
I20250115 03:06:12.539969 14809 ts_tablet_manager.cc:1428] T 99fccdbe3a924305af7794e1ce8b092a P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.540169 14820 ts_tablet_manager.cc:1939] T 99e3ff9f0b364b2f968e0faf3cace781 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.540226 14809 tablet_bootstrap.cc:492] T 3633e83ec7144230b08e36f6846dbc8e P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.541514 14861 raft_consensus.cc:491] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.541591 14861 raft_consensus.cc:513] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:12.541857 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 4d6b5a30f27e47a3a515fb34691b9b15 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d6b5a30f27e47a3a515fb34691b9b15 already in progress: creating tablet
I20250115 03:06:12.541918 14861 leader_election.cc:290] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.543183 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 7479bfc96d634011ba15069b5cf6794c (DEFAULT_TABLE table=default.Table_3c64fb94bd9c4890989ce6f63525a733 [id=f8ea91c4841b470db1b9f05fa47244fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.543413 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7479bfc96d634011ba15069b5cf6794c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.544433 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 99e3ff9f0b364b2f968e0faf3cace781 (table default.Table_0dd17013f6334efe97618f0b520512c5 [id=488a9a6268f64669ae17e1a67caa73a5]) successfully deleted
I20250115 03:06:12.545027 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c8dc386a6b4468d920dbfab906e1366" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.545135 14556 raft_consensus.cc:2463] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:12.545107 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 338592cc70ad4e61ba90905a424a3871 (DEFAULT_TABLE table=default.Table_ee5faa8cd90f40199372322d90c5a204 [id=ff4681cab8ce44ceb45ed76c70c1132e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.545303 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 338592cc70ad4e61ba90905a424a3871. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.545972 14356 leader_election.cc:304] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.546121 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 07d9c5e89ac34af29a406e79d2ace824 (DEFAULT_TABLE table=default.Table_64c4e8668b494feca761d65318286040 [id=265c248361624c3f9e5ece27144d8e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.546322 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07d9c5e89ac34af29a406e79d2ace824. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.546955 14556 raft_consensus.cc:1270] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.547072 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a78c3eca7d542d7a71f9a33e8585c18 with delete_type TABLET_DATA_DELETED (replaced by tablet 03884812ed8d4bd390a8c547494c16e2) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.547199 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c with delete_type TABLET_DATA_DELETED (replaced by tablet 7479bfc96d634011ba15069b5cf6794c) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.547356 15055 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" instance_seqno: 1736910370740004) as {username='slave'} at 127.13.186.194:37695; Asking this server to re-register.
I20250115 03:06:12.547577 14395 tablet_service.cc:1514] Processing DeleteTablet for tablet 99e3ff9f0b364b2f968e0faf3cace781 with delete_type TABLET_DATA_DELETED (replaced by tablet d4f76d0b3a394a5f8aa33ad17b11da34) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.547737 14885 raft_consensus.cc:2798] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:12.547785 14885 raft_consensus.cc:491] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:12.547818 14885 raft_consensus.cc:3054] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.548617 14885 raft_consensus.cc:513] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.548883 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_6bc8705ef8c145478d0ccd20fcb3ce34 [id=37d965d8a5eb408ea3f33313bd22c152] alter complete (version 1)
I20250115 03:06:12.548882 14861 consensus_queue.cc:1035] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.549557 14819 tablet_replica.cc:331] T 7a78c3eca7d542d7a71f9a33e8585c18 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.549531 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c8dc386a6b4468d920dbfab906e1366" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:12.549623 14556 raft_consensus.cc:3054] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.549670 14819 raft_consensus.cc:2238] T 7a78c3eca7d542d7a71f9a33e8585c18 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.549746 14819 raft_consensus.cc:2267] T 7a78c3eca7d542d7a71f9a33e8585c18 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.550413 14556 raft_consensus.cc:2463] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:12.550668 14356 leader_election.cc:304] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.551127 14813 raft_consensus.cc:2798] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.551204 14813 raft_consensus.cc:695] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:12.551332 14813 consensus_queue.cc:237] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.552023 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 99abc1c7a20a4475921597f93c2db56c (DEFAULT_TABLE table=default.Table_3f71dfd17bb74691b7956682ae89fb38 [id=f94088b0776f4be990ddcec04afbca43]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.552230 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99abc1c7a20a4475921597f93c2db56c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.552323 14604 heartbeater.cc:463] Registering TS with master...
I20250115 03:06:12.552763 14819 ts_tablet_manager.cc:1905] T 7a78c3eca7d542d7a71f9a33e8585c18 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.553045 14532 tablet_service.cc:1467] Processing CreateTablet for tablet 876e28105fbb47bdb722e5d3f8467341 (DEFAULT_TABLE table=default.Table_94ee8224b6cd468bb2a1c15b2c2af35a [id=63bd4ca58023436eb2b453f6fa0ea0c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.553225 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 876e28105fbb47bdb722e5d3f8467341. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.554441 15056 ts_manager.cc:194] Registered new tserver with Master: 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:12.555475 14532 tablet_service.cc:1467] Processing CreateTablet for tablet 8a75594237ce49edbc5ba410038bbecd (DEFAULT_TABLE table=default.Table_6d37c9d2d26e4b73ba4c230e471aeef7 [id=b3cf5d084f3348c091c6e5be7be4808f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.555673 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a75594237ce49edbc5ba410038bbecd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.557775 14533 tablet_service.cc:1467] Processing CreateTablet for tablet feefb2be7f1440b98d7893dbbf9db46c (DEFAULT_TABLE table=default.Table_0d7244964d874f5eb1f508e5c0dabf1a [id=15762a37e62f4e89852b2eb66a008dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.557981 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet feefb2be7f1440b98d7893dbbf9db46c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.558712 14819 ts_tablet_manager.cc:1918] T 7a78c3eca7d542d7a71f9a33e8585c18 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.558766 14819 log.cc:1198] T 7a78c3eca7d542d7a71f9a33e8585c18 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/7a78c3eca7d542d7a71f9a33e8585c18
I20250115 03:06:12.558940 14819 ts_tablet_manager.cc:1939] T 7a78c3eca7d542d7a71f9a33e8585c18 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.559187 14811 tablet_bootstrap.cc:492] T 3633e83ec7144230b08e36f6846dbc8e P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.559259 14811 ts_tablet_manager.cc:1397] T 3633e83ec7144230b08e36f6846dbc8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20250115 03:06:12.559343 14819 tablet_replica.cc:331] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.559438 14819 raft_consensus.cc:2238] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.559506 14819 raft_consensus.cc:2267] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.559682 14811 raft_consensus.cc:357] T 3633e83ec7144230b08e36f6846dbc8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.559768 14811 raft_consensus.cc:383] T 3633e83ec7144230b08e36f6846dbc8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.559796 14811 raft_consensus.cc:738] T 3633e83ec7144230b08e36f6846dbc8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.559908 14819 ts_tablet_manager.cc:1905] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.559895 14811 consensus_queue.cc:260] T 3633e83ec7144230b08e36f6846dbc8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.560621 14811 ts_tablet_manager.cc:1428] T 3633e83ec7144230b08e36f6846dbc8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.560767 14811 tablet_bootstrap.cc:492] T 99fccdbe3a924305af7794e1ce8b092a P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.562256 14811 tablet_bootstrap.cc:654] T 99fccdbe3a924305af7794e1ce8b092a P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.562407 14877 mvcc.cc:204] Tried to move back new op lower bound from 7114384885937410048 to 7114384885396221952. Current Snapshot: MvccSnapshot[applied={T|T < 7114384885937410048}]
I20250115 03:06:12.562536 14969 tablet.cc:1721] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.563719 14819 ts_tablet_manager.cc:1918] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.563781 14819 log.cc:1198] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/81bf6bbe775e42d2a8b0dc61a12a8f5c
I20250115 03:06:12.564052 14809 tablet_bootstrap.cc:654] T 3633e83ec7144230b08e36f6846dbc8e P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.564061 14554 raft_consensus.cc:1270] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.564723 14893 tablet.cc:1721] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.566068 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet 3a786573d49e45669a3a2ef2385afc19 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.566802 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 20f86f93812849c4b7852a884f677c04 (DEFAULT_TABLE table=default.Table_6c1f266c05474fa4a4ff61f5078bcfbc [id=f8f2a7ff6fdc40468de6ae9ba28c1427]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.567055 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20f86f93812849c4b7852a884f677c04. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.564287 14395 tablet_service.cc:1467] Processing CreateTablet for tablet 07d9c5e89ac34af29a406e79d2ace824 (DEFAULT_TABLE table=default.Table_64c4e8668b494feca761d65318286040 [id=265c248361624c3f9e5ece27144d8e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.567337 14395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07d9c5e89ac34af29a406e79d2ace824. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.569983 14420 raft_consensus.cc:1270] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 5c0dae7aff4b44c699275b95ba639770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.564524 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 876e28105fbb47bdb722e5d3f8467341 (DEFAULT_TABLE table=default.Table_94ee8224b6cd468bb2a1c15b2c2af35a [id=63bd4ca58023436eb2b453f6fa0ea0c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.570591 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 876e28105fbb47bdb722e5d3f8467341. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.564577 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 20f86f93812849c4b7852a884f677c04 (DEFAULT_TABLE table=default.Table_6c1f266c05474fa4a4ff61f5078bcfbc [id=f8f2a7ff6fdc40468de6ae9ba28c1427]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.572510 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20f86f93812849c4b7852a884f677c04. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.538385 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c21c254cdef4015bfaba3fb5d61f554. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.573789 14812 tablet_bootstrap.cc:492] T 3633e83ec7144230b08e36f6846dbc8e P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.573865 14812 ts_tablet_manager.cc:1397] T 3633e83ec7144230b08e36f6846dbc8e P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.111s user 0.001s sys 0.000s
I20250115 03:06:12.564626 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 99fccdbe3a924305af7794e1ce8b092a with delete_type TABLET_DATA_DELETED (replaced by tablet 876e28105fbb47bdb722e5d3f8467341) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.548940 14885 leader_election.cc:290] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.564719 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet fa7043f411374111b4c81467260fc6cd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.564738 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 8a75594237ce49edbc5ba410038bbecd (DEFAULT_TABLE table=default.Table_6d37c9d2d26e4b73ba4c230e471aeef7 [id=b3cf5d084f3348c091c6e5be7be4808f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.574277 14812 raft_consensus.cc:357] T 3633e83ec7144230b08e36f6846dbc8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.574362 14812 raft_consensus.cc:383] T 3633e83ec7144230b08e36f6846dbc8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.574390 14812 raft_consensus.cc:738] T 3633e83ec7144230b08e36f6846dbc8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.574438 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a75594237ce49edbc5ba410038bbecd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.574504 14812 consensus_queue.cc:260] T 3633e83ec7144230b08e36f6846dbc8e P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.574764 14812 ts_tablet_manager.cc:1428] T 3633e83ec7144230b08e36f6846dbc8e P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.540676 14669 tablet_service.cc:1467] Processing CreateTablet for tablet e379cb7a61a8415994a7419266346d8e (DEFAULT_TABLE table=default.Table_5088472ecce645e2935b8b4cb18aa458 [id=295264d1a39945ee9ff50a0997a5becd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.574923 14812 tablet_bootstrap.cc:492] T 594f755feed947afba1ba8b0853d5788 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.540968 14668 tablet_service.cc:1467] Processing CreateTablet for tablet 9adb11e56c394461a91c924dc1eb404b (DEFAULT_TABLE table=default.Table_228cef4dbc0c49bc9c06dffd88d1074c [id=d0fc70d8af9944099f8e7ee9a416399d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.575104 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9adb11e56c394461a91c924dc1eb404b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.575789 14812 tablet_bootstrap.cc:654] T 594f755feed947afba1ba8b0853d5788 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.564977 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 99abc1c7a20a4475921597f93c2db56c (DEFAULT_TABLE table=default.Table_3f71dfd17bb74691b7956682ae89fb38 [id=f94088b0776f4be990ddcec04afbca43]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.576294 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99abc1c7a20a4475921597f93c2db56c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.541487 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 7479bfc96d634011ba15069b5cf6794c (DEFAULT_TABLE table=default.Table_3c64fb94bd9c4890989ce6f63525a733 [id=f8ea91c4841b470db1b9f05fa47244fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.576985 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7479bfc96d634011ba15069b5cf6794c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.565678 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 9adb11e56c394461a91c924dc1eb404b (DEFAULT_TABLE table=default.Table_228cef4dbc0c49bc9c06dffd88d1074c [id=d0fc70d8af9944099f8e7ee9a416399d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.578192 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9adb11e56c394461a91c924dc1eb404b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.541838 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 338592cc70ad4e61ba90905a424a3871 (DEFAULT_TABLE table=default.Table_ee5faa8cd90f40199372322d90c5a204 [id=ff4681cab8ce44ceb45ed76c70c1132e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.578792 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 338592cc70ad4e61ba90905a424a3871. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.542248 14671 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a78c3eca7d542d7a71f9a33e8585c18 with delete_type TABLET_DATA_DELETED (replaced by tablet 03884812ed8d4bd390a8c547494c16e2) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.542470 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 99e3ff9f0b364b2f968e0faf3cace781 with delete_type TABLET_DATA_DELETED (replaced by tablet d4f76d0b3a394a5f8aa33ad17b11da34) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.580549 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e379cb7a61a8415994a7419266346d8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.544087 15153 raft_consensus.cc:491] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.542248 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c8dc386a6b4468d920dbfab906e1366" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:12.580871 15153 raft_consensus.cc:513] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.581328 15153 leader_election.cc:290] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:12.566300 14861 consensus_queue.cc:1035] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.581549 14688 raft_consensus.cc:2463] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:12.567101 14400 tablet_service.cc:1467] Processing CreateTablet for tablet feefb2be7f1440b98d7893dbbf9db46c (DEFAULT_TABLE table=default.Table_0d7244964d874f5eb1f508e5c0dabf1a [id=15762a37e62f4e89852b2eb66a008dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.582265 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet feefb2be7f1440b98d7893dbbf9db46c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.549129 14691 raft_consensus.cc:1270] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.583361 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c8dc386a6b4468d920dbfab906e1366" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:12.550436 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c8dc386a6b4468d920dbfab906e1366" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:12.583575 14689 raft_consensus.cc:3054] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.584141 14823 log.cc:1198] T fa7043f411374111b4c81467260fc6cd P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/fa7043f411374111b4c81467260fc6cd
I20250115 03:06:12.584342 14689 raft_consensus.cc:2463] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:12.554953 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 07d9c5e89ac34af29a406e79d2ace824 (DEFAULT_TABLE table=default.Table_64c4e8668b494feca761d65318286040 [id=265c248361624c3f9e5ece27144d8e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.584340 14823 ts_tablet_manager.cc:1939] T fa7043f411374111b4c81467260fc6cd P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.584618 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07d9c5e89ac34af29a406e79d2ace824. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.584861 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c8dc386a6b4468d920dbfab906e1366" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.585038 14556 raft_consensus.cc:2388] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c0dae7aff4b44c699275b95ba639770 in current term 1: Already voted for candidate e8df7247ee9c43e0aa18216da2661f75 in this term.
I20250115 03:06:12.585081 14823 tablet_replica.cc:331] T 66a38c4ffeb44212906dbe1207f91005 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.585199 14823 raft_consensus.cc:2238] T 66a38c4ffeb44212906dbe1207f91005 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.555166 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 99abc1c7a20a4475921597f93c2db56c (DEFAULT_TABLE table=default.Table_3f71dfd17bb74691b7956682ae89fb38 [id=f94088b0776f4be990ddcec04afbca43]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.586432 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99abc1c7a20a4475921597f93c2db56c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.588387 15115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "147c86b98f604d518e71f2ba3121fc21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:12.588485 15115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [sys.catalog]: This master's current role is: FOLLOWER
I20250115 03:06:12.588564 15115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 147c86b98f604d518e71f2ba3121fc21. Latest consensus state: current_term: 2 leader_uuid: "147c86b98f604d518e71f2ba3121fc21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:12.588624 15115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [sys.catalog]: This master's current role is: FOLLOWER
I20250115 03:06:12.555338 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 876e28105fbb47bdb722e5d3f8467341 (DEFAULT_TABLE table=default.Table_94ee8224b6cd468bb2a1c15b2c2af35a [id=63bd4ca58023436eb2b453f6fa0ea0c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.588693 15115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "147c86b98f604d518e71f2ba3121fc21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:12.588765 15115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4 [sys.catalog]: This master's current role is: FOLLOWER
I20250115 03:06:12.588817 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 876e28105fbb47bdb722e5d3f8467341. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.555505 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 8a75594237ce49edbc5ba410038bbecd (DEFAULT_TABLE table=default.Table_6d37c9d2d26e4b73ba4c230e471aeef7 [id=b3cf5d084f3348c091c6e5be7be4808f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.589273 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a75594237ce49edbc5ba410038bbecd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.589381 14819 ts_tablet_manager.cc:1939] T 81bf6bbe775e42d2a8b0dc61a12a8f5c P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.589957 14819 tablet_replica.cc:331] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.555810 14661 tablet_service.cc:1467] Processing CreateTablet for tablet feefb2be7f1440b98d7893dbbf9db46c (DEFAULT_TABLE table=default.Table_0d7244964d874f5eb1f508e5c0dabf1a [id=15762a37e62f4e89852b2eb66a008dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.590920 14823 raft_consensus.cc:2267] T 66a38c4ffeb44212906dbe1207f91005 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.591076 14661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet feefb2be7f1440b98d7893dbbf9db46c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.591476 14823 ts_tablet_manager.cc:1905] T 66a38c4ffeb44212906dbe1207f91005 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.592653 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_dd57412817d54552a349868025274901" } modify_external_catalogs: true
I20250115 03:06:12.592741 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_dd57412817d54552a349868025274901" } modify_external_catalogs: true
I20250115 03:06:12.556023 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 20f86f93812849c4b7852a884f677c04 (DEFAULT_TABLE table=default.Table_6c1f266c05474fa4a4ff61f5078bcfbc [id=f8f2a7ff6fdc40468de6ae9ba28c1427]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.593029 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_43170518d7504d1d8d17374e281f2728"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:12.593252 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20f86f93812849c4b7852a884f677c04. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:12.593348 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_43170518d7504d1d8d17374e281f2728 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.563436 14687 raft_consensus.cc:1270] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.570336 14828 consensus_queue.cc:1035] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.593904 14885 consensus_queue.cc:1035] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:12.594677 14626 consensus_peers.cc:487] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 99e3ff9f0b364b2f968e0faf3cace781. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:12.594818 14626 leader_election.cc:304] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:12.594969 15153 raft_consensus.cc:2743] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:12.590063 14819 raft_consensus.cc:2238] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.596230 14819 raft_consensus.cc:2267] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.596654 14819 ts_tablet_manager.cc:1905] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.598050 14885 consensus_queue.cc:1035] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.593262 14823 ts_tablet_manager.cc:1918] T 66a38c4ffeb44212906dbe1207f91005 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.598206 14823 log.cc:1198] T 66a38c4ffeb44212906dbe1207f91005 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/66a38c4ffeb44212906dbe1207f91005
I20250115 03:06:12.598277 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 3a786573d49e45669a3a2ef2385afc19
I20250115 03:06:12.598517 14168 catalog_manager.cc:6596] ReplaceTablet: tablet 3a786573d49e45669a3a2ef2385afc19 deleted and replaced by tablet 24ab3f7a627047069a03f49e2b832bc3
I20250115 03:06:12.598613 14823 ts_tablet_manager.cc:1939] T 66a38c4ffeb44212906dbe1207f91005 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.599187 14880 mvcc.cc:204] Tried to move back new op lower bound from 7114384885937410048 to 7114384885396221952. Current Snapshot: MvccSnapshot[applied={T|T < 7114384885937410048}]
I20250115 03:06:12.599293 14891 tablet.cc:1721] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.599366 14811 tablet_bootstrap.cc:492] T 99fccdbe3a924305af7794e1ce8b092a P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.599447 14811 ts_tablet_manager.cc:1397] T 99fccdbe3a924305af7794e1ce8b092a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.000s
I20250115 03:06:12.599611 14809 tablet_bootstrap.cc:492] T 3633e83ec7144230b08e36f6846dbc8e P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.599675 14809 ts_tablet_manager.cc:1397] T 3633e83ec7144230b08e36f6846dbc8e P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.059s user 0.002s sys 0.000s
I20250115 03:06:12.599862 14811 raft_consensus.cc:357] T 99fccdbe3a924305af7794e1ce8b092a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.599956 14811 raft_consensus.cc:383] T 99fccdbe3a924305af7794e1ce8b092a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.599988 14811 raft_consensus.cc:738] T 99fccdbe3a924305af7794e1ce8b092a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.600097 14811 consensus_queue.cc:260] T 99fccdbe3a924305af7794e1ce8b092a P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.600098 14809 raft_consensus.cc:357] T 3633e83ec7144230b08e36f6846dbc8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.600191 14809 raft_consensus.cc:383] T 3633e83ec7144230b08e36f6846dbc8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.600224 14809 raft_consensus.cc:738] T 3633e83ec7144230b08e36f6846dbc8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
W20250115 03:06:12.600365 14628 consensus_peers.cc:487] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 -> Peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Couldn't send request to peer e8df7247ee9c43e0aa18216da2661f75. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:12.600350 14809 consensus_queue.cc:260] T 3633e83ec7144230b08e36f6846dbc8e P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:12.600487 14628 consensus_peers.cc:487] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 -> Peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Couldn't send request to peer e8df7247ee9c43e0aa18216da2661f75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 66d755eaad304219a1235031b4841ba8. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:12.601343 14811 ts_tablet_manager.cc:1428] T 99fccdbe3a924305af7794e1ce8b092a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:12.601511 14811 tablet_bootstrap.cc:492] T fa7043f411374111b4c81467260fc6cd P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.602540 14823 tablet_replica.cc:331] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.602653 14823 raft_consensus.cc:2238] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.602731 14823 raft_consensus.cc:2267] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.602891 14811 tablet_bootstrap.cc:654] T fa7043f411374111b4c81467260fc6cd P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.604580 14809 ts_tablet_manager.cc:1428] T 3633e83ec7144230b08e36f6846dbc8e P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:12.605178 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet 876e28105fbb47bdb722e5d3f8467341 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.605935 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a786573d49e45669a3a2ef2385afc19 with delete_type TABLET_DATA_DELETED (replaced by tablet 24ab3f7a627047069a03f49e2b832bc3) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.606127 14820 tablet_replica.cc:331] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.606227 14820 raft_consensus.cc:2238] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:12.606397 14820 raft_consensus.cc:2267] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.607000 14820 ts_tablet_manager.cc:1905] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.608186 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a786573d49e45669a3a2ef2385afc19 with delete_type TABLET_DATA_DELETED (replaced by tablet 24ab3f7a627047069a03f49e2b832bc3) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.608726 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 81bf6bbe775e42d2a8b0dc61a12a8f5c (table default.Table_3c64fb94bd9c4890989ce6f63525a733 [id=f8ea91c4841b470db1b9f05fa47244fc]) successfully deleted
I20250115 03:06:12.608992 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet fa7043f411374111b4c81467260fc6cd (table default.Table_c1020680eef344a49998eddac921fea5 [id=649880f1ff0d4cfda2358b408110d8dc]) successfully deleted
I20250115 03:06:12.609001 14820 ts_tablet_manager.cc:1918] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.609063 14820 log.cc:1198] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/3a786573d49e45669a3a2ef2385afc19
I20250115 03:06:12.609279 14820 ts_tablet_manager.cc:1939] T 3a786573d49e45669a3a2ef2385afc19 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.609349 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 66a38c4ffeb44212906dbe1207f91005 (table default.Table_3f71dfd17bb74691b7956682ae89fb38 [id=f94088b0776f4be990ddcec04afbca43]) successfully deleted
I20250115 03:06:12.609395 14823 ts_tablet_manager.cc:1905] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.610041 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a86904006b724cb980fe7575e6b6c13a" } modify_external_catalogs: true
I20250115 03:06:12.610131 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a86904006b724cb980fe7575e6b6c13a" } modify_external_catalogs: true
I20250115 03:06:12.610706 14809 tablet_bootstrap.cc:492] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.610791 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 36bda89e64d44e708bccbcab847634a1 (table default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52]). Expected version 1 got 0
I20250115 03:06:12.611070 14969 tablet.cc:1721] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.611130 14162 catalog_manager.cc:5526] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:12.611325 14162 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 0c8dc386a6b4468d920dbfab906e1366 (table default.Table_fefc309a5c684cc7a466bc67cba0f109 [id=6d1418f7a3b64aebace43d4478f10915]). Expected version 1 got 0
I20250115 03:06:12.611682 14809 tablet_bootstrap.cc:654] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.612756 14893 tablet.cc:1721] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.612824 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a786573d49e45669a3a2ef2385afc19 with delete_type TABLET_DATA_DELETED (replaced by tablet 24ab3f7a627047069a03f49e2b832bc3) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.613606 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 3a786573d49e45669a3a2ef2385afc19 (table default.Table_a86904006b724cb980fe7575e6b6c13a [id=9f650455cb3440a49a11010f541084db]) successfully deleted
I20250115 03:06:12.614183 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_3e807f869ab742ddaf0412ef84007cfe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.614485 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3e807f869ab742ddaf0412ef84007cfe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.614805 14819 ts_tablet_manager.cc:1918] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.614866 14819 log.cc:1198] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/99e3ff9f0b364b2f968e0faf3cace781
W20250115 03:06:12.615679 14628 consensus_peers.cc:487] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 -> Peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Couldn't send request to peer e8df7247ee9c43e0aa18216da2661f75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a78c3eca7d542d7a71f9a33e8585c18. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:12.615854 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_669098605d764723aba5170b1c1f12a1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.616120 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_669098605d764723aba5170b1c1f12a1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.616758 14165 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet 24ab3f7a627047069a03f49e2b832bc3
I20250115 03:06:12.617427 14557 raft_consensus.cc:3054] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.618309 14557 raft_consensus.cc:1270] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer 5c0dae7aff4b44c699275b95ba639770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.618466 14819 ts_tablet_manager.cc:1939] T 99e3ff9f0b364b2f968e0faf3cace781 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.618927 14828 consensus_queue.cc:1035] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.619805 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 3633e83ec7144230b08e36f6846dbc8e
I20250115 03:06:12.620725 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 99e3ff9f0b364b2f968e0faf3cace781 (table default.Table_0dd17013f6334efe97618f0b520512c5 [id=488a9a6268f64669ae17e1a67caa73a5]) successfully deleted
I20250115 03:06:12.621168 14858 consensus_queue.cc:1035] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.621289 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35 (DEFAULT_TABLE table=default.Table_f156dd7a2fb54e1c95bcf4039284acdf [id=32678ca965cc458daceb1ca31e5214af]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.621497 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.623520 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet c6771c98e569468ea932882858981d23 (table default.Table_44e943822f744731aaee3411219a0765 [id=f56a0de6e56441bfa620902f175feb4a]). Expected version 1 got 0
I20250115 03:06:12.623674 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 3633e83ec7144230b08e36f6846dbc8e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.624478 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 3633e83ec7144230b08e36f6846dbc8e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.625411 14395 tablet_service.cc:1467] Processing CreateTablet for tablet b6a8a8da56274eaa8fef29fff38a7dae (DEFAULT_TABLE table=default.Table_f590b49e96644f96934046c984688857 [id=fd7ff6d50c944f3a80089030d9fc895d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.625644 14395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6a8a8da56274eaa8fef29fff38a7dae. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.626269 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e83118015644482ca56543e76b9691bd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.626559 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e83118015644482ca56543e76b9691bd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.627208 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_8a1be82e3d5648a9b4b5399d2f56e72f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.627485 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a1be82e3d5648a9b4b5399d2f56e72f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.627777 14809 tablet_bootstrap.cc:492] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.627846 14809 ts_tablet_manager.cc:1397] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20250115 03:06:12.628113 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_937e6e7e4b014731b5945f00775aa14c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:12.628273 14809 raft_consensus.cc:357] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:12.628360 14809 raft_consensus.cc:383] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20250115 03:06:12.628367 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_937e6e7e4b014731b5945f00775aa14c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.628391 14809 raft_consensus.cc:738] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.628516 14809 consensus_queue.cc:260] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:12.628814 14809 ts_tablet_manager.cc:1428] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.628988 14809 tablet_bootstrap.cc:492] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.629807 14809 tablet_bootstrap.cc:654] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.632566 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 3633e83ec7144230b08e36f6846dbc8e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.632753 14820 tablet_replica.cc:331] T 3633e83ec7144230b08e36f6846dbc8e P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.632788 14532 tablet_service.cc:1467] Processing CreateTablet for tablet b6a8a8da56274eaa8fef29fff38a7dae (DEFAULT_TABLE table=default.Table_f590b49e96644f96934046c984688857 [id=fd7ff6d50c944f3a80089030d9fc895d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.632854 14820 raft_consensus.cc:2238] T 3633e83ec7144230b08e36f6846dbc8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.633013 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6a8a8da56274eaa8fef29fff38a7dae. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.633262 14163 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52] alter complete (version 1)
I20250115 03:06:12.634351 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35 (DEFAULT_TABLE table=default.Table_f156dd7a2fb54e1c95bcf4039284acdf [id=32678ca965cc458daceb1ca31e5214af]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.634404 14820 raft_consensus.cc:2267] T 3633e83ec7144230b08e36f6846dbc8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.634574 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.634732 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a78c3eca7d542d7a71f9a33e8585c18 with delete_type TABLET_DATA_DELETED (replaced by tablet 03884812ed8d4bd390a8c547494c16e2) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.635114 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35 (DEFAULT_TABLE table=default.Table_f156dd7a2fb54e1c95bcf4039284acdf [id=32678ca965cc458daceb1ca31e5214af]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.635308 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.636183 14820 ts_tablet_manager.cc:1905] T 3633e83ec7144230b08e36f6846dbc8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.636441 14891 tablet.cc:1721] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.638430 14813 raft_consensus.cc:491] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.638509 14813 raft_consensus.cc:513] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:12.638829 14813 leader_election.cc:290] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.639235 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "594f755feed947afba1ba8b0853d5788" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.639362 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "594f755feed947afba1ba8b0853d5788" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:12.639597 14356 leader_election.cc:343] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:12.639719 14358 leader_election.cc:343] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:12.639782 14358 leader_election.cc:304] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:12.639966 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_92483ac55d3d40249e8b468ee1511225"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.640236 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_92483ac55d3d40249e8b468ee1511225 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.641736 14820 ts_tablet_manager.cc:1918] T 3633e83ec7144230b08e36f6846dbc8e P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.641796 14820 log.cc:1198] T 3633e83ec7144230b08e36f6846dbc8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/3633e83ec7144230b08e36f6846dbc8e
I20250115 03:06:12.641912 14813 raft_consensus.cc:2743] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:12.641978 14820 ts_tablet_manager.cc:1939] T 3633e83ec7144230b08e36f6846dbc8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.643513 14820 tablet_replica.cc:331] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.643620 14820 raft_consensus.cc:2238] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.643695 14820 pending_rounds.cc:62] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1: Trying to abort 2 pending ops.
I20250115 03:06:12.643719 14820 pending_rounds.cc:69] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7114384886213754880 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "7a78c3eca7d542d7a71f9a33e8585c18" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_66cd278959e849dfaf7c9fc9e604d46a" dest_uuid: "5c0dae7aff4b44c699275b95ba639770" new_extra_config { } }
I20250115 03:06:12.643863 14820 pending_rounds.cc:69] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384885505015808 op_type: NO_OP noop_request { }
I20250115 03:06:12.643914 14820 raft_consensus.cc:2883] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:12.645787 14820 raft_consensus.cc:2267] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.648643 14819 tablet_replica.cc:331] T 99fccdbe3a924305af7794e1ce8b092a P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.648772 14819 raft_consensus.cc:2238] T 99fccdbe3a924305af7794e1ce8b092a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.648859 14819 raft_consensus.cc:2267] T 99fccdbe3a924305af7794e1ce8b092a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.649282 14819 ts_tablet_manager.cc:1905] T 99fccdbe3a924305af7794e1ce8b092a P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.650208 14823 ts_tablet_manager.cc:1918] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.650286 14823 log.cc:1198] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/21b146e38bb345fca7a9a159983db68f
I20250115 03:06:12.650488 14823 ts_tablet_manager.cc:1939] T 21b146e38bb345fca7a9a159983db68f P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.650822 14162 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_44e943822f744731aaee3411219a0765 [id=f56a0de6e56441bfa620902f175feb4a] alter complete (version 1)
I20250115 03:06:12.651079 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 21b146e38bb345fca7a9a159983db68f (table default.Table_7a59f65551fc4aff908198db770fc111 [id=7a9994216f8d437491e6b24c201e9422]) successfully deleted
I20250115 03:06:12.652045 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 594f755feed947afba1ba8b0853d5788 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.653208 14809 tablet_bootstrap.cc:492] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.653290 14809 ts_tablet_manager.cc:1397] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.000s
I20250115 03:06:12.653717 14809 raft_consensus.cc:357] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.654557 14809 raft_consensus.cc:383] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.654606 14809 raft_consensus.cc:738] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.654724 14809 consensus_queue.cc:260] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.655496 14809 ts_tablet_manager.cc:1428] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:12.655666 14809 tablet_bootstrap.cc:492] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.656713 14809 tablet_bootstrap.cc:654] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.657078 14823 tablet_replica.cc:331] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.657193 14823 raft_consensus.cc:2238] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.657279 14823 raft_consensus.cc:2267] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.657712 14823 ts_tablet_manager.cc:1905] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.659345 14556 raft_consensus.cc:1270] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.659373 14689 raft_consensus.cc:1270] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250115 03:06:12.659420 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 594f755feed947afba1ba8b0853d5788 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 594f755feed947afba1ba8b0853d5788 already in progress: creating tablet
I20250115 03:06:12.659679 14813 consensus_queue.cc:1035] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.659799 14861 consensus_queue.cc:1035] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.660539 14823 ts_tablet_manager.cc:1918] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.660645 14823 log.cc:1198] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/99fccdbe3a924305af7794e1ce8b092a
I20250115 03:06:12.660954 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 3633e83ec7144230b08e36f6846dbc8e (table default.Table_dd57412817d54552a349868025274901 [id=8144a2c71b2d475597cd54b3ff2ba720]) successfully deleted
I20250115 03:06:12.661094 14823 ts_tablet_manager.cc:1939] T 99fccdbe3a924305af7794e1ce8b092a P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.661182 14166 master_service.cc:946] ReplaceTablet: received request to replace tablet cc4b426be9da40a09b64f0e412536fa7 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.661396 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e9f7a3afca894ff18b614971924bbd50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.661723 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e9f7a3afca894ff18b614971924bbd50 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.661854 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f156dd7a2fb54e1c95bcf4039284acdf" } new_table_name: "default.Table_80b3b9aaffe94c7a9fef7e98b4fefb53" modify_external_catalogs: true
I20250115 03:06:12.663074 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3f71dfd17bb74691b7956682ae89fb38" } new_table_name: "default.Table_35d1dd748a6f4e79adced77fe7b2d08d" modify_external_catalogs: true
I20250115 03:06:12.663094 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3c64fb94bd9c4890989ce6f63525a733" } new_table_name: "default.Table_dffdb65871dc41b0ba48073022408925" modify_external_catalogs: true
I20250115 03:06:12.664242 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a565af7122644be3bb6f89259eaefa36" } new_table_name: "default.Table_f957fe3487ec4546a46a8392108003f0" modify_external_catalogs: true
I20250115 03:06:12.664896 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_228cef4dbc0c49bc9c06dffd88d1074c" } new_table_name: "default.Table_ea2592c9eb974e6ab84a4e2f799b5cae" modify_external_catalogs: true
I20250115 03:06:12.665601 14819 ts_tablet_manager.cc:1918] T 99fccdbe3a924305af7794e1ce8b092a P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.665657 14819 log.cc:1198] T 99fccdbe3a924305af7794e1ce8b092a P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/99fccdbe3a924305af7794e1ce8b092a
I20250115 03:06:12.665835 14819 ts_tablet_manager.cc:1939] T 99fccdbe3a924305af7794e1ce8b092a P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.666062 14171 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 0c8dc386a6b4468d920dbfab906e1366 (table default.Table_fefc309a5c684cc7a466bc67cba0f109 [id=6d1418f7a3b64aebace43d4478f10915]). Expected version 1 got 0
I20250115 03:06:12.666853 14811 tablet_bootstrap.cc:492] T fa7043f411374111b4c81467260fc6cd P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.666935 14811 ts_tablet_manager.cc:1397] T fa7043f411374111b4c81467260fc6cd P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.065s user 0.001s sys 0.000s
I20250115 03:06:12.667356 14811 raft_consensus.cc:357] T fa7043f411374111b4c81467260fc6cd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.667464 14811 raft_consensus.cc:383] T fa7043f411374111b4c81467260fc6cd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.667495 14811 raft_consensus.cc:738] T fa7043f411374111b4c81467260fc6cd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.667629 14811 consensus_queue.cc:260] T fa7043f411374111b4c81467260fc6cd P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.668977 14811 ts_tablet_manager.cc:1428] T fa7043f411374111b4c81467260fc6cd P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:12.669140 14811 tablet_bootstrap.cc:492] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.670027 14811 tablet_bootstrap.cc:654] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.674158 14853 consensus_queue.cc:1035] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.674876 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet cc4b426be9da40a09b64f0e412536fa7
I20250115 03:06:12.675125 14166 catalog_manager.cc:6596] ReplaceTablet: tablet cc4b426be9da40a09b64f0e412536fa7 deleted and replaced by tablet f21761ff87ac43d5b9b47b5905fa5867
I20250115 03:06:12.676680 14969 tablet.cc:1721] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.677760 14891 tablet.cc:1721] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20250115 03:06:12.678360 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 99abc1c7a20a4475921597f93c2db56c consensus state has no leader
W20250115 03:06:12.678606 14162 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35 consensus state has no leader
W20250115 03:06:12.679050 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7479bfc96d634011ba15069b5cf6794c consensus state has no leader
I20250115 03:06:12.679643 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet cc4b426be9da40a09b64f0e412536fa7 with delete_type TABLET_DATA_DELETED (replaced by tablet f21761ff87ac43d5b9b47b5905fa5867) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.681095 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 99fccdbe3a924305af7794e1ce8b092a with delete_type TABLET_DATA_DELETED (replaced by tablet 876e28105fbb47bdb722e5d3f8467341) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.682494 14689 raft_consensus.cc:1270] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.682718 14554 raft_consensus.cc:1270] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:12.682919 14813 consensus_queue.cc:1035] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.683235 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_040bb2a68caa47cd995d0f4936302d43"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.683547 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_040bb2a68caa47cd995d0f4936302d43 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.684846 14813 consensus_queue.cc:1035] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:12.685073 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f5eb08f6d7974c8895385279b9b47805"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.685354 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f5eb08f6d7974c8895385279b9b47805 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.686499 14164 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 0c8dc386a6b4468d920dbfab906e1366 (table default.Table_fefc309a5c684cc7a466bc67cba0f109 [id=6d1418f7a3b64aebace43d4478f10915]). Expected version 1 got 0
I20250115 03:06:12.687817 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet cc4b426be9da40a09b64f0e412536fa7 with delete_type TABLET_DATA_DELETED (replaced by tablet f21761ff87ac43d5b9b47b5905fa5867) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.689332 14400 tablet_service.cc:1467] Processing CreateTablet for tablet fc056b9dbc374d26bf3feaf524f4d834 (DEFAULT_TABLE table=default.Table_b947c3886a464283ad54fe14793e5618 [id=3bcfb6a13f2b4d56b801fcf210764fea]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.689584 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc056b9dbc374d26bf3feaf524f4d834. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.689828 14663 tablet_service.cc:1467] Processing CreateTablet for tablet b6a8a8da56274eaa8fef29fff38a7dae (DEFAULT_TABLE table=default.Table_f590b49e96644f96934046c984688857 [id=fd7ff6d50c944f3a80089030d9fc895d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.690047 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6a8a8da56274eaa8fef29fff38a7dae. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.691087 14813 raft_consensus.cc:491] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.691156 14813 raft_consensus.cc:513] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.691494 14813 leader_election.cc:290] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.691869 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecd4741aacb45e2ae6404675f885125" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:12.691996 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecd4741aacb45e2ae6404675f885125" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:12.692183 14358 leader_election.cc:343] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:12.692379 14356 leader_election.cc:343] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:12.692479 14356 leader_election.cc:304] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:12.692651 14823 tablet_replica.cc:331] T b4ea80ef8b3946129c8d5ba57d582504 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.692709 14813 raft_consensus.cc:2743] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:12.692785 14823 raft_consensus.cc:2238] T b4ea80ef8b3946129c8d5ba57d582504 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.692867 14823 raft_consensus.cc:2267] T b4ea80ef8b3946129c8d5ba57d582504 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.693298 14823 ts_tablet_manager.cc:1905] T b4ea80ef8b3946129c8d5ba57d582504 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.694555 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 99fccdbe3a924305af7794e1ce8b092a (table default.Table_94ee8224b6cd468bb2a1c15b2c2af35a [id=63bd4ca58023436eb2b453f6fa0ea0c4]) successfully deleted
I20250115 03:06:12.694741 14819 tablet_replica.cc:331] T fa7043f411374111b4c81467260fc6cd P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
W20250115 03:06:12.694818 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9adb11e56c394461a91c924dc1eb404b consensus state has no leader
I20250115 03:06:12.696344 14823 ts_tablet_manager.cc:1918] T b4ea80ef8b3946129c8d5ba57d582504 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.696417 14823 log.cc:1198] T b4ea80ef8b3946129c8d5ba57d582504 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/b4ea80ef8b3946129c8d5ba57d582504
I20250115 03:06:12.696615 14823 ts_tablet_manager.cc:1939] T b4ea80ef8b3946129c8d5ba57d582504 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.696848 14819 raft_consensus.cc:2238] T fa7043f411374111b4c81467260fc6cd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.696975 14819 raft_consensus.cc:2267] T fa7043f411374111b4c81467260fc6cd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.697393 14819 ts_tablet_manager.cc:1905] T fa7043f411374111b4c81467260fc6cd P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.697185 14165 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_fefc309a5c684cc7a466bc67cba0f109 [id=6d1418f7a3b64aebace43d4478f10915] alter complete (version 1)
I20250115 03:06:12.699194 14819 ts_tablet_manager.cc:1918] T fa7043f411374111b4c81467260fc6cd P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.699263 14819 log.cc:1198] T fa7043f411374111b4c81467260fc6cd P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/fa7043f411374111b4c81467260fc6cd
I20250115 03:06:12.699390 14168 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 9c54a5df35e7422b84acc91e3a10aac3 (table default.Table_f957fe3487ec4546a46a8392108003f0 [id=fb3a764b51e74e08a281d128e21466e3]). Expected version 1 got 0
I20250115 03:06:12.699440 14819 ts_tablet_manager.cc:1939] T fa7043f411374111b4c81467260fc6cd P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.699543 14663 tablet_service.cc:1467] Processing CreateTablet for tablet fc056b9dbc374d26bf3feaf524f4d834 (DEFAULT_TABLE table=default.Table_b947c3886a464283ad54fe14793e5618 [id=3bcfb6a13f2b4d56b801fcf210764fea]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.699800 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc056b9dbc374d26bf3feaf524f4d834. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.699800 14400 tablet_service.cc:1467] Processing CreateTablet for tablet d4f76d0b3a394a5f8aa33ad17b11da34 (DEFAULT_TABLE table=default.Table_0dd17013f6334efe97618f0b520512c5 [id=488a9a6268f64669ae17e1a67caa73a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.700162 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4f76d0b3a394a5f8aa33ad17b11da34. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.701756 14969 tablet.cc:1721] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.707365 14663 tablet_service.cc:1467] Processing CreateTablet for tablet d4f76d0b3a394a5f8aa33ad17b11da34 (DEFAULT_TABLE table=default.Table_0dd17013f6334efe97618f0b520512c5 [id=488a9a6268f64669ae17e1a67caa73a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.707630 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4f76d0b3a394a5f8aa33ad17b11da34. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.708048 14168 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 9c54a5df35e7422b84acc91e3a10aac3 (table default.Table_f957fe3487ec4546a46a8392108003f0 [id=fb3a764b51e74e08a281d128e21466e3]). Expected version 1 got 0
I20250115 03:06:12.709436 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet b4ea80ef8b3946129c8d5ba57d582504 (table default.Table_e376c0b790ce48278ea203fdb6bda08d [id=841fc23508e147aeb4f8b8f60f22c94b]) successfully deleted
I20250115 03:06:12.709259 14823 tablet_replica.cc:331] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.710382 14823 raft_consensus.cc:2238] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:12.710541 14823 pending_rounds.cc:62] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 1 pending ops.
I20250115 03:06:12.710585 14823 pending_rounds.cc:69] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384884578865152 op_type: NO_OP noop_request { }
I20250115 03:06:12.710645 14823 raft_consensus.cc:2883] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:12.710703 14823 raft_consensus.cc:2267] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.697158 14811 tablet_bootstrap.cc:492] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.711282 14811 ts_tablet_manager.cc:1397] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.042s user 0.002s sys 0.000s
I20250115 03:06:12.711549 14823 ts_tablet_manager.cc:1905] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.711833 14811 raft_consensus.cc:357] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.711985 14811 raft_consensus.cc:383] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.712054 14811 raft_consensus.cc:738] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.712210 14811 consensus_queue.cc:260] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.712540 14811 ts_tablet_manager.cc:1428] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.712982 14531 tablet_service.cc:1467] Processing CreateTablet for tablet d4f76d0b3a394a5f8aa33ad17b11da34 (DEFAULT_TABLE table=default.Table_0dd17013f6334efe97618f0b520512c5 [id=488a9a6268f64669ae17e1a67caa73a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.713078 14819 tablet_replica.cc:331] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.713177 14819 raft_consensus.cc:2238] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.713188 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet fa7043f411374111b4c81467260fc6cd (table default.Table_c1020680eef344a49998eddac921fea5 [id=649880f1ff0d4cfda2358b408110d8dc]) successfully deleted
I20250115 03:06:12.713770 14531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4f76d0b3a394a5f8aa33ad17b11da34. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.714752 14811 tablet_bootstrap.cc:492] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.715611 14165 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 0c8dc386a6b4468d920dbfab906e1366 (table default.Table_fefc309a5c684cc7a466bc67cba0f109 [id=6d1418f7a3b64aebace43d4478f10915]). Expected version 1 got 0
I20250115 03:06:12.715703 14165 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 9c54a5df35e7422b84acc91e3a10aac3 (table default.Table_f957fe3487ec4546a46a8392108003f0 [id=fb3a764b51e74e08a281d128e21466e3]). Expected version 1 got 0
I20250115 03:06:12.715972 14812 tablet_bootstrap.cc:492] T 594f755feed947afba1ba8b0853d5788 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.716044 14812 ts_tablet_manager.cc:1397] T 594f755feed947afba1ba8b0853d5788 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.141s user 0.001s sys 0.000s
I20250115 03:06:12.716478 14812 raft_consensus.cc:357] T 594f755feed947afba1ba8b0853d5788 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:12.716573 14812 raft_consensus.cc:383] T 594f755feed947afba1ba8b0853d5788 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.716527 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_80b3b9aaffe94c7a9fef7e98b4fefb53" } modify_external_catalogs: true
I20250115 03:06:12.716602 14812 raft_consensus.cc:738] T 594f755feed947afba1ba8b0853d5788 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.716625 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_80b3b9aaffe94c7a9fef7e98b4fefb53" } modify_external_catalogs: true
I20250115 03:06:12.716708 14812 consensus_queue.cc:260] T 594f755feed947afba1ba8b0853d5788 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:12.716977 14812 ts_tablet_manager.cc:1428] T 594f755feed947afba1ba8b0853d5788 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.717149 14812 tablet_bootstrap.cc:492] T 0ecd4741aacb45e2ae6404675f885125 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.718111 14812 tablet_bootstrap.cc:654] T 0ecd4741aacb45e2ae6404675f885125 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.718482 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_fefc309a5c684cc7a466bc67cba0f109" } modify_external_catalogs: true
I20250115 03:06:12.718581 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_fefc309a5c684cc7a466bc67cba0f109" } modify_external_catalogs: true
I20250115 03:06:12.720016 14811 tablet_bootstrap.cc:654] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.702142 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet fa7043f411374111b4c81467260fc6cd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.702178 14530 tablet_service.cc:1467] Processing CreateTablet for tablet fc056b9dbc374d26bf3feaf524f4d834 (DEFAULT_TABLE table=default.Table_b947c3886a464283ad54fe14793e5618 [id=3bcfb6a13f2b4d56b801fcf210764fea]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.703722 14893 tablet.cc:1721] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.720860 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc056b9dbc374d26bf3feaf524f4d834. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.705350 15157 tablet.cc:1721] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.698325 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet cc4b426be9da40a09b64f0e412536fa7 with delete_type TABLET_DATA_DELETED (replaced by tablet f21761ff87ac43d5b9b47b5905fa5867) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.727967 14809 tablet_bootstrap.cc:492] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.728047 14809 ts_tablet_manager.cc:1397] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.072s user 0.002s sys 0.000s
I20250115 03:06:12.728453 14809 raft_consensus.cc:357] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.728545 14809 raft_consensus.cc:383] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.728574 14809 raft_consensus.cc:738] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.728677 14809 consensus_queue.cc:260] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.728909 14809 ts_tablet_manager.cc:1428] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.729094 14809 tablet_bootstrap.cc:492] T 4d6b5a30f27e47a3a515fb34691b9b15 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.713279 14819 raft_consensus.cc:2267] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.729961 14809 tablet_bootstrap.cc:654] T 4d6b5a30f27e47a3a515fb34691b9b15 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.730278 14819 ts_tablet_manager.cc:1905] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.733191 14891 tablet.cc:1721] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:12.734627 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35
I20250115 03:06:12.736106 14168 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_f957fe3487ec4546a46a8392108003f0 [id=fb3a764b51e74e08a281d128e21466e3] alter complete (version 1)
I20250115 03:06:12.736838 14811 tablet_bootstrap.cc:492] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.736915 14811 ts_tablet_manager.cc:1397] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.000s
I20250115 03:06:12.737102 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 0c8dc386a6b4468d920dbfab906e1366
I20250115 03:06:12.737735 14811 raft_consensus.cc:357] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.737828 14811 raft_consensus.cc:383] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.737859 14811 raft_consensus.cc:738] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.737986 14811 consensus_queue.cc:260] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.738590 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.739800 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c8dc386a6b4468d920dbfab906e1366 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.739974 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 594f755feed947afba1ba8b0853d5788 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.740406 14819 ts_tablet_manager.cc:1918] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.739820 14823 ts_tablet_manager.cc:1918] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:12.740583 14402 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c8dc386a6b4468d920dbfab906e1366 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.740612 14823 log.cc:1198] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/66d755eaad304219a1235031b4841ba8
I20250115 03:06:12.740882 14823 ts_tablet_manager.cc:1939] T 66d755eaad304219a1235031b4841ba8 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.741430 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 66d755eaad304219a1235031b4841ba8 (table default.Table_0d7244964d874f5eb1f508e5c0dabf1a [id=15762a37e62f4e89852b2eb66a008dbb]) successfully deleted
I20250115 03:06:12.742370 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.742802 14811 ts_tablet_manager.cc:1428] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
I20250115 03:06:12.740466 14819 log.cc:1198] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/3a786573d49e45669a3a2ef2385afc19
I20250115 03:06:12.743844 14819 ts_tablet_manager.cc:1939] T 3a786573d49e45669a3a2ef2385afc19 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.744330 14819 tablet_replica.cc:331] T 3633e83ec7144230b08e36f6846dbc8e P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.744758 14811 tablet_bootstrap.cc:492] T 594f755feed947afba1ba8b0853d5788 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.745735 14811 tablet_bootstrap.cc:654] T 594f755feed947afba1ba8b0853d5788 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.745844 14823 tablet_replica.cc:331] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.745977 14823 raft_consensus.cc:2238] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:12.746116 14823 pending_rounds.cc:62] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 2 pending ops.
I20250115 03:06:12.746160 14823 pending_rounds.cc:69] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7114384886213754880 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "7a78c3eca7d542d7a71f9a33e8585c18" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_66cd278959e849dfaf7c9fc9e604d46a" dest_uuid: "5c0dae7aff4b44c699275b95ba639770" new_extra_config { } }
W20250115 03:06:12.746304 14823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:36830: Aborted: Op aborted
I20250115 03:06:12.746526 14819 raft_consensus.cc:2238] T 3633e83ec7144230b08e36f6846dbc8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
W20250115 03:06:12.746595 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 876e28105fbb47bdb722e5d3f8467341 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 876e28105fbb47bdb722e5d3f8467341 already in progress: creating tablet
I20250115 03:06:12.746630 14819 raft_consensus.cc:2267] T 3633e83ec7144230b08e36f6846dbc8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:12.746738 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 4d6b5a30f27e47a3a515fb34691b9b15 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d6b5a30f27e47a3a515fb34691b9b15 already in progress: creating tablet
I20250115 03:06:12.746779 14823 pending_rounds.cc:69] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384885505015808 op_type: NO_OP noop_request { }
I20250115 03:06:12.746846 14823 raft_consensus.cc:2883] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:12.746904 14823 raft_consensus.cc:2267] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.747267 14819 ts_tablet_manager.cc:1905] T 3633e83ec7144230b08e36f6846dbc8e P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.748983 14819 ts_tablet_manager.cc:1918] T 3633e83ec7144230b08e36f6846dbc8e P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.749056 14819 log.cc:1198] T 3633e83ec7144230b08e36f6846dbc8e P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/3633e83ec7144230b08e36f6846dbc8e
I20250115 03:06:12.749243 14819 ts_tablet_manager.cc:1939] T 3633e83ec7144230b08e36f6846dbc8e P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.749961 14819 tablet_replica.cc:331] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.750061 14809 tablet_bootstrap.cc:492] T 4d6b5a30f27e47a3a515fb34691b9b15 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.750152 14809 ts_tablet_manager.cc:1397] T 4d6b5a30f27e47a3a515fb34691b9b15 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.002s
I20250115 03:06:12.750427 14823 ts_tablet_manager.cc:1905] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.750557 14809 raft_consensus.cc:357] T 4d6b5a30f27e47a3a515fb34691b9b15 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.750662 14809 raft_consensus.cc:383] T 4d6b5a30f27e47a3a515fb34691b9b15 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.750708 14809 raft_consensus.cc:738] T 4d6b5a30f27e47a3a515fb34691b9b15 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.750828 14809 consensus_queue.cc:260] T 4d6b5a30f27e47a3a515fb34691b9b15 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.751089 14809 ts_tablet_manager.cc:1428] T 4d6b5a30f27e47a3a515fb34691b9b15 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:12.751258 14809 tablet_bootstrap.cc:492] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.752151 14809 tablet_bootstrap.cc:654] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.754787 14823 ts_tablet_manager.cc:1918] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.754899 14823 log.cc:1198] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/7a78c3eca7d542d7a71f9a33e8585c18
I20250115 03:06:12.755656 14823 ts_tablet_manager.cc:1939] T 7a78c3eca7d542d7a71f9a33e8585c18 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.756579 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 3633e83ec7144230b08e36f6846dbc8e (table default.Table_dd57412817d54552a349868025274901 [id=8144a2c71b2d475597cd54b3ff2ba720]) successfully deleted
I20250115 03:06:12.750064 14819 raft_consensus.cc:2238] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.756757 14819 raft_consensus.cc:2267] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.756943 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d6b5a30f27e47a3a515fb34691b9b15 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.757316 14823 tablet_replica.cc:331] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.757359 14809 tablet_bootstrap.cc:492] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.757444 14809 ts_tablet_manager.cc:1397] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:12.757582 14823 raft_consensus.cc:2238] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:12.757719 14819 ts_tablet_manager.cc:1905] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.757828 14809 raft_consensus.cc:357] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.757928 14809 raft_consensus.cc:383] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.757966 14809 raft_consensus.cc:738] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.758081 14809 consensus_queue.cc:260] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.758251 14811 tablet_bootstrap.cc:492] T 594f755feed947afba1ba8b0853d5788 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.758328 14809 ts_tablet_manager.cc:1428] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.758498 14809 tablet_bootstrap.cc:492] T 8c21c254cdef4015bfaba3fb5d61f554 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.758368 14811 ts_tablet_manager.cc:1397] T 594f755feed947afba1ba8b0853d5788 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.001s
I20250115 03:06:12.759277 14809 tablet_bootstrap.cc:654] T 8c21c254cdef4015bfaba3fb5d61f554 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.759373 14819 ts_tablet_manager.cc:1918] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.759444 14819 log.cc:1198] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/594f755feed947afba1ba8b0853d5788
I20250115 03:06:12.759567 14811 raft_consensus.cc:357] T 594f755feed947afba1ba8b0853d5788 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:12.759629 14819 ts_tablet_manager.cc:1939] T 594f755feed947afba1ba8b0853d5788 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.759714 14811 raft_consensus.cc:383] T 594f755feed947afba1ba8b0853d5788 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.759790 14811 raft_consensus.cc:738] T 594f755feed947afba1ba8b0853d5788 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.759974 14811 consensus_queue.cc:260] T 594f755feed947afba1ba8b0853d5788 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:12.761883 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 594f755feed947afba1ba8b0853d5788 (table default.Table_dd9da62650c241fb9b1b74438090cf87 [id=a4d048fc192e46ae9dee9635a45a1229]) successfully deleted
I20250115 03:06:12.762074 14819 tablet_replica.cc:331] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.762061 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c8dc386a6b4468d920dbfab906e1366 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.762341 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.762595 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet 07d9c5e89ac34af29a406e79d2ace824 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.762871 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_92483ac55d3d40249e8b468ee1511225" } modify_external_catalogs: true
I20250115 03:06:12.762962 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_92483ac55d3d40249e8b468ee1511225" } modify_external_catalogs: true
I20250115 03:06:12.763278 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet b6a8a8da56274eaa8fef29fff38a7dae from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.763718 14811 ts_tablet_manager.cc:1428] T 594f755feed947afba1ba8b0853d5788 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:12.763831 14819 raft_consensus.cc:2238] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:12.764007 14819 raft_consensus.cc:2267] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.764453 14819 ts_tablet_manager.cc:1905] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.765547 14811 tablet_bootstrap.cc:492] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.766752 14811 tablet_bootstrap.cc:654] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.767524 14819 ts_tablet_manager.cc:1918] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.767596 14819 log.cc:1198] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/cc4b426be9da40a09b64f0e412536fa7
I20250115 03:06:12.767794 14823 pending_rounds.cc:62] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 2 pending ops.
I20250115 03:06:12.767835 14819 ts_tablet_manager.cc:1939] T cc4b426be9da40a09b64f0e412536fa7 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.767905 14823 pending_rounds.cc:69] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7114384886042869760 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "99e3ff9f0b364b2f968e0faf3cace781" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 2 new_table_name: "default.Table_0dd17013f6334efe97618f0b520512c5" dest_uuid: "5c0dae7aff4b44c699275b95ba639770" new_extra_config { } }
W20250115 03:06:12.768196 15174 consensus_queue.cc:1159] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20250115 03:06:12.768412 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet cc4b426be9da40a09b64f0e412536fa7 (table default.Table_6bc8705ef8c145478d0ccd20fcb3ce34 [id=37d965d8a5eb408ea3f33313bd22c152]) successfully deleted
I20250115 03:06:12.768599 14819 tablet_replica.cc:331] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
W20250115 03:06:12.768383 14823 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:36830: Aborted: Op aborted
I20250115 03:06:12.768754 14820 ts_tablet_manager.cc:1905] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.768748 14819 raft_consensus.cc:2238] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:12.768910 14819 raft_consensus.cc:2267] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.769423 14819 ts_tablet_manager.cc:1905] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.769593 14823 pending_rounds.cc:69] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384885211680768 op_type: NO_OP noop_request { }
I20250115 03:06:12.770740 14820 ts_tablet_manager.cc:1918] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.770816 14820 log.cc:1198] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/7a78c3eca7d542d7a71f9a33e8585c18
I20250115 03:06:12.770896 14823 raft_consensus.cc:2883] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:12.770998 14823 raft_consensus.cc:2267] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.771032 14820 ts_tablet_manager.cc:1939] T 7a78c3eca7d542d7a71f9a33e8585c18 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.771039 14819 ts_tablet_manager.cc:1918] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.771090 14819 log.cc:1198] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/0c8dc386a6b4468d920dbfab906e1366
I20250115 03:06:12.771325 14819 ts_tablet_manager.cc:1939] T 0c8dc386a6b4468d920dbfab906e1366 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.771596 14820 tablet_replica.cc:331] T 99fccdbe3a924305af7794e1ce8b092a P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.771703 14820 raft_consensus.cc:2238] T 99fccdbe3a924305af7794e1ce8b092a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.771803 14820 raft_consensus.cc:2267] T 99fccdbe3a924305af7794e1ce8b092a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.771853 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 0c8dc386a6b4468d920dbfab906e1366 (table default.Table_fefc309a5c684cc7a466bc67cba0f109 [id=6d1418f7a3b64aebace43d4478f10915]) successfully deleted
I20250115 03:06:12.771836 14823 ts_tablet_manager.cc:1905] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.772133 14820 ts_tablet_manager.cc:1905] T 99fccdbe3a924305af7794e1ce8b092a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:12.772243 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35 already in progress: creating tablet
I20250115 03:06:12.772426 14819 tablet_replica.cc:331] T 4d6b5a30f27e47a3a515fb34691b9b15 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.772526 14819 raft_consensus.cc:2238] T 4d6b5a30f27e47a3a515fb34691b9b15 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.772607 14819 raft_consensus.cc:2267] T 4d6b5a30f27e47a3a515fb34691b9b15 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.772982 14819 ts_tablet_manager.cc:1905] T 4d6b5a30f27e47a3a515fb34691b9b15 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.773828 14820 ts_tablet_manager.cc:1918] T 99fccdbe3a924305af7794e1ce8b092a P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.773900 14820 log.cc:1198] T 99fccdbe3a924305af7794e1ce8b092a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/99fccdbe3a924305af7794e1ce8b092a
I20250115 03:06:12.774077 14820 ts_tablet_manager.cc:1939] T 99fccdbe3a924305af7794e1ce8b092a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.774726 14820 tablet_replica.cc:331] T fa7043f411374111b4c81467260fc6cd P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.774843 14820 raft_consensus.cc:2238] T fa7043f411374111b4c81467260fc6cd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.774940 14820 raft_consensus.cc:2267] T fa7043f411374111b4c81467260fc6cd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.774981 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 7a78c3eca7d542d7a71f9a33e8585c18 (table default.Table_66cd278959e849dfaf7c9fc9e604d46a [id=6ecde174f5dc422387c42ec4e2f3a053]) successfully deleted
I20250115 03:06:12.775094 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 99fccdbe3a924305af7794e1ce8b092a (table default.Table_94ee8224b6cd468bb2a1c15b2c2af35a [id=63bd4ca58023436eb2b453f6fa0ea0c4]) successfully deleted
I20250115 03:06:12.775326 14819 ts_tablet_manager.cc:1918] T 4d6b5a30f27e47a3a515fb34691b9b15 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.775389 14819 log.cc:1198] T 4d6b5a30f27e47a3a515fb34691b9b15 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/4d6b5a30f27e47a3a515fb34691b9b15
I20250115 03:06:12.775499 14820 ts_tablet_manager.cc:1905] T fa7043f411374111b4c81467260fc6cd P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.775570 14819 ts_tablet_manager.cc:1939] T 4d6b5a30f27e47a3a515fb34691b9b15 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.776063 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 4d6b5a30f27e47a3a515fb34691b9b15 (table default.Table_fadcbb64abcd4a1b812defb8ed9d99f5 [id=e3077355979a401a89f0558104eb53cf]) successfully deleted
I20250115 03:06:12.777240 14820 ts_tablet_manager.cc:1918] T fa7043f411374111b4c81467260fc6cd P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.777352 14820 log.cc:1198] T fa7043f411374111b4c81467260fc6cd P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/fa7043f411374111b4c81467260fc6cd
I20250115 03:06:12.777577 14820 ts_tablet_manager.cc:1939] T fa7043f411374111b4c81467260fc6cd P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.777920 14812 tablet_bootstrap.cc:492] T 0ecd4741aacb45e2ae6404675f885125 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.778008 14812 ts_tablet_manager.cc:1397] T 0ecd4741aacb45e2ae6404675f885125 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.061s user 0.002s sys 0.000s
I20250115 03:06:12.778198 14820 tablet_replica.cc:331] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.778329 14820 raft_consensus.cc:2238] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.778453 14812 raft_consensus.cc:357] T 0ecd4741aacb45e2ae6404675f885125 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.778558 14812 raft_consensus.cc:383] T 0ecd4741aacb45e2ae6404675f885125 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.778596 14812 raft_consensus.cc:738] T 0ecd4741aacb45e2ae6404675f885125 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.778712 14812 consensus_queue.cc:260] T 0ecd4741aacb45e2ae6404675f885125 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.778858 14820 raft_consensus.cc:2267] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.778980 14812 ts_tablet_manager.cc:1428] T 0ecd4741aacb45e2ae6404675f885125 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.779163 14812 tablet_bootstrap.cc:492] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.780068 14812 tablet_bootstrap.cc:654] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.780786 14820 ts_tablet_manager.cc:1905] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.782019 14809 tablet_bootstrap.cc:492] T 8c21c254cdef4015bfaba3fb5d61f554 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.782095 14809 ts_tablet_manager.cc:1397] T 8c21c254cdef4015bfaba3fb5d61f554 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.000s
I20250115 03:06:12.782480 14809 raft_consensus.cc:357] T 8c21c254cdef4015bfaba3fb5d61f554 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.782582 14809 raft_consensus.cc:383] T 8c21c254cdef4015bfaba3fb5d61f554 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.782622 14809 raft_consensus.cc:738] T 8c21c254cdef4015bfaba3fb5d61f554 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.782768 14823 ts_tablet_manager.cc:1918] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.782729 14809 consensus_queue.cc:260] T 8c21c254cdef4015bfaba3fb5d61f554 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.782975 14809 ts_tablet_manager.cc:1428] T 8c21c254cdef4015bfaba3fb5d61f554 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.783156 14809 tablet_bootstrap.cc:492] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.783633 14823 log.cc:1198] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/99e3ff9f0b364b2f968e0faf3cace781
I20250115 03:06:12.784027 14809 tablet_bootstrap.cc:654] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.784313 14823 ts_tablet_manager.cc:1939] T 99e3ff9f0b364b2f968e0faf3cace781 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.787202 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 99e3ff9f0b364b2f968e0faf3cace781 (table default.Table_0dd17013f6334efe97618f0b520512c5 [id=488a9a6268f64669ae17e1a67caa73a5]) successfully deleted
I20250115 03:06:12.788182 14811 tablet_bootstrap.cc:492] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.788309 14811 ts_tablet_manager.cc:1397] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s
I20250115 03:06:12.788764 14811 raft_consensus.cc:357] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.788918 14811 raft_consensus.cc:383] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.789028 14811 raft_consensus.cc:738] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.789264 14811 consensus_queue.cc:260] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.791401 14811 ts_tablet_manager.cc:1428] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:12.793031 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet fa7043f411374111b4c81467260fc6cd (table default.Table_c1020680eef344a49998eddac921fea5 [id=649880f1ff0d4cfda2358b408110d8dc]) successfully deleted
I20250115 03:06:12.793247 14811 tablet_bootstrap.cc:492] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.793751 14812 tablet_bootstrap.cc:492] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.793846 14812 ts_tablet_manager.cc:1397] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20250115 03:06:12.794322 14811 tablet_bootstrap.cc:654] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.794576 14812 raft_consensus.cc:357] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.794682 14812 raft_consensus.cc:383] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.794726 14812 raft_consensus.cc:738] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.794857 14812 consensus_queue.cc:260] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.795138 14812 ts_tablet_manager.cc:1428] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.795317 14812 tablet_bootstrap.cc:492] T 4d6b5a30f27e47a3a515fb34691b9b15 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.796176 14820 ts_tablet_manager.cc:1918] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.796223 14812 tablet_bootstrap.cc:654] T 4d6b5a30f27e47a3a515fb34691b9b15 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.795322 14823 tablet_replica.cc:331] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.796453 14823 raft_consensus.cc:2238] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.796584 14823 raft_consensus.cc:2267] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.796751 14820 log.cc:1198] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/cc4b426be9da40a09b64f0e412536fa7
I20250115 03:06:12.797601 14809 tablet_bootstrap.cc:492] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.797679 14809 ts_tablet_manager.cc:1397] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20250115 03:06:12.798089 14820 ts_tablet_manager.cc:1939] T cc4b426be9da40a09b64f0e412536fa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.798132 14809 raft_consensus.cc:357] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.798236 14809 raft_consensus.cc:383] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.798280 14809 raft_consensus.cc:738] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.798391 14809 consensus_queue.cc:260] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.798638 14809 ts_tablet_manager.cc:1428] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.798810 14809 tablet_bootstrap.cc:492] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.799660 14809 tablet_bootstrap.cc:654] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.801719 14820 tablet_replica.cc:331] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.802332 14820 raft_consensus.cc:2238] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.802479 14820 raft_consensus.cc:2267] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.804241 14820 ts_tablet_manager.cc:1905] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.806249 14820 ts_tablet_manager.cc:1918] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.806319 14820 log.cc:1198] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/0c8dc386a6b4468d920dbfab906e1366
I20250115 03:06:12.806670 14820 ts_tablet_manager.cc:1939] T 0c8dc386a6b4468d920dbfab906e1366 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.808679 14811 tablet_bootstrap.cc:492] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.808770 14811 ts_tablet_manager.cc:1397] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20250115 03:06:12.808900 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 9ffc6d1d09e04cc99a5e7a1e1af7bef8 (DEFAULT_TABLE table=default.Table_040bb2a68caa47cd995d0f4936302d43 [id=8d5d65560792413ea180b35bb9319f04]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.809144 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ffc6d1d09e04cc99a5e7a1e1af7bef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.809185 14811 raft_consensus.cc:357] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.809263 14811 raft_consensus.cc:383] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.809289 14811 raft_consensus.cc:738] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.809393 14811 consensus_queue.cc:260] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.809617 14811 ts_tablet_manager.cc:1428] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.809832 14811 tablet_bootstrap.cc:492] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.810806 14811 tablet_bootstrap.cc:654] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.811792 14812 tablet_bootstrap.cc:492] T 4d6b5a30f27e47a3a515fb34691b9b15 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.811865 14812 ts_tablet_manager.cc:1397] T 4d6b5a30f27e47a3a515fb34691b9b15 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20250115 03:06:12.812197 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet cc4b426be9da40a09b64f0e412536fa7 (table default.Table_6bc8705ef8c145478d0ccd20fcb3ce34 [id=37d965d8a5eb408ea3f33313bd22c152]) successfully deleted
I20250115 03:06:12.812302 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 0c8dc386a6b4468d920dbfab906e1366 (table default.Table_fefc309a5c684cc7a466bc67cba0f109 [id=6d1418f7a3b64aebace43d4478f10915]) successfully deleted
I20250115 03:06:12.812269 14812 raft_consensus.cc:357] T 4d6b5a30f27e47a3a515fb34691b9b15 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.812366 14812 raft_consensus.cc:383] T 4d6b5a30f27e47a3a515fb34691b9b15 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.812397 14812 raft_consensus.cc:738] T 4d6b5a30f27e47a3a515fb34691b9b15 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
W20250115 03:06:12.812407 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35 already in progress: creating tablet
I20250115 03:06:12.812503 14812 consensus_queue.cc:260] T 4d6b5a30f27e47a3a515fb34691b9b15 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.812770 14812 ts_tablet_manager.cc:1428] T 4d6b5a30f27e47a3a515fb34691b9b15 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.812953 14812 tablet_bootstrap.cc:492] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.813112 14165 master_service.cc:946] ReplaceTablet: received request to replace tablet aa88f604011741f2b3eb2699b9328eae from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.813822 14812 tablet_bootstrap.cc:654] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.815303 14661 tablet_service.cc:1467] Processing CreateTablet for tablet 0e78c18fd98f44bcbd08b747f2a0ffd0 (DEFAULT_TABLE table=default.Table_669098605d764723aba5170b1c1f12a1 [id=9c5da4ce43f3446e824ea626b0509209]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.815519 14661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e78c18fd98f44bcbd08b747f2a0ffd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.815659 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 044651a15c8346a6a0d11713a82a3641 (DEFAULT_TABLE table=default.Table_43170518d7504d1d8d17374e281f2728 [id=540df72d30804bdf8595bc7df61e1f46]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.815840 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 044651a15c8346a6a0d11713a82a3641. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.816509 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 0e78c18fd98f44bcbd08b747f2a0ffd0 (DEFAULT_TABLE table=default.Table_669098605d764723aba5170b1c1f12a1 [id=9c5da4ce43f3446e824ea626b0509209]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.816716 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e78c18fd98f44bcbd08b747f2a0ffd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.817031 14809 tablet_bootstrap.cc:492] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.817096 14809 ts_tablet_manager.cc:1397] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20250115 03:06:12.817103 14671 tablet_service.cc:1467] Processing CreateTablet for tablet f4a0952cb8d1412081f3852ce9f60843 (DEFAULT_TABLE table=default.Table_8a1be82e3d5648a9b4b5399d2f56e72f [id=72bddfc9455d4d60bd138b4218f77013]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.817296 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4a0952cb8d1412081f3852ce9f60843. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.817515 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 551d5ffdcd2d43fcb63e42c135fc2bd5 (DEFAULT_TABLE table=default.Table_3e807f869ab742ddaf0412ef84007cfe [id=1b7d3a80c5424e5692c2e6c556919342]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.817615 14809 raft_consensus.cc:357] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.817692 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 551d5ffdcd2d43fcb63e42c135fc2bd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.817711 14809 raft_consensus.cc:383] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.817744 14809 raft_consensus.cc:738] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.817842 14809 consensus_queue.cc:260] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.818068 14809 ts_tablet_manager.cc:1428] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:12.818223 14809 tablet_bootstrap.cc:492] T 7479bfc96d634011ba15069b5cf6794c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.818657 14395 tablet_service.cc:1467] Processing CreateTablet for tablet 551d5ffdcd2d43fcb63e42c135fc2bd5 (DEFAULT_TABLE table=default.Table_3e807f869ab742ddaf0412ef84007cfe [id=1b7d3a80c5424e5692c2e6c556919342]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.818852 14395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 551d5ffdcd2d43fcb63e42c135fc2bd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.819061 14809 tablet_bootstrap.cc:654] T 7479bfc96d634011ba15069b5cf6794c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.819099 14666 tablet_service.cc:1467] Processing CreateTablet for tablet f21761ff87ac43d5b9b47b5905fa5867 (DEFAULT_TABLE table=default.Table_6bc8705ef8c145478d0ccd20fcb3ce34 [id=37d965d8a5eb408ea3f33313bd22c152]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.819288 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f21761ff87ac43d5b9b47b5905fa5867. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.819425 14664 tablet_service.cc:1467] Processing CreateTablet for tablet bdc2e882dc7e469e857d74e41ee2f435 (DEFAULT_TABLE table=default.Table_e9f7a3afca894ff18b614971924bbd50 [id=29daeed94f414e5abdb17e05a26b7516]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.819600 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdc2e882dc7e469e857d74e41ee2f435. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.820689 14398 tablet_service.cc:1467] Processing CreateTablet for tablet bdc2e882dc7e469e857d74e41ee2f435 (DEFAULT_TABLE table=default.Table_e9f7a3afca894ff18b614971924bbd50 [id=29daeed94f414e5abdb17e05a26b7516]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.820878 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdc2e882dc7e469e857d74e41ee2f435. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.821020 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 7b13f8b40c2f40b8a93ce8030845665c (DEFAULT_TABLE table=default.Table_92483ac55d3d40249e8b468ee1511225 [id=bd1ddc98267f4afe9bcd4dd0c0951ff7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.821204 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b13f8b40c2f40b8a93ce8030845665c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.821281 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 9aa9dbbbb8c548a9b3943f0305cb6e9f (DEFAULT_TABLE table=default.Table_937e6e7e4b014731b5945f00775aa14c [id=93321e5844114a63bb53decd1f836d98]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.821465 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9aa9dbbbb8c548a9b3943f0305cb6e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.822737 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 9aa9dbbbb8c548a9b3943f0305cb6e9f (DEFAULT_TABLE table=default.Table_937e6e7e4b014731b5945f00775aa14c [id=93321e5844114a63bb53decd1f836d98]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.822937 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9aa9dbbbb8c548a9b3943f0305cb6e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.815495 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 186b3bf9f6e14c9cb435a1094564f66a (DEFAULT_TABLE table=default.Table_e83118015644482ca56543e76b9691bd [id=089b3b2c658a4702869cf70ddd9baf73]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.823216 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 186b3bf9f6e14c9cb435a1094564f66a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.823993 14823 ts_tablet_manager.cc:1905] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.824748 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 186b3bf9f6e14c9cb435a1094564f66a (DEFAULT_TABLE table=default.Table_e83118015644482ca56543e76b9691bd [id=089b3b2c658a4702869cf70ddd9baf73]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.824971 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 186b3bf9f6e14c9cb435a1094564f66a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.826725 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 044651a15c8346a6a0d11713a82a3641 (DEFAULT_TABLE table=default.Table_43170518d7504d1d8d17374e281f2728 [id=540df72d30804bdf8595bc7df61e1f46]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.826933 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 044651a15c8346a6a0d11713a82a3641. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.827997 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 28324608e3764b7e8a550bc71a4bad58 (DEFAULT_TABLE table=default.Table_f5eb08f6d7974c8895385279b9b47805 [id=39c58ec19ba2491b93975df1e0690559]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.828637 14399 tablet_service.cc:1467] Processing CreateTablet for tablet f4a0952cb8d1412081f3852ce9f60843 (DEFAULT_TABLE table=default.Table_8a1be82e3d5648a9b4b5399d2f56e72f [id=72bddfc9455d4d60bd138b4218f77013]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.828819 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4a0952cb8d1412081f3852ce9f60843. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.829644 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 9ffc6d1d09e04cc99a5e7a1e1af7bef8 (DEFAULT_TABLE table=default.Table_040bb2a68caa47cd995d0f4936302d43 [id=8d5d65560792413ea180b35bb9319f04]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.830595 14401 tablet_service.cc:1467] Processing CreateTablet for tablet f21761ff87ac43d5b9b47b5905fa5867 (DEFAULT_TABLE table=default.Table_6bc8705ef8c145478d0ccd20fcb3ce34 [id=37d965d8a5eb408ea3f33313bd22c152]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.830781 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f21761ff87ac43d5b9b47b5905fa5867. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.830996 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28324608e3764b7e8a550bc71a4bad58. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.831008 14823 ts_tablet_manager.cc:1918] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.831616 14823 log.cc:1198] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/3a786573d49e45669a3a2ef2385afc19
I20250115 03:06:12.831849 14823 ts_tablet_manager.cc:1939] T 3a786573d49e45669a3a2ef2385afc19 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.832294 14823 tablet_replica.cc:331] T 3633e83ec7144230b08e36f6846dbc8e P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.832394 14823 raft_consensus.cc:2238] T 3633e83ec7144230b08e36f6846dbc8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.832473 14823 raft_consensus.cc:2267] T 3633e83ec7144230b08e36f6846dbc8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.832723 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ffc6d1d09e04cc99a5e7a1e1af7bef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.816509 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 7b13f8b40c2f40b8a93ce8030845665c (DEFAULT_TABLE table=default.Table_92483ac55d3d40249e8b468ee1511225 [id=bd1ddc98267f4afe9bcd4dd0c0951ff7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.833593 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b13f8b40c2f40b8a93ce8030845665c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.835192 14817 raft_consensus.cc:491] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.835268 14817 raft_consensus.cc:513] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.835577 14817 leader_election.cc:290] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.835958 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecd4741aacb45e2ae6404675f885125" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:12.836057 14689 raft_consensus.cc:2463] T 0ecd4741aacb45e2ae6404675f885125 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:12.836302 14493 leader_election.cc:304] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:12.837286 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecd4741aacb45e2ae6404675f885125" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:12.837399 14422 raft_consensus.cc:2463] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:12.838327 14531 tablet_service.cc:1467] Processing CreateTablet for tablet 551d5ffdcd2d43fcb63e42c135fc2bd5 (DEFAULT_TABLE table=default.Table_3e807f869ab742ddaf0412ef84007cfe [id=1b7d3a80c5424e5692c2e6c556919342]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.838503 14528 tablet_service.cc:1467] Processing CreateTablet for tablet f21761ff87ac43d5b9b47b5905fa5867 (DEFAULT_TABLE table=default.Table_6bc8705ef8c145478d0ccd20fcb3ce34 [id=37d965d8a5eb408ea3f33313bd22c152]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.838582 14535 tablet_service.cc:1467] Processing CreateTablet for tablet bdc2e882dc7e469e857d74e41ee2f435 (DEFAULT_TABLE table=default.Table_e9f7a3afca894ff18b614971924bbd50 [id=29daeed94f414e5abdb17e05a26b7516]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.838634 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 28324608e3764b7e8a550bc71a4bad58 (DEFAULT_TABLE table=default.Table_f5eb08f6d7974c8895385279b9b47805 [id=39c58ec19ba2491b93975df1e0690559]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.838328 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 186b3bf9f6e14c9cb435a1094564f66a (DEFAULT_TABLE table=default.Table_e83118015644482ca56543e76b9691bd [id=089b3b2c658a4702869cf70ddd9baf73]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.838861 14532 tablet_service.cc:1467] Processing CreateTablet for tablet 0e78c18fd98f44bcbd08b747f2a0ffd0 (DEFAULT_TABLE table=default.Table_669098605d764723aba5170b1c1f12a1 [id=9c5da4ce43f3446e824ea626b0509209]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.838999 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 7b13f8b40c2f40b8a93ce8030845665c (DEFAULT_TABLE table=default.Table_92483ac55d3d40249e8b468ee1511225 [id=bd1ddc98267f4afe9bcd4dd0c0951ff7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.839098 14823 ts_tablet_manager.cc:1905] T 3633e83ec7144230b08e36f6846dbc8e P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.839154 14526 tablet_service.cc:1514] Processing DeleteTablet for tablet 594f755feed947afba1ba8b0853d5788 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.838588 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 9aa9dbbbb8c548a9b3943f0305cb6e9f (DEFAULT_TABLE table=default.Table_937e6e7e4b014731b5945f00775aa14c [id=93321e5844114a63bb53decd1f836d98]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.838738 14529 tablet_service.cc:1467] Processing CreateTablet for tablet f4a0952cb8d1412081f3852ce9f60843 (DEFAULT_TABLE table=default.Table_8a1be82e3d5648a9b4b5399d2f56e72f [id=72bddfc9455d4d60bd138b4218f77013]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.839506 14534 tablet_service.cc:1467] Processing CreateTablet for tablet 044651a15c8346a6a0d11713a82a3641 (DEFAULT_TABLE table=default.Table_43170518d7504d1d8d17374e281f2728 [id=540df72d30804bdf8595bc7df61e1f46]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.839615 14531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 551d5ffdcd2d43fcb63e42c135fc2bd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.838510 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 9ffc6d1d09e04cc99a5e7a1e1af7bef8 (DEFAULT_TABLE table=default.Table_040bb2a68caa47cd995d0f4936302d43 [id=8d5d65560792413ea180b35bb9319f04]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.840684 14823 ts_tablet_manager.cc:1918] T 3633e83ec7144230b08e36f6846dbc8e P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.840749 14823 log.cc:1198] T 3633e83ec7144230b08e36f6846dbc8e P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/3633e83ec7144230b08e36f6846dbc8e
I20250115 03:06:12.840735 14817 raft_consensus.cc:2798] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:12.840788 14817 raft_consensus.cc:491] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:12.840826 14817 raft_consensus.cc:3054] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.840947 14823 ts_tablet_manager.cc:1939] T 3633e83ec7144230b08e36f6846dbc8e P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.841670 14817 raft_consensus.cc:513] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.841953 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 044651a15c8346a6a0d11713a82a3641. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.841980 14817 leader_election.cc:290] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.842379 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecd4741aacb45e2ae6404675f885125" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:12.842479 14689 raft_consensus.cc:3054] T 0ecd4741aacb45e2ae6404675f885125 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.843225 14689 raft_consensus.cc:2463] T 0ecd4741aacb45e2ae6404675f885125 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:12.843559 14493 leader_election.cc:304] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:12.843792 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ffc6d1d09e04cc99a5e7a1e1af7bef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.843940 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ecd4741aacb45e2ae6404675f885125" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:12.844029 14422 raft_consensus.cc:3054] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.844771 14422 raft_consensus.cc:2463] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:12.845093 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet b6a8a8da56274eaa8fef29fff38a7dae
I20250115 03:06:12.845355 14164 catalog_manager.cc:6596] ReplaceTablet: tablet b6a8a8da56274eaa8fef29fff38a7dae deleted and replaced by tablet 1ef14443e7fd464a8ab82eb33819dac3
I20250115 03:06:12.845988 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 07d9c5e89ac34af29a406e79d2ace824
I20250115 03:06:12.846228 14168 catalog_manager.cc:6596] ReplaceTablet: tablet 07d9c5e89ac34af29a406e79d2ace824 deleted and replaced by tablet de2fcf7797684ef78e2c4f40b7298afd
I20250115 03:06:12.848012 14823 tablet_replica.cc:331] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.848141 14823 raft_consensus.cc:2238] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.848248 14823 raft_consensus.cc:2267] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.848466 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d9c5e89ac34af29a406e79d2ace824 with delete_type TABLET_DATA_DELETED (replaced by tablet de2fcf7797684ef78e2c4f40b7298afd) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.848656 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet b6a8a8da56274eaa8fef29fff38a7dae with delete_type TABLET_DATA_DELETED (replaced by tablet 1ef14443e7fd464a8ab82eb33819dac3) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.848760 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet b6a8a8da56274eaa8fef29fff38a7dae with delete_type TABLET_DATA_DELETED (replaced by tablet 1ef14443e7fd464a8ab82eb33819dac3) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.849066 14817 raft_consensus.cc:2798] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.849146 14817 raft_consensus.cc:695] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:12.849191 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28324608e3764b7e8a550bc71a4bad58. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.849256 14817 consensus_queue.cc:237] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.851034 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdc2e882dc7e469e857d74e41ee2f435. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.851835 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d9c5e89ac34af29a406e79d2ace824 with delete_type TABLET_DATA_DELETED (replaced by tablet de2fcf7797684ef78e2c4f40b7298afd) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.852077 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 28324608e3764b7e8a550bc71a4bad58 (DEFAULT_TABLE table=default.Table_f5eb08f6d7974c8895385279b9b47805 [id=39c58ec19ba2491b93975df1e0690559]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.852262 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28324608e3764b7e8a550bc71a4bad58. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.853019 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9aa9dbbbb8c548a9b3943f0305cb6e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.854801 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4a0952cb8d1412081f3852ce9f60843. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:12.855123 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet b6a8a8da56274eaa8fef29fff38a7dae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6a8a8da56274eaa8fef29fff38a7dae already in progress: creating tablet
W20250115 03:06:12.855243 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 07d9c5e89ac34af29a406e79d2ace824 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07d9c5e89ac34af29a406e79d2ace824 already in progress: creating tablet
I20250115 03:06:12.855401 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 3633e83ec7144230b08e36f6846dbc8e (table default.Table_dd57412817d54552a349868025274901 [id=8144a2c71b2d475597cd54b3ff2ba720]) successfully deleted
I20250115 03:06:12.856626 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e78c18fd98f44bcbd08b747f2a0ffd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.858397 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 186b3bf9f6e14c9cb435a1094564f66a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.860245 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f21761ff87ac43d5b9b47b5905fa5867. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.860769 14823 ts_tablet_manager.cc:1905] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.862056 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b13f8b40c2f40b8a93ce8030845665c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.862599 14823 ts_tablet_manager.cc:1918] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.862653 14823 log.cc:1198] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/cc4b426be9da40a09b64f0e412536fa7
I20250115 03:06:12.862855 14823 ts_tablet_manager.cc:1939] T cc4b426be9da40a09b64f0e412536fa7 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.863365 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet cc4b426be9da40a09b64f0e412536fa7 (table default.Table_6bc8705ef8c145478d0ccd20fcb3ce34 [id=37d965d8a5eb408ea3f33313bd22c152]) successfully deleted
I20250115 03:06:12.863922 14812 tablet_bootstrap.cc:492] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.863987 14812 ts_tablet_manager.cc:1397] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.051s user 0.001s sys 0.000s
I20250115 03:06:12.864459 14812 raft_consensus.cc:357] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.864550 14812 raft_consensus.cc:383] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.864578 14812 raft_consensus.cc:738] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.864672 14812 consensus_queue.cc:260] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.864903 14812 ts_tablet_manager.cc:1428] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.865085 14812 tablet_bootstrap.cc:492] T 8c21c254cdef4015bfaba3fb5d61f554 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.865965 14812 tablet_bootstrap.cc:654] T 8c21c254cdef4015bfaba3fb5d61f554 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.867837 14811 tablet_bootstrap.cc:492] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.867920 14811 ts_tablet_manager.cc:1397] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.058s user 0.002s sys 0.000s
I20250115 03:06:12.868316 14811 raft_consensus.cc:357] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.868402 14811 raft_consensus.cc:383] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.868425 14811 raft_consensus.cc:738] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.868528 14811 consensus_queue.cc:260] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.868752 14811 ts_tablet_manager.cc:1428] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.868901 14811 tablet_bootstrap.cc:492] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.869748 14811 tablet_bootstrap.cc:654] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.870360 14820 tablet_replica.cc:331] T 594f755feed947afba1ba8b0853d5788 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.870528 14820 raft_consensus.cc:2238] T 594f755feed947afba1ba8b0853d5788 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.870666 14820 raft_consensus.cc:2267] T 594f755feed947afba1ba8b0853d5788 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.871124 14820 ts_tablet_manager.cc:1905] T 594f755feed947afba1ba8b0853d5788 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:12.873333 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35 already in progress: creating tablet
I20250115 03:06:12.873528 14823 tablet_replica.cc:331] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.873639 14823 raft_consensus.cc:2238] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.873736 14823 raft_consensus.cc:2267] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.874321 14823 ts_tablet_manager.cc:1905] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.874338 14809 tablet_bootstrap.cc:492] T 7479bfc96d634011ba15069b5cf6794c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.874415 14809 ts_tablet_manager.cc:1397] T 7479bfc96d634011ba15069b5cf6794c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.056s user 0.001s sys 0.000s
I20250115 03:06:12.874847 14809 raft_consensus.cc:357] T 7479bfc96d634011ba15069b5cf6794c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.874958 14809 raft_consensus.cc:383] T 7479bfc96d634011ba15069b5cf6794c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.874997 14809 raft_consensus.cc:738] T 7479bfc96d634011ba15069b5cf6794c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.875115 14809 consensus_queue.cc:260] T 7479bfc96d634011ba15069b5cf6794c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.875303 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d6b5a30f27e47a3a515fb34691b9b15 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.875412 14809 ts_tablet_manager.cc:1428] T 7479bfc96d634011ba15069b5cf6794c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.875599 14809 tablet_bootstrap.cc:492] T 876e28105fbb47bdb722e5d3f8467341 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.876507 14809 tablet_bootstrap.cc:654] T 876e28105fbb47bdb722e5d3f8467341 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.880125 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet aa88f604011741f2b3eb2699b9328eae
I20250115 03:06:12.880394 14165 catalog_manager.cc:6596] ReplaceTablet: tablet aa88f604011741f2b3eb2699b9328eae deleted and replaced by tablet b561abc942014fb79f22fd6a9fec38d0
I20250115 03:06:12.880782 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 7b13f8b40c2f40b8a93ce8030845665c
I20250115 03:06:12.880859 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet aa88f604011741f2b3eb2699b9328eae with delete_type TABLET_DATA_DELETED (replaced by tablet b561abc942014fb79f22fd6a9fec38d0) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.881047 14819 tablet_replica.cc:331] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.881145 14819 raft_consensus.cc:2238] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:12.881286 14819 raft_consensus.cc:2267] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.881733 14819 ts_tablet_manager.cc:1905] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.883565 14819 ts_tablet_manager.cc:1918] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.883627 14819 log.cc:1198] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/aa88f604011741f2b3eb2699b9328eae
I20250115 03:06:12.883849 14819 ts_tablet_manager.cc:1939] T aa88f604011741f2b3eb2699b9328eae P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.884913 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet aa88f604011741f2b3eb2699b9328eae (table default.Table_fa83f65f02b04ad491cee3b2a617927f [id=46f096547b3a4de9a46522b978ac4b8c]) successfully deleted
I20250115 03:06:12.885113 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b13f8b40c2f40b8a93ce8030845665c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.885308 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b13f8b40c2f40b8a93ce8030845665c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.885579 14812 tablet_bootstrap.cc:492] T 8c21c254cdef4015bfaba3fb5d61f554 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.885649 14812 ts_tablet_manager.cc:1397] T 8c21c254cdef4015bfaba3fb5d61f554 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.000s
I20250115 03:06:12.886047 14812 raft_consensus.cc:357] T 8c21c254cdef4015bfaba3fb5d61f554 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.886139 14812 raft_consensus.cc:383] T 8c21c254cdef4015bfaba3fb5d61f554 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.886170 14812 raft_consensus.cc:738] T 8c21c254cdef4015bfaba3fb5d61f554 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.886246 14823 ts_tablet_manager.cc:1918] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.886296 14823 log.cc:1198] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/0c8dc386a6b4468d920dbfab906e1366
I20250115 03:06:12.886274 14812 consensus_queue.cc:260] T 8c21c254cdef4015bfaba3fb5d61f554 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.886507 14823 ts_tablet_manager.cc:1939] T 0c8dc386a6b4468d920dbfab906e1366 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.886533 14812 ts_tablet_manager.cc:1428] T 8c21c254cdef4015bfaba3fb5d61f554 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.886700 14812 tablet_bootstrap.cc:492] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.886966 14823 tablet_replica.cc:331] T 594f755feed947afba1ba8b0853d5788 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:12.887064 14823 raft_consensus.cc:2238] T 594f755feed947afba1ba8b0853d5788 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.887145 14823 raft_consensus.cc:2267] T 594f755feed947afba1ba8b0853d5788 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.887576 14812 tablet_bootstrap.cc:654] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.888089 15134 mvcc.cc:204] Tried to move back new op lower bound from 7114384885388513280 to 7114384880797831168. Current Snapshot: MvccSnapshot[applied={T|T < 7114384885388513280 or (T in {7114384885388513280})}]
I20250115 03:06:12.889082 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 0c8dc386a6b4468d920dbfab906e1366 (table default.Table_fefc309a5c684cc7a466bc67cba0f109 [id=6d1418f7a3b64aebace43d4478f10915]) successfully deleted
I20250115 03:06:12.889726 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet aa88f604011741f2b3eb2699b9328eae with delete_type TABLET_DATA_DELETED (replaced by tablet b561abc942014fb79f22fd6a9fec38d0) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.877462 14820 ts_tablet_manager.cc:1918] T 594f755feed947afba1ba8b0853d5788 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20250115 03:06:12.890532 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 7b13f8b40c2f40b8a93ce8030845665c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7b13f8b40c2f40b8a93ce8030845665c already in progress: creating tablet
I20250115 03:06:12.891377 14820 log.cc:1198] T 594f755feed947afba1ba8b0853d5788 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/594f755feed947afba1ba8b0853d5788
I20250115 03:06:12.891618 14820 ts_tablet_manager.cc:1939] T 594f755feed947afba1ba8b0853d5788 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.891790 14812 tablet_bootstrap.cc:492] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.891853 14812 ts_tablet_manager.cc:1397] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:12.892223 14812 raft_consensus.cc:357] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.892302 14812 raft_consensus.cc:383] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.892328 14812 raft_consensus.cc:738] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.892423 14812 consensus_queue.cc:260] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.892638 14812 ts_tablet_manager.cc:1428] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.892778 14812 tablet_bootstrap.cc:492] T 7479bfc96d634011ba15069b5cf6794c P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.894758 14812 tablet_bootstrap.cc:654] T 7479bfc96d634011ba15069b5cf6794c P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.895563 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet b6a8a8da56274eaa8fef29fff38a7dae with delete_type TABLET_DATA_DELETED (replaced by tablet 1ef14443e7fd464a8ab82eb33819dac3) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.895648 14812 tablet_bootstrap.cc:492] T 7479bfc96d634011ba15069b5cf6794c P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.895709 14812 ts_tablet_manager.cc:1397] T 7479bfc96d634011ba15069b5cf6794c P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:12.896062 14812 raft_consensus.cc:357] T 7479bfc96d634011ba15069b5cf6794c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.896138 14812 raft_consensus.cc:383] T 7479bfc96d634011ba15069b5cf6794c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.896163 14812 raft_consensus.cc:738] T 7479bfc96d634011ba15069b5cf6794c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.896255 14812 consensus_queue.cc:260] T 7479bfc96d634011ba15069b5cf6794c P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.896461 14812 ts_tablet_manager.cc:1428] T 7479bfc96d634011ba15069b5cf6794c P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.896595 14812 tablet_bootstrap.cc:492] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.896919 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d9c5e89ac34af29a406e79d2ace824 with delete_type TABLET_DATA_DELETED (replaced by tablet de2fcf7797684ef78e2c4f40b7298afd) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.898816 14812 tablet_bootstrap.cc:654] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.899555 14168 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 338592cc70ad4e61ba90905a424a3871 (table default.Table_ee5faa8cd90f40199372322d90c5a204 [id=ff4681cab8ce44ceb45ed76c70c1132e]). Expected version 1 got 0
W20250115 03:06:12.899843 14168 catalog_manager.cc:5718] Failed to send AlterTable RPC for tablet 338592cc70ad4e61ba90905a424a3871 (table default.Table_ee5faa8cd90f40199372322d90c5a204 [id=ff4681cab8ce44ceb45ed76c70c1132e], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 338592cc70ad4e61ba90905a424a3871 consensus state has no leader
I20250115 03:06:12.900954 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 1ef14443e7fd464a8ab82eb33819dac3 (DEFAULT_TABLE table=default.Table_f590b49e96644f96934046c984688857 [id=fd7ff6d50c944f3a80089030d9fc895d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.901707 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ef14443e7fd464a8ab82eb33819dac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.904031 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 1ef14443e7fd464a8ab82eb33819dac3 (DEFAULT_TABLE table=default.Table_f590b49e96644f96934046c984688857 [id=fd7ff6d50c944f3a80089030d9fc895d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.904309 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ef14443e7fd464a8ab82eb33819dac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.905274 14861 raft_consensus.cc:491] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.905301 14811 tablet_bootstrap.cc:492] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.905381 14811 ts_tablet_manager.cc:1397] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.037s user 0.001s sys 0.000s
I20250115 03:06:12.905359 14861 raft_consensus.cc:513] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.905682 14861 leader_election.cc:290] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.905803 14811 raft_consensus.cc:357] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.905905 14811 raft_consensus.cc:383] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.905936 14811 raft_consensus.cc:738] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.906035 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e379cb7a61a8415994a7419266346d8e" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:12.906042 14811 consensus_queue.cc:260] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.906456 14811 ts_tablet_manager.cc:1428] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.906617 14811 tablet_bootstrap.cc:492] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.907473 14811 tablet_bootstrap.cc:654] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.907589 14853 raft_consensus.cc:491] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.907660 14853 raft_consensus.cc:513] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.907941 14853 leader_election.cc:290] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.908095 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet aa88f604011741f2b3eb2699b9328eae with delete_type TABLET_DATA_DELETED (replaced by tablet b561abc942014fb79f22fd6a9fec38d0) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:12.908378 14358 leader_election.cc:343] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:12.908632 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b13f8b40c2f40b8a93ce8030845665c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.908694 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e379cb7a61a8415994a7419266346d8e" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.908730 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c21c254cdef4015bfaba3fb5d61f554" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:12.908826 14689 raft_consensus.cc:2463] T 8c21c254cdef4015bfaba3fb5d61f554 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:12.908952 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c21c254cdef4015bfaba3fb5d61f554" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:12.909054 14422 raft_consensus.cc:2463] T 8c21c254cdef4015bfaba3fb5d61f554 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:12.909202 14493 leader_election.cc:304] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
W20250115 03:06:12.909536 14356 leader_election.cc:343] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:12.909595 14356 leader_election.cc:304] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:12.909711 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 594f755feed947afba1ba8b0853d5788 (table default.Table_dd9da62650c241fb9b1b74438090cf87 [id=a4d048fc192e46ae9dee9635a45a1229]) successfully deleted
I20250115 03:06:12.910625 14861 raft_consensus.cc:2743] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:12.910804 14853 raft_consensus.cc:2798] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:12.910856 14853 raft_consensus.cc:491] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:12.910888 14853 raft_consensus.cc:3054] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.911712 14853 raft_consensus.cc:513] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.911763 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_dffdb65871dc41b0ba48073022408925" } modify_external_catalogs: true
I20250115 03:06:12.911872 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_dffdb65871dc41b0ba48073022408925" } modify_external_catalogs: true
I20250115 03:06:12.911991 14853 leader_election.cc:290] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.912187 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_4d6634c15deb4073a13c251e972e912c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:12.912297 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_6fe85b67783046f99e0b160677207732"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.912508 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d6634c15deb4073a13c251e972e912c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:12.912640 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6fe85b67783046f99e0b160677207732 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.913859 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c21c254cdef4015bfaba3fb5d61f554" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:12.913947 14689 raft_consensus.cc:3054] T 8c21c254cdef4015bfaba3fb5d61f554 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.914696 14689 raft_consensus.cc:2463] T 8c21c254cdef4015bfaba3fb5d61f554 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:12.915517 14493 leader_election.cc:304] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:12.915647 14817 raft_consensus.cc:491] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.915699 14817 raft_consensus.cc:513] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.915944 14817 leader_election.cc:290] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.916005 14817 raft_consensus.cc:2798] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.916055 14817 raft_consensus.cc:695] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:12.916149 14817 consensus_queue.cc:237] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.916606 14820 tablet_replica.cc:331] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:12.916805 14820 raft_consensus.cc:2238] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.916915 14820 raft_consensus.cc:2267] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.916904 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c21c254cdef4015bfaba3fb5d61f554" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:12.917013 14422 raft_consensus.cc:3054] T 8c21c254cdef4015bfaba3fb5d61f554 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.917321 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4994733ae904c99986cabe4c6efc503" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:12.917433 14420 raft_consensus.cc:2463] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:12.917857 14422 raft_consensus.cc:2463] T 8c21c254cdef4015bfaba3fb5d61f554 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:12.919018 14820 ts_tablet_manager.cc:1905] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.919598 14493 leader_election.cc:304] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.919847 14851 raft_consensus.cc:2798] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:12.919903 14851 raft_consensus.cc:491] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:12.919940 14851 raft_consensus.cc:3054] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.920780 14851 raft_consensus.cc:513] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.921118 14851 leader_election.cc:290] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:12.921267 15151 raft_consensus.cc:491] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.921360 15151 raft_consensus.cc:513] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.921571 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4994733ae904c99986cabe4c6efc503" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:12.921660 14691 raft_consensus.cc:3054] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.921700 15151 leader_election.cc:290] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:12.922236 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4994733ae904c99986cabe4c6efc503" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:12.922343 14422 raft_consensus.cc:2463] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:12.922363 14691 raft_consensus.cc:2463] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:12.922441 14811 tablet_bootstrap.cc:492] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.922508 14811 ts_tablet_manager.cc:1397] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20250115 03:06:12.922708 14628 leader_election.cc:304] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.922816 14493 leader_election.cc:304] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:12.922950 14851 raft_consensus.cc:2798] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.922927 14811 raft_consensus.cc:357] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.923003 14851 raft_consensus.cc:695] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:12.923013 14811 raft_consensus.cc:383] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.923044 14811 raft_consensus.cc:738] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.923095 14851 consensus_queue.cc:237] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.923141 14811 consensus_queue.cc:260] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.923215 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4994733ae904c99986cabe4c6efc503" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:12.923290 14422 raft_consensus.cc:3054] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.923380 14811 ts_tablet_manager.cc:1428] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.923518 14811 tablet_bootstrap.cc:492] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.924064 14422 raft_consensus.cc:2463] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:12.924355 14811 tablet_bootstrap.cc:654] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.924456 15151 raft_consensus.cc:2758] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20250115 03:06:12.925750 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4994733ae904c99986cabe4c6efc503" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:12.925849 14689 raft_consensus.cc:2371] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1. Re-sending same reply.
I20250115 03:06:12.921264 14823 ts_tablet_manager.cc:1905] T 594f755feed947afba1ba8b0853d5788 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.927732 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4994733ae904c99986cabe4c6efc503" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.929126 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_6b67142e0bf14518ac39cdebdd31f278"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.929467 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6b67142e0bf14518ac39cdebdd31f278 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.930883 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2635fdd75c304cb1a3fd33bf84b66d27" } modify_external_catalogs: true
I20250115 03:06:12.931018 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2635fdd75c304cb1a3fd33bf84b66d27" } modify_external_catalogs: true
I20250115 03:06:12.931094 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b386f9bd2bc4416792a7480d1539e13d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.931378 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b386f9bd2bc4416792a7480d1539e13d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.930920 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet 36bda89e64d44e708bccbcab847634a1 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.934270 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 1ef14443e7fd464a8ab82eb33819dac3 (DEFAULT_TABLE table=default.Table_f590b49e96644f96934046c984688857 [id=fd7ff6d50c944f3a80089030d9fc895d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.934672 14811 tablet_bootstrap.cc:492] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.934738 14811 ts_tablet_manager.cc:1397] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20250115 03:06:12.935104 14811 raft_consensus.cc:357] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.935192 14811 raft_consensus.cc:383] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.935222 14811 raft_consensus.cc:738] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.935325 14811 consensus_queue.cc:260] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.935581 14811 ts_tablet_manager.cc:1428] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.936021 14811 tablet_bootstrap.cc:492] T e379cb7a61a8415994a7419266346d8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.936875 14811 tablet_bootstrap.cc:654] T e379cb7a61a8415994a7419266346d8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.939007 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2d802ab3dfcf4a54ae7c59c4c98d8446"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.939327 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2d802ab3dfcf4a54ae7c59c4c98d8446 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.939730 14812 tablet_bootstrap.cc:492] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.939811 14812 ts_tablet_manager.cc:1397] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.043s user 0.002s sys 0.000s
I20250115 03:06:12.940205 14812 raft_consensus.cc:357] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.940294 14812 raft_consensus.cc:383] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.940323 14812 raft_consensus.cc:738] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.940418 14812 consensus_queue.cc:260] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.940570 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ad994d4298c546df9e4808628545682a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:12.940658 14812 ts_tablet_manager.cc:1428] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.940842 14812 tablet_bootstrap.cc:492] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:12.940853 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ad994d4298c546df9e4808628545682a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.941671 14812 tablet_bootstrap.cc:654] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.943305 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 7479bfc96d634011ba15069b5cf6794c (table default.Table_dffdb65871dc41b0ba48073022408925 [id=f8ea91c4841b470db1b9f05fa47244fc]). Expected version 2 got 0
I20250115 03:06:12.944036 14400 tablet_service.cc:1467] Processing CreateTablet for tablet de2fcf7797684ef78e2c4f40b7298afd (DEFAULT_TABLE table=default.Table_64c4e8668b494feca761d65318286040 [id=265c248361624c3f9e5ece27144d8e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.944247 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de2fcf7797684ef78e2c4f40b7298afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.946138 14401 tablet_service.cc:1467] Processing CreateTablet for tablet b561abc942014fb79f22fd6a9fec38d0 (DEFAULT_TABLE table=default.Table_fa83f65f02b04ad491cee3b2a617927f [id=46f096547b3a4de9a46522b978ac4b8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.946326 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b561abc942014fb79f22fd6a9fec38d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.948503 14660 tablet_service.cc:1467] Processing CreateTablet for tablet b561abc942014fb79f22fd6a9fec38d0 (DEFAULT_TABLE table=default.Table_fa83f65f02b04ad491cee3b2a617927f [id=46f096547b3a4de9a46522b978ac4b8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.948743 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b561abc942014fb79f22fd6a9fec38d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.951308 14811 tablet_bootstrap.cc:492] T e379cb7a61a8415994a7419266346d8e P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.951398 14811 ts_tablet_manager.cc:1397] T e379cb7a61a8415994a7419266346d8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20250115 03:06:12.951807 14811 raft_consensus.cc:357] T e379cb7a61a8415994a7419266346d8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.951921 14811 raft_consensus.cc:383] T e379cb7a61a8415994a7419266346d8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.951967 14811 raft_consensus.cc:738] T e379cb7a61a8415994a7419266346d8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.952081 14811 consensus_queue.cc:260] T e379cb7a61a8415994a7419266346d8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.952345 14811 ts_tablet_manager.cc:1428] T e379cb7a61a8415994a7419266346d8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:12.952536 14811 tablet_bootstrap.cc:492] T 07d9c5e89ac34af29a406e79d2ace824 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.952697 14663 tablet_service.cc:1467] Processing CreateTablet for tablet de2fcf7797684ef78e2c4f40b7298afd (DEFAULT_TABLE table=default.Table_64c4e8668b494feca761d65318286040 [id=265c248361624c3f9e5ece27144d8e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.953020 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de2fcf7797684ef78e2c4f40b7298afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.953399 14811 tablet_bootstrap.cc:654] T 07d9c5e89ac34af29a406e79d2ace824 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:12.956620 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet b6a8a8da56274eaa8fef29fff38a7dae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6a8a8da56274eaa8fef29fff38a7dae already in progress: creating tablet
W20250115 03:06:12.957130 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 07d9c5e89ac34af29a406e79d2ace824 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 07d9c5e89ac34af29a406e79d2ace824 already in progress: creating tablet
I20250115 03:06:12.957370 14809 tablet_bootstrap.cc:492] T 876e28105fbb47bdb722e5d3f8467341 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.957479 14809 ts_tablet_manager.cc:1397] T 876e28105fbb47bdb722e5d3f8467341 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.082s user 0.002s sys 0.000s
I20250115 03:06:12.957895 14809 raft_consensus.cc:357] T 876e28105fbb47bdb722e5d3f8467341 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.957988 14809 raft_consensus.cc:383] T 876e28105fbb47bdb722e5d3f8467341 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.958015 14809 raft_consensus.cc:738] T 876e28105fbb47bdb722e5d3f8467341 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.958117 14809 consensus_queue.cc:260] T 876e28105fbb47bdb722e5d3f8467341 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.958307 14528 tablet_service.cc:1467] Processing CreateTablet for tablet de2fcf7797684ef78e2c4f40b7298afd (DEFAULT_TABLE table=default.Table_64c4e8668b494feca761d65318286040 [id=265c248361624c3f9e5ece27144d8e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.958369 14809 ts_tablet_manager.cc:1428] T 876e28105fbb47bdb722e5d3f8467341 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.958532 14809 tablet_bootstrap.cc:492] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.958734 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ef14443e7fd464a8ab82eb33819dac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.959085 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de2fcf7797684ef78e2c4f40b7298afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.959302 14812 tablet_bootstrap.cc:492] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.959373 14812 ts_tablet_manager.cc:1397] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.000s
I20250115 03:06:12.959519 14809 tablet_bootstrap.cc:654] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.959806 14812 raft_consensus.cc:357] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.959894 14812 raft_consensus.cc:383] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.959923 14812 raft_consensus.cc:738] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.960028 14812 consensus_queue.cc:260] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.960275 14812 ts_tablet_manager.cc:1428] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:12.961342 14812 tablet_bootstrap.cc:492] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.962384 14812 tablet_bootstrap.cc:654] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.964421 14527 tablet_service.cc:1467] Processing CreateTablet for tablet b561abc942014fb79f22fd6a9fec38d0 (DEFAULT_TABLE table=default.Table_fa83f65f02b04ad491cee3b2a617927f [id=46f096547b3a4de9a46522b978ac4b8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:12.964639 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b561abc942014fb79f22fd6a9fec38d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:12.966459 14820 ts_tablet_manager.cc:1918] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.967038 15151 raft_consensus.cc:491] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:12.967115 15151 raft_consensus.cc:513] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.967422 15151 leader_election.cc:290] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:12.967741 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2f815e6fe4e4936823d51fcd16b453b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:12.967837 14422 raft_consensus.cc:2463] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:12.968067 14628 leader_election.cc:304] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.968298 15151 raft_consensus.cc:2798] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:12.968345 15151 raft_consensus.cc:491] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:12.968376 15151 raft_consensus.cc:3054] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.969610 15151 raft_consensus.cc:513] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.969883 15151 leader_election.cc:290] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:12.970314 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2f815e6fe4e4936823d51fcd16b453b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:12.970417 14422 raft_consensus.cc:3054] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.971187 14422 raft_consensus.cc:2463] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:12.971544 14628 leader_election.cc:304] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:12.971843 15151 raft_consensus.cc:2798] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:12.971915 15151 raft_consensus.cc:695] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:12.972018 15151 consensus_queue.cc:237] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.975454 14812 tablet_bootstrap.cc:492] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.975545 14812 ts_tablet_manager.cc:1397] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20250115 03:06:12.975955 14812 raft_consensus.cc:357] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.976047 14812 raft_consensus.cc:383] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.976061 14809 tablet_bootstrap.cc:492] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:12.976078 14812 raft_consensus.cc:738] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.976133 14809 ts_tablet_manager.cc:1397] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20250115 03:06:12.976135 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e8e168dcba994883bb43ad269eac0c91"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:12.976177 14812 consensus_queue.cc:260] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.976408 14812 ts_tablet_manager.cc:1428] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20250115 03:06:12.976444 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e8e168dcba994883bb43ad269eac0c91 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.976578 14812 tablet_bootstrap.cc:492] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.976545 14809 raft_consensus.cc:357] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.976635 14809 raft_consensus.cc:383] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.976665 14809 raft_consensus.cc:738] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.976768 14809 consensus_queue.cc:260] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.977025 14809 ts_tablet_manager.cc:1428] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.977177 14809 tablet_bootstrap.cc:492] T 8a75594237ce49edbc5ba410038bbecd P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:12.977181 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 3ed567d482aa43ee9e24f992c346ebc3
I20250115 03:06:12.977545 14171 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6bc8705ef8c145478d0ccd20fcb3ce34" } new_table_name: "default.Table_53c9a73ae8ef459c987964efac552c96" modify_external_catalogs: true
I20250115 03:06:12.978000 14812 tablet_bootstrap.cc:654] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.978030 14809 tablet_bootstrap.cc:654] T 8a75594237ce49edbc5ba410038bbecd P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.978410 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3e807f869ab742ddaf0412ef84007cfe" } new_table_name: "default.Table_05065fcbba614582b828f0b729c8c07c" modify_external_catalogs: true
I20250115 03:06:12.979118 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 36bda89e64d44e708bccbcab847634a1
I20250115 03:06:12.979338 14163 catalog_manager.cc:6596] ReplaceTablet: tablet 36bda89e64d44e708bccbcab847634a1 deleted and replaced by tablet 7b2210a8bd9946508870844ee2e88fb1
I20250115 03:06:12.979493 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5b29b7e1e58f49c09b26040407bf49ce" } new_table_name: "default.Table_3bd81cb8bc2d4b74ae3dbcbdb74aae8b" modify_external_catalogs: true
I20250115 03:06:12.981758 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ed567d482aa43ee9e24f992c346ebc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.981792 14812 tablet_bootstrap.cc:492] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.981865 14812 ts_tablet_manager.cc:1397] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20250115 03:06:12.981875 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 36bda89e64d44e708bccbcab847634a1 with delete_type TABLET_DATA_DELETED (replaced by tablet 7b2210a8bd9946508870844ee2e88fb1) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:12.981972 14819 tablet_replica.cc:331] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.982075 14819 raft_consensus.cc:2238] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:12.982159 14819 raft_consensus.cc:2267] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.982240 14812 raft_consensus.cc:357] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.982335 14812 raft_consensus.cc:383] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.982363 14812 raft_consensus.cc:738] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.982460 14812 consensus_queue.cc:260] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.982652 14819 ts_tablet_manager.cc:1905] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.982723 14812 ts_tablet_manager.cc:1428] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:12.982863 14812 tablet_bootstrap.cc:492] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.983703 14812 tablet_bootstrap.cc:654] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.984464 14819 ts_tablet_manager.cc:1918] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.984517 14819 log.cc:1198] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/3ed567d482aa43ee9e24f992c346ebc3
I20250115 03:06:12.984719 14819 ts_tablet_manager.cc:1939] T 3ed567d482aa43ee9e24f992c346ebc3 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.985181 14819 tablet_replica.cc:331] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:12.985220 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 36bda89e64d44e708bccbcab847634a1 with delete_type TABLET_DATA_DELETED (replaced by tablet 7b2210a8bd9946508870844ee2e88fb1) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.985273 14819 raft_consensus.cc:2238] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:12.985412 14819 raft_consensus.cc:2267] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:12.985904 14819 ts_tablet_manager.cc:1905] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:12.968619 14811 tablet_bootstrap.cc:492] T 07d9c5e89ac34af29a406e79d2ace824 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.986310 14811 ts_tablet_manager.cc:1397] T 07d9c5e89ac34af29a406e79d2ace824 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.034s user 0.000s sys 0.002s
I20250115 03:06:12.986665 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ed567d482aa43ee9e24f992c346ebc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.986707 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 3ed567d482aa43ee9e24f992c346ebc3 (table default.Table_2635fdd75c304cb1a3fd33bf84b66d27 [id=e34781ee2864498bbc532144911630da]) successfully deleted
I20250115 03:06:12.986224 14820 log.cc:1198] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/aa88f604011741f2b3eb2699b9328eae
I20250115 03:06:12.974951 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2f815e6fe4e4936823d51fcd16b453b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:12.986980 14557 raft_consensus.cc:3054] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:12.987149 14820 ts_tablet_manager.cc:1939] T aa88f604011741f2b3eb2699b9328eae P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:12.987818 14557 raft_consensus.cc:2463] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:12.986783 14811 raft_consensus.cc:357] T 07d9c5e89ac34af29a406e79d2ace824 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.988063 14811 raft_consensus.cc:383] T 07d9c5e89ac34af29a406e79d2ace824 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.988078 14823 ts_tablet_manager.cc:1918] T 594f755feed947afba1ba8b0853d5788 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:12.988106 14811 raft_consensus.cc:738] T 07d9c5e89ac34af29a406e79d2ace824 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.988139 14823 log.cc:1198] T 594f755feed947afba1ba8b0853d5788 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/594f755feed947afba1ba8b0853d5788
I20250115 03:06:12.988230 14811 consensus_queue.cc:260] T 07d9c5e89ac34af29a406e79d2ace824 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.988320 14823 ts_tablet_manager.cc:1939] T 594f755feed947afba1ba8b0853d5788 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:12.988494 14811 ts_tablet_manager.cc:1428] T 07d9c5e89ac34af29a406e79d2ace824 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:12.988667 14811 tablet_bootstrap.cc:492] T 99abc1c7a20a4475921597f93c2db56c P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:12.989058 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 36bda89e64d44e708bccbcab847634a1 with delete_type TABLET_DATA_DELETED (replaced by tablet 7b2210a8bd9946508870844ee2e88fb1) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:12.989574 14811 tablet_bootstrap.cc:654] T 99abc1c7a20a4475921597f93c2db56c P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.989660 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ed567d482aa43ee9e24f992c346ebc3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:12.974829 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2f815e6fe4e4936823d51fcd16b453b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:12.990113 14556 raft_consensus.cc:2371] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1. Re-sending same reply.
I20250115 03:06:12.991242 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_040bb2a68caa47cd995d0f4936302d43" } new_table_name: "default.Table_a07d634b43f842438f4990cd7ff748b5" modify_external_catalogs: true
I20250115 03:06:12.992236 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_9432efe75c694779976428f3dea16679"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:12.992596 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9432efe75c694779976428f3dea16679 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:12.993918 14819 ts_tablet_manager.cc:1918] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:12.993995 14819 log.cc:1198] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/36bda89e64d44e708bccbcab847634a1
I20250115 03:06:12.994220 14819 ts_tablet_manager.cc:1939] T 36bda89e64d44e708bccbcab847634a1 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:12.994707 14812 tablet_bootstrap.cc:492] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:12.994769 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 36bda89e64d44e708bccbcab847634a1 (table default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52]) successfully deleted
I20250115 03:06:12.994794 14812 ts_tablet_manager.cc:1397] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.001s
I20250115 03:06:12.995195 14812 raft_consensus.cc:357] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.995318 14812 raft_consensus.cc:383] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.995354 14812 raft_consensus.cc:738] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.995467 14812 consensus_queue.cc:260] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.995743 14812 ts_tablet_manager.cc:1428] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.995914 14812 tablet_bootstrap.cc:492] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:12.996748 14812 tablet_bootstrap.cc:654] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:12.997442 14165 master_service.cc:946] ReplaceTablet: received request to replace tablet b41d9a854bac49599d3390c15f31e0fa from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:12.998291 14168 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 7479bfc96d634011ba15069b5cf6794c (table default.Table_dffdb65871dc41b0ba48073022408925 [id=f8ea91c4841b470db1b9f05fa47244fc]). Expected version 2 got 0
I20250115 03:06:12.998387 14168 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 9adb11e56c394461a91c924dc1eb404b (table default.Table_ea2592c9eb974e6ab84a4e2f799b5cae [id=d0fc70d8af9944099f8e7ee9a416399d]). Expected version 1 got 0
I20250115 03:06:12.998673 14811 tablet_bootstrap.cc:492] T 99abc1c7a20a4475921597f93c2db56c P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:12.998769 14811 ts_tablet_manager.cc:1397] T 99abc1c7a20a4475921597f93c2db56c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20250115 03:06:12.999178 14811 raft_consensus.cc:357] T 99abc1c7a20a4475921597f93c2db56c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.999280 14811 raft_consensus.cc:383] T 99abc1c7a20a4475921597f93c2db56c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:12.999318 14811 raft_consensus.cc:738] T 99abc1c7a20a4475921597f93c2db56c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:12.999431 14811 consensus_queue.cc:260] T 99abc1c7a20a4475921597f93c2db56c P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:12.999692 14811 ts_tablet_manager.cc:1428] T 99abc1c7a20a4475921597f93c2db56c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:12.999873 14811 tablet_bootstrap.cc:492] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.001065 14811 tablet_bootstrap.cc:654] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.002427 14853 raft_consensus.cc:491] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.002513 14853 raft_consensus.cc:513] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.002837 14853 leader_election.cc:290] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.003198 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d6b5a30f27e47a3a515fb34691b9b15" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:13.003208 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet aa88f604011741f2b3eb2699b9328eae (table default.Table_fa83f65f02b04ad491cee3b2a617927f [id=46f096547b3a4de9a46522b978ac4b8c]) successfully deleted
I20250115 03:06:13.003295 14689 raft_consensus.cc:2463] T 4d6b5a30f27e47a3a515fb34691b9b15 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:13.004225 14493 leader_election.cc:304] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:13.004297 14812 tablet_bootstrap.cc:492] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.004367 14812 ts_tablet_manager.cc:1397] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20250115 03:06:13.004678 14400 tablet_service.cc:1467] Processing CreateTablet for tablet e43ce064a02347efa81ca274d76c6960 (DEFAULT_TABLE table=default.Table_4d6634c15deb4073a13c251e972e912c [id=0d8deaf903704c00a4a0576354ecfbe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.004761 14812 raft_consensus.cc:357] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.004850 14812 raft_consensus.cc:383] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.004881 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e43ce064a02347efa81ca274d76c6960. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.004880 14812 raft_consensus.cc:738] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.005007 14812 consensus_queue.cc:260] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.005272 14812 ts_tablet_manager.cc:1428] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.005414 14812 tablet_bootstrap.cc:492] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.006235 14812 tablet_bootstrap.cc:654] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.006759 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 21cc97960f7c49c7bf57499e66fd7190 (DEFAULT_TABLE table=default.Table_6fe85b67783046f99e0b160677207732 [id=f89110bc2f1e4619a821be08eb35695f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.006945 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21cc97960f7c49c7bf57499e66fd7190. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.007875 14820 tablet_replica.cc:331] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.007992 14820 raft_consensus.cc:2238] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.008090 14820 raft_consensus.cc:2267] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.008668 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 594f755feed947afba1ba8b0853d5788 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.008944 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 21cc97960f7c49c7bf57499e66fd7190 (DEFAULT_TABLE table=default.Table_6fe85b67783046f99e0b160677207732 [id=f89110bc2f1e4619a821be08eb35695f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.009124 14853 raft_consensus.cc:2798] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.009177 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21cc97960f7c49c7bf57499e66fd7190. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.009193 14853 raft_consensus.cc:491] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.009230 14853 raft_consensus.cc:3054] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.009744 14811 tablet_bootstrap.cc:492] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.009811 14811 ts_tablet_manager.cc:1397] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20250115 03:06:13.010046 14853 raft_consensus.cc:513] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.010206 14811 raft_consensus.cc:357] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.010295 14811 raft_consensus.cc:383] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.010322 14811 raft_consensus.cc:738] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.010349 14853 leader_election.cc:290] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.010411 14811 consensus_queue.cc:260] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.010656 14811 ts_tablet_manager.cc:1428] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:13.010823 14811 tablet_bootstrap.cc:492] T 8a75594237ce49edbc5ba410038bbecd P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.011026 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d6b5a30f27e47a3a515fb34691b9b15" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:13.011121 14689 raft_consensus.cc:3054] T 4d6b5a30f27e47a3a515fb34691b9b15 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.011270 14663 tablet_service.cc:1467] Processing CreateTablet for tablet e43ce064a02347efa81ca274d76c6960 (DEFAULT_TABLE table=default.Table_4d6634c15deb4073a13c251e972e912c [id=0d8deaf903704c00a4a0576354ecfbe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.011508 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e43ce064a02347efa81ca274d76c6960. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.011685 14811 tablet_bootstrap.cc:654] T 8a75594237ce49edbc5ba410038bbecd P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.011927 14689 raft_consensus.cc:2463] T 4d6b5a30f27e47a3a515fb34691b9b15 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:13.012001 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d6b5a30f27e47a3a515fb34691b9b15 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:13.012295 14493 leader_election.cc:343] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 4d6b5a30f27e47a3a515fb34691b9b15
I20250115 03:06:13.012408 14493 leader_election.cc:304] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:13.012650 14858 raft_consensus.cc:2798] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.012724 14858 raft_consensus.cc:695] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:13.012835 14858 consensus_queue.cc:237] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:13.016552 14493 leader_election.cc:343] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 4d6b5a30f27e47a3a515fb34691b9b15
I20250115 03:06:13.017028 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_52ccd0b580b74eb8a28800769ed69007"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.017376 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_52ccd0b580b74eb8a28800769ed69007 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.018069 15153 raft_consensus.cc:491] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.018144 14817 raft_consensus.cc:491] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.018151 15153 raft_consensus.cc:513] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.018213 14817 raft_consensus.cc:513] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:13.018338 14171 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f21761ff87ac43d5b9b47b5905fa5867 consensus state has no leader
I20250115 03:06:13.018481 15153 leader_election.cc:290] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:13.018502 14817 leader_election.cc:290] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.018553 14171 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6c1f266c05474fa4a4ff61f5078bcfbc" } new_table_name: "default.Table_5965fe6c20c84944b7697734b1dba127" modify_external_catalogs: true
I20250115 03:06:13.018865 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99abc1c7a20a4475921597f93c2db56c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:13.018985 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7479bfc96d634011ba15069b5cf6794c" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:13.019089 14420 raft_consensus.cc:2463] T 7479bfc96d634011ba15069b5cf6794c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
W20250115 03:06:13.019196 14628 leader_election.cc:343] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:13.019317 14167 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 551d5ffdcd2d43fcb63e42c135fc2bd5 consensus state has no leader
I20250115 03:06:13.019308 14493 leader_election.cc:304] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:13.019447 14817 raft_consensus.cc:2798] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.019482 14817 raft_consensus.cc:491] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.019511 14817 raft_consensus.cc:3054] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.019562 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_938d94ee23e648b8ba1fbfcbec658c41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:13.019798 14812 tablet_bootstrap.cc:492] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
W20250115 03:06:13.019836 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_938d94ee23e648b8ba1fbfcbec658c41 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.019874 14812 ts_tablet_manager.cc:1397] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20250115 03:06:13.020304 14817 raft_consensus.cc:513] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.020295 14812 raft_consensus.cc:357] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.020396 14812 raft_consensus.cc:383] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.020430 14812 raft_consensus.cc:738] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.020527 14812 consensus_queue.cc:260] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.020582 14817 leader_election.cc:290] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.020720 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6d37c9d2d26e4b73ba4c230e471aeef7" } modify_external_catalogs: true
I20250115 03:06:13.020802 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6d37c9d2d26e4b73ba4c230e471aeef7" } modify_external_catalogs: true
I20250115 03:06:13.020849 14812 ts_tablet_manager.cc:1428] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.021027 14812 tablet_bootstrap.cc:492] T 876e28105fbb47bdb722e5d3f8467341 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.021222 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7479bfc96d634011ba15069b5cf6794c" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:13.021310 14691 raft_consensus.cc:3054] T 7479bfc96d634011ba15069b5cf6794c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.021894 14812 tablet_bootstrap.cc:654] T 876e28105fbb47bdb722e5d3f8467341 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.022039 14691 raft_consensus.cc:2463] T 7479bfc96d634011ba15069b5cf6794c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:13.022389 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7479bfc96d634011ba15069b5cf6794c" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:13.022493 14420 raft_consensus.cc:3054] T 7479bfc96d634011ba15069b5cf6794c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:13.023141 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9ffc6d1d09e04cc99a5e7a1e1af7bef8 consensus state has no leader
I20250115 03:06:13.023285 14420 raft_consensus.cc:2463] T 7479bfc96d634011ba15069b5cf6794c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:13.023334 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3e8022382eb84ab9bae6fa5bc222cd1d" } modify_external_catalogs: true
I20250115 03:06:13.023414 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3e8022382eb84ab9bae6fa5bc222cd1d" } modify_external_catalogs: true
I20250115 03:06:13.023546 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7479bfc96d634011ba15069b5cf6794c" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:13.023634 14689 raft_consensus.cc:2371] T 7479bfc96d634011ba15069b5cf6794c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1. Re-sending same reply.
I20250115 03:06:13.023768 14811 tablet_bootstrap.cc:492] T 8a75594237ce49edbc5ba410038bbecd P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.023845 14811 ts_tablet_manager.cc:1397] T 8a75594237ce49edbc5ba410038bbecd P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20250115 03:06:13.023849 14493 leader_election.cc:304] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:13.023998 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 7479bfc96d634011ba15069b5cf6794c
I20250115 03:06:13.024235 14811 raft_consensus.cc:357] T 8a75594237ce49edbc5ba410038bbecd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.024328 14811 raft_consensus.cc:383] T 8a75594237ce49edbc5ba410038bbecd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.024361 14811 raft_consensus.cc:738] T 8a75594237ce49edbc5ba410038bbecd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.024471 14811 consensus_queue.cc:260] T 8a75594237ce49edbc5ba410038bbecd P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.024709 14811 ts_tablet_manager.cc:1428] T 8a75594237ce49edbc5ba410038bbecd P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:13.024840 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet b41d9a854bac49599d3390c15f31e0fa
I20250115 03:06:13.024878 14811 tablet_bootstrap.cc:492] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.025096 14165 catalog_manager.cc:6596] ReplaceTablet: tablet b41d9a854bac49599d3390c15f31e0fa deleted and replaced by tablet 5883306d8b23451bad367b1809f38bcd
I20250115 03:06:13.025784 14811 tablet_bootstrap.cc:654] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.027498 14969 tablet.cc:1721] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:13.027565 14812 tablet_bootstrap.cc:492] T 876e28105fbb47bdb722e5d3f8467341 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.027631 14812 ts_tablet_manager.cc:1397] T 876e28105fbb47bdb722e5d3f8467341 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:13.028016 14812 raft_consensus.cc:357] T 876e28105fbb47bdb722e5d3f8467341 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.028101 14812 raft_consensus.cc:383] T 876e28105fbb47bdb722e5d3f8467341 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.028128 14812 raft_consensus.cc:738] T 876e28105fbb47bdb722e5d3f8467341 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.028236 14812 consensus_queue.cc:260] T 876e28105fbb47bdb722e5d3f8467341 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.028491 14812 ts_tablet_manager.cc:1428] T 876e28105fbb47bdb722e5d3f8467341 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.028635 14812 tablet_bootstrap.cc:492] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.029181 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet b41d9a854bac49599d3390c15f31e0fa with delete_type TABLET_DATA_DELETED (replaced by tablet 5883306d8b23451bad367b1809f38bcd) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.029551 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99abc1c7a20a4475921597f93c2db56c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.029649 14556 raft_consensus.cc:2463] T 99abc1c7a20a4475921597f93c2db56c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:13.029757 14812 tablet_bootstrap.cc:654] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.029891 14858 raft_consensus.cc:2798] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.029959 14858 raft_consensus.cc:695] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:13.030057 14858 consensus_queue.cc:237] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.032066 15157 tablet.cc:1721] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:13.032963 14812 tablet_bootstrap.cc:492] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.033035 14812 ts_tablet_manager.cc:1397] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250115 03:06:13.033280 14626 leader_election.cc:304] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:13.033489 15153 raft_consensus.cc:2798] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.033449 14812 raft_consensus.cc:357] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.033542 15153 raft_consensus.cc:491] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.033553 14812 raft_consensus.cc:383] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.033581 15153 raft_consensus.cc:3054] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.033584 14812 raft_consensus.cc:738] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.033702 14812 consensus_queue.cc:260] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.033972 14812 ts_tablet_manager.cc:1428] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.034108 14812 tablet_bootstrap.cc:492] T b6a8a8da56274eaa8fef29fff38a7dae P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.034440 15153 raft_consensus.cc:513] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.034718 14885 raft_consensus.cc:491] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.034754 15153 leader_election.cc:290] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:13.034785 14885 raft_consensus.cc:513] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.034932 14812 tablet_bootstrap.cc:654] T b6a8a8da56274eaa8fef29fff38a7dae P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.035059 14885 leader_election.cc:290] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.035737 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99abc1c7a20a4475921597f93c2db56c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:13.036026 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "338592cc70ad4e61ba90905a424a3871" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:13.036039 14628 leader_election.cc:343] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.036132 14691 raft_consensus.cc:2463] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:13.036377 14358 leader_election.cc:304] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:13.036507 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "338592cc70ad4e61ba90905a424a3871" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.036530 14885 raft_consensus.cc:2798] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.036579 14885 raft_consensus.cc:491] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.036597 14557 raft_consensus.cc:2463] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:13.036614 14885 raft_consensus.cc:3054] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.036803 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99abc1c7a20a4475921597f93c2db56c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:13.036875 14556 raft_consensus.cc:3054] T 99abc1c7a20a4475921597f93c2db56c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.037480 14885 raft_consensus.cc:513] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.037664 14556 raft_consensus.cc:2463] T 99abc1c7a20a4475921597f93c2db56c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:13.037804 14885 leader_election.cc:290] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.037935 14626 leader_election.cc:304] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:13.038116 15151 raft_consensus.cc:2798] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.038190 15151 raft_consensus.cc:695] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:13.038257 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "338592cc70ad4e61ba90905a424a3871" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:13.038293 15151 consensus_queue.cc:237] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.038343 14691 raft_consensus.cc:3054] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:13.038347 14628 consensus_peers.cc:487] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 -> Peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Couldn't send request to peer e8df7247ee9c43e0aa18216da2661f75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ed567d482aa43ee9e24f992c346ebc3. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:13.039139 14691 raft_consensus.cc:2463] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:13.039476 14811 tablet_bootstrap.cc:492] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.039458 14358 leader_election.cc:304] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:13.039551 14811 ts_tablet_manager.cc:1397] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20250115 03:06:13.039955 14811 raft_consensus.cc:357] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.040043 14811 raft_consensus.cc:383] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.040071 14811 raft_consensus.cc:738] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.040149 14811 consensus_queue.cc:260] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.040381 14811 ts_tablet_manager.cc:1428] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:13.040443 15153 raft_consensus.cc:491] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.040534 14811 tablet_bootstrap.cc:492] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.040509 15153 raft_consensus.cc:513] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.040761 15153 leader_election.cc:290] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:13.040777 14820 ts_tablet_manager.cc:1905] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.041488 14811 tablet_bootstrap.cc:654] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.042481 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d9c5e89ac34af29a406e79d2ace824" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:13.042814 14820 ts_tablet_manager.cc:1918] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20250115 03:06:13.042814 14628 leader_election.cc:343] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.042876 14820 log.cc:1198] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/3ed567d482aa43ee9e24f992c346ebc3
I20250115 03:06:13.042925 14817 raft_consensus.cc:491] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.042995 14817 raft_consensus.cc:513] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.043102 14820 ts_tablet_manager.cc:1939] T 3ed567d482aa43ee9e24f992c346ebc3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.043249 14817 leader_election.cc:290] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.044189 14885 raft_consensus.cc:2798] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.044261 14885 raft_consensus.cc:695] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:13.044361 14885 consensus_queue.cc:237] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.047348 14811 tablet_bootstrap.cc:492] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.047426 14811 ts_tablet_manager.cc:1397] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:13.047652 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "876e28105fbb47bdb722e5d3f8467341" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:13.047751 14687 raft_consensus.cc:2463] T 876e28105fbb47bdb722e5d3f8467341 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:13.047844 14811 raft_consensus.cc:357] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.047930 14811 raft_consensus.cc:383] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.047956 14811 raft_consensus.cc:738] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.048056 14811 consensus_queue.cc:260] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.048090 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "876e28105fbb47bdb722e5d3f8467341" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:13.048188 14420 raft_consensus.cc:2463] T 876e28105fbb47bdb722e5d3f8467341 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:13.048291 14811 ts_tablet_manager.cc:1428] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.048434 14811 tablet_bootstrap.cc:492] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.049311 14811 tablet_bootstrap.cc:654] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.053071 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "338592cc70ad4e61ba90905a424a3871" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:13.053162 14556 raft_consensus.cc:3054] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.053937 14556 raft_consensus.cc:2463] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:13.054411 14493 leader_election.cc:304] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:13.054630 14817 raft_consensus.cc:2798] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.054667 14817 raft_consensus.cc:491] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.054698 14817 raft_consensus.cc:3054] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.055462 14817 raft_consensus.cc:513] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.055738 14817 leader_election.cc:290] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:13.056043 14158 catalog_manager.cc:4945] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): alter failed for tablet 99e3ff9f0b364b2f968e0faf3cace781 (table default.Table_0dd17013f6334efe97618f0b520512c5 [id=488a9a6268f64669ae17e1a67caa73a5]),no further retry: Not found: Tablet not found: 99e3ff9f0b364b2f968e0faf3cace781
I20250115 03:06:13.057121 15153 raft_consensus.cc:491] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.057191 15153 raft_consensus.cc:513] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.057449 15153 leader_election.cc:290] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:13.057786 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e379cb7a61a8415994a7419266346d8e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:13.057889 14422 raft_consensus.cc:2463] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:13.057986 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet b41d9a854bac49599d3390c15f31e0fa with delete_type TABLET_DATA_DELETED (replaced by tablet 5883306d8b23451bad367b1809f38bcd) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.058140 14628 leader_election.cc:304] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:13.058696 15153 raft_consensus.cc:2798] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.058756 15153 raft_consensus.cc:491] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.058794 15153 raft_consensus.cc:3054] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.059269 14891 tablet.cc:1721] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:13.059702 15153 raft_consensus.cc:513] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.060050 15153 leader_election.cc:290] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:13.060722 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 7479bfc96d634011ba15069b5cf6794c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.061475 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e379cb7a61a8415994a7419266346d8e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:13.061563 14420 raft_consensus.cc:3054] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.062319 14420 raft_consensus.cc:2463] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:13.062567 14628 leader_election.cc:304] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:13.062745 15153 raft_consensus.cc:2798] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.062817 15153 raft_consensus.cc:695] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:13.062922 15153 consensus_queue.cc:237] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.064275 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "876e28105fbb47bdb722e5d3f8467341" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:13.064365 14689 raft_consensus.cc:3054] T 876e28105fbb47bdb722e5d3f8467341 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.065148 14689 raft_consensus.cc:2463] T 876e28105fbb47bdb722e5d3f8467341 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:13.065771 14493 leader_election.cc:304] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:13.066004 14817 raft_consensus.cc:2798] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.066071 14817 raft_consensus.cc:695] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:13.066174 14817 consensus_queue.cc:237] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.066535 14809 tablet_bootstrap.cc:492] T 8a75594237ce49edbc5ba410038bbecd P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.066612 14809 ts_tablet_manager.cc:1397] T 8a75594237ce49edbc5ba410038bbecd P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.089s user 0.001s sys 0.000s
I20250115 03:06:13.067173 14809 raft_consensus.cc:357] T 8a75594237ce49edbc5ba410038bbecd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.067272 14809 raft_consensus.cc:383] T 8a75594237ce49edbc5ba410038bbecd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.067301 14809 raft_consensus.cc:738] T 8a75594237ce49edbc5ba410038bbecd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.067428 14809 consensus_queue.cc:260] T 8a75594237ce49edbc5ba410038bbecd P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.067755 14809 ts_tablet_manager.cc:1428] T 8a75594237ce49edbc5ba410038bbecd P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.067899 14809 tablet_bootstrap.cc:492] T 99abc1c7a20a4475921597f93c2db56c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.068825 14809 tablet_bootstrap.cc:654] T 99abc1c7a20a4475921597f93c2db56c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.073058 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "876e28105fbb47bdb722e5d3f8467341" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:13.073153 14420 raft_consensus.cc:3054] T 876e28105fbb47bdb722e5d3f8467341 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.073894 14420 raft_consensus.cc:2463] T 876e28105fbb47bdb722e5d3f8467341 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:13.074664 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 338592cc70ad4e61ba90905a424a3871 (table default.Table_ee5faa8cd90f40199372322d90c5a204 [id=ff4681cab8ce44ceb45ed76c70c1132e]). Expected version 1 got 0
I20250115 03:06:13.074815 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 99abc1c7a20a4475921597f93c2db56c (table default.Table_35d1dd748a6f4e79adced77fe7b2d08d [id=f94088b0776f4be990ddcec04afbca43]). Expected version 1 got 0
I20250115 03:06:13.074904 14162 catalog_manager.cc:5526] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:13.075423 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 8a75594237ce49edbc5ba410038bbecd
I20250115 03:06:13.075572 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e379cb7a61a8415994a7419266346d8e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.075668 14557 raft_consensus.cc:2463] T e379cb7a61a8415994a7419266346d8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:13.075877 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e379cb7a61a8415994a7419266346d8e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:13.075943 14557 raft_consensus.cc:3054] T e379cb7a61a8415994a7419266346d8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.075955 14528 tablet_service.cc:1467] Processing CreateTablet for tablet 21cc97960f7c49c7bf57499e66fd7190 (DEFAULT_TABLE table=default.Table_6fe85b67783046f99e0b160677207732 [id=f89110bc2f1e4619a821be08eb35695f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.076122 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 7479bfc96d634011ba15069b5cf6794c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.076228 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet b41d9a854bac49599d3390c15f31e0fa with delete_type TABLET_DATA_DELETED (replaced by tablet 5883306d8b23451bad367b1809f38bcd) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.076694 14557 raft_consensus.cc:2463] T e379cb7a61a8415994a7419266346d8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:13.076992 14527 tablet_service.cc:1467] Processing CreateTablet for tablet e43ce064a02347efa81ca274d76c6960 (DEFAULT_TABLE table=default.Table_4d6634c15deb4073a13c251e972e912c [id=0d8deaf903704c00a4a0576354ecfbe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.077200 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21cc97960f7c49c7bf57499e66fd7190. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.077260 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a75594237ce49edbc5ba410038bbecd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.077380 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a75594237ce49edbc5ba410038bbecd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.077404 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d9c5e89ac34af29a406e79d2ace824" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.077507 14556 raft_consensus.cc:2463] T 07d9c5e89ac34af29a406e79d2ace824 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:13.078454 14401 tablet_service.cc:1467] Processing CreateTablet for tablet c4a63742df234d30bdac0173693159e4 (DEFAULT_TABLE table=default.Table_2d802ab3dfcf4a54ae7c59c4c98d8446 [id=8d381e00c93343238da5478825775d04]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.078553 14660 tablet_service.cc:1467] Processing CreateTablet for tablet c4a63742df234d30bdac0173693159e4 (DEFAULT_TABLE table=default.Table_2d802ab3dfcf4a54ae7c59c4c98d8446 [id=8d381e00c93343238da5478825775d04]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.078658 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4a63742df234d30bdac0173693159e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.078776 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4a63742df234d30bdac0173693159e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.079458 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e43ce064a02347efa81ca274d76c6960. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.079849 15151 raft_consensus.cc:491] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.079912 15151 raft_consensus.cc:513] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.080193 15151 leader_election.cc:290] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:13.080659 14626 leader_election.cc:304] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:13.080858 15151 raft_consensus.cc:2798] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.080904 15151 raft_consensus.cc:491] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.080966 15151 raft_consensus.cc:3054] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.081274 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9adb11e56c394461a91c924dc1eb404b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:13.081775 14819 tablet_replica.cc:331] T b41d9a854bac49599d3390c15f31e0fa P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.081779 15151 raft_consensus.cc:513] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.081977 14819 raft_consensus.cc:2238] T b41d9a854bac49599d3390c15f31e0fa P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.082095 14819 raft_consensus.cc:2267] T b41d9a854bac49599d3390c15f31e0fa P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.082090 15151 leader_election.cc:290] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
W20250115 03:06:13.082511 14628 leader_election.cc:343] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.082695 14819 ts_tablet_manager.cc:1905] T b41d9a854bac49599d3390c15f31e0fa P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.083186 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9adb11e56c394461a91c924dc1eb404b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.083279 14556 raft_consensus.cc:2463] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:13.083446 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d9c5e89ac34af29a406e79d2ace824" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:13.083475 14626 leader_election.cc:304] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:13.083513 14557 raft_consensus.cc:3054] T 07d9c5e89ac34af29a406e79d2ace824 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.083690 15151 raft_consensus.cc:2798] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.083752 15151 raft_consensus.cc:491] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.083791 15151 raft_consensus.cc:3054] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.084204 14557 raft_consensus.cc:2463] T 07d9c5e89ac34af29a406e79d2ace824 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:13.084487 15151 raft_consensus.cc:513] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.084609 14400 tablet_service.cc:1467] Processing CreateTablet for tablet ae0efe52ac6c40fbae45c725b07b03a3 (DEFAULT_TABLE table=default.Table_6b67142e0bf14518ac39cdebdd31f278 [id=6492de6e91c74ac6b2896a3cc81061dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.084805 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae0efe52ac6c40fbae45c725b07b03a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.084825 15151 leader_election.cc:290] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:13.084976 14667 tablet_service.cc:1467] Processing CreateTablet for tablet ae0efe52ac6c40fbae45c725b07b03a3 (DEFAULT_TABLE table=default.Table_6b67142e0bf14518ac39cdebdd31f278 [id=6492de6e91c74ac6b2896a3cc81061dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.085186 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae0efe52ac6c40fbae45c725b07b03a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.085222 14626 leader_election.cc:304] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:13.085350 15151 raft_consensus.cc:2798] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.085412 15151 raft_consensus.cc:695] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:13.085511 15151 consensus_queue.cc:237] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.087073 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9adb11e56c394461a91c924dc1eb404b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:13.087082 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9adb11e56c394461a91c924dc1eb404b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:13.087164 14557 raft_consensus.cc:3054] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:13.087370 14628 leader_election.cc:343] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.087913 14557 raft_consensus.cc:2463] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:13.088146 14626 leader_election.cc:304] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:13.089293 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 7b2210a8bd9946508870844ee2e88fb1 (DEFAULT_TABLE table=default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.089450 15151 raft_consensus.cc:2798] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.089511 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b2210a8bd9946508870844ee2e88fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.089532 15151 raft_consensus.cc:695] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:13.089637 15151 consensus_queue.cc:237] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.091116 14819 ts_tablet_manager.cc:1918] T b41d9a854bac49599d3390c15f31e0fa P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:13.092319 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 72f9b43975074ac5a7012b132ef74662 (DEFAULT_TABLE table=default.Table_b386f9bd2bc4416792a7480d1539e13d [id=4ae12906873e4ed59c179ec32ae13fa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.092533 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72f9b43975074ac5a7012b132ef74662. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.095882 14660 tablet_service.cc:1467] Processing CreateTablet for tablet f8c3406d44a74b20a4ef4b42e8d04ad0 (DEFAULT_TABLE table=default.Table_ad994d4298c546df9e4808628545682a [id=f88b8c3027ae47ceacd99aa8a2811628]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.096086 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8c3406d44a74b20a4ef4b42e8d04ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.098549 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 0ecd4741aacb45e2ae6404675f885125
I20250115 03:06:13.093598 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 7b2210a8bd9946508870844ee2e88fb1 (DEFAULT_TABLE table=default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.094839 14396 tablet_service.cc:1467] Processing CreateTablet for tablet f8c3406d44a74b20a4ef4b42e8d04ad0 (DEFAULT_TABLE table=default.Table_ad994d4298c546df9e4808628545682a [id=f88b8c3027ae47ceacd99aa8a2811628]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.100275 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8c3406d44a74b20a4ef4b42e8d04ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.101998 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ecd4741aacb45e2ae6404675f885125 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.102185 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b2210a8bd9946508870844ee2e88fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.102929 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ecd4741aacb45e2ae6404675f885125 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.094923 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 72f9b43975074ac5a7012b132ef74662 (DEFAULT_TABLE table=default.Table_b386f9bd2bc4416792a7480d1539e13d [id=4ae12906873e4ed59c179ec32ae13fa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.104108 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72f9b43975074ac5a7012b132ef74662. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:13.104077 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 7b13f8b40c2f40b8a93ce8030845665c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7b13f8b40c2f40b8a93ce8030845665c already in progress: creating tablet
I20250115 03:06:13.095014 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d9c5e89ac34af29a406e79d2ace824" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:13.106204 14819 log.cc:1198] T b41d9a854bac49599d3390c15f31e0fa P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/b41d9a854bac49599d3390c15f31e0fa
I20250115 03:06:13.106514 14819 ts_tablet_manager.cc:1939] T b41d9a854bac49599d3390c15f31e0fa P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.107474 14819 tablet_replica.cc:331] T 7479bfc96d634011ba15069b5cf6794c P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.107646 14819 raft_consensus.cc:2238] T 7479bfc96d634011ba15069b5cf6794c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.107769 14819 raft_consensus.cc:2267] T 7479bfc96d634011ba15069b5cf6794c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:13.109530 14628 leader_election.cc:343] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.112582 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet b41d9a854bac49599d3390c15f31e0fa (table default.Table_0bdf66c6040b4709b0a6be57188b9d86 [id=c8fdb0f9f9064f3eb8d19f4adf3c439f]) successfully deleted
I20250115 03:06:13.113914 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 594f755feed947afba1ba8b0853d5788 (table default.Table_dd9da62650c241fb9b1b74438090cf87 [id=a4d048fc192e46ae9dee9635a45a1229]) successfully deleted
I20250115 03:06:13.114410 14823 tablet_replica.cc:331] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.114714 14823 raft_consensus.cc:2238] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:13.114893 14823 pending_rounds.cc:62] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 1 pending ops.
I20250115 03:06:13.114955 14823 pending_rounds.cc:69] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384888159399936 op_type: NO_OP noop_request { }
I20250115 03:06:13.115041 14823 raft_consensus.cc:2883] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:13.115125 14823 raft_consensus.cc:2267] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.116503 14819 ts_tablet_manager.cc:1905] T 7479bfc96d634011ba15069b5cf6794c P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:13.118249 14171 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 20f86f93812849c4b7852a884f677c04 consensus state has no leader
I20250115 03:06:13.118353 14819 ts_tablet_manager.cc:1918] T 7479bfc96d634011ba15069b5cf6794c P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.118407 14819 log.cc:1198] T 7479bfc96d634011ba15069b5cf6794c P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/7479bfc96d634011ba15069b5cf6794c
I20250115 03:06:13.118582 14819 ts_tablet_manager.cc:1939] T 7479bfc96d634011ba15069b5cf6794c P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.119282 14819 tablet_replica.cc:331] T 8a75594237ce49edbc5ba410038bbecd P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.119380 14819 raft_consensus.cc:2238] T 8a75594237ce49edbc5ba410038bbecd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.119556 14167 catalog_manager.cc:5526] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:13.119462 14819 raft_consensus.cc:2267] T 8a75594237ce49edbc5ba410038bbecd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.119915 14167 catalog_manager.cc:5526] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:13.120072 14819 ts_tablet_manager.cc:1905] T 8a75594237ce49edbc5ba410038bbecd P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.120167 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 338592cc70ad4e61ba90905a424a3871 (table default.Table_ee5faa8cd90f40199372322d90c5a204 [id=ff4681cab8ce44ceb45ed76c70c1132e]). Expected version 1 got 0
I20250115 03:06:13.120255 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 99abc1c7a20a4475921597f93c2db56c (table default.Table_35d1dd748a6f4e79adced77fe7b2d08d [id=f94088b0776f4be990ddcec04afbca43]). Expected version 1 got 0
I20250115 03:06:13.120326 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 9adb11e56c394461a91c924dc1eb404b (table default.Table_ea2592c9eb974e6ab84a4e2f799b5cae [id=d0fc70d8af9944099f8e7ee9a416399d]). Expected version 1 got 0
I20250115 03:06:13.121701 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 876e28105fbb47bdb722e5d3f8467341 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.122251 14823 ts_tablet_manager.cc:1905] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.123656 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 7479bfc96d634011ba15069b5cf6794c (table default.Table_dffdb65871dc41b0ba48073022408925 [id=f8ea91c4841b470db1b9f05fa47244fc]) successfully deleted
I20250115 03:06:13.124159 14823 ts_tablet_manager.cc:1918] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:13.124234 14823 log.cc:1198] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/07d9c5e89ac34af29a406e79d2ace824
I20250115 03:06:13.124470 14823 ts_tablet_manager.cc:1939] T 07d9c5e89ac34af29a406e79d2ace824 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.125029 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 07d9c5e89ac34af29a406e79d2ace824 (table default.Table_64c4e8668b494feca761d65318286040 [id=265c248361624c3f9e5ece27144d8e04]) successfully deleted
I20250115 03:06:13.125561 14823 tablet_replica.cc:331] T 4d6b5a30f27e47a3a515fb34691b9b15 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
W20250115 03:06:13.125391 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet b6a8a8da56274eaa8fef29fff38a7dae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b6a8a8da56274eaa8fef29fff38a7dae already in progress: creating tablet
I20250115 03:06:13.125757 14823 raft_consensus.cc:2238] T 4d6b5a30f27e47a3a515fb34691b9b15 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.125841 14823 raft_consensus.cc:2267] T 4d6b5a30f27e47a3a515fb34691b9b15 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.127166 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 7479bfc96d634011ba15069b5cf6794c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.127593 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a75594237ce49edbc5ba410038bbecd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.128232 14528 tablet_service.cc:1467] Processing CreateTablet for tablet c4a63742df234d30bdac0173693159e4 (DEFAULT_TABLE table=default.Table_2d802ab3dfcf4a54ae7c59c4c98d8446 [id=8d381e00c93343238da5478825775d04]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.128459 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4a63742df234d30bdac0173693159e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.128588 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 7b2210a8bd9946508870844ee2e88fb1 (DEFAULT_TABLE table=default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.128795 14812 tablet_bootstrap.cc:492] T b6a8a8da56274eaa8fef29fff38a7dae P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.128826 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b2210a8bd9946508870844ee2e88fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.128862 14812 ts_tablet_manager.cc:1397] T b6a8a8da56274eaa8fef29fff38a7dae P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.095s user 0.001s sys 0.000s
I20250115 03:06:13.129194 14529 tablet_service.cc:1467] Processing CreateTablet for tablet f8c3406d44a74b20a4ef4b42e8d04ad0 (DEFAULT_TABLE table=default.Table_ad994d4298c546df9e4808628545682a [id=f88b8c3027ae47ceacd99aa8a2811628]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.129276 14812 raft_consensus.cc:357] T b6a8a8da56274eaa8fef29fff38a7dae P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.129364 14812 raft_consensus.cc:383] T b6a8a8da56274eaa8fef29fff38a7dae P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.129395 14812 raft_consensus.cc:738] T b6a8a8da56274eaa8fef29fff38a7dae P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.129436 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8c3406d44a74b20a4ef4b42e8d04ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.129509 14812 consensus_queue.cc:260] T b6a8a8da56274eaa8fef29fff38a7dae P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.129770 14812 ts_tablet_manager.cc:1428] T b6a8a8da56274eaa8fef29fff38a7dae P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.129931 14812 tablet_bootstrap.cc:492] T fc056b9dbc374d26bf3feaf524f4d834 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.130475 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 72f9b43975074ac5a7012b132ef74662 (DEFAULT_TABLE table=default.Table_b386f9bd2bc4416792a7480d1539e13d [id=4ae12906873e4ed59c179ec32ae13fa4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.130674 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72f9b43975074ac5a7012b132ef74662. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.130786 14812 tablet_bootstrap.cc:654] T fc056b9dbc374d26bf3feaf524f4d834 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.131312 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ecd4741aacb45e2ae6404675f885125 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.128468 14532 tablet_service.cc:1467] Processing CreateTablet for tablet ae0efe52ac6c40fbae45c725b07b03a3 (DEFAULT_TABLE table=default.Table_6b67142e0bf14518ac39cdebdd31f278 [id=6492de6e91c74ac6b2896a3cc81061dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.132586 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae0efe52ac6c40fbae45c725b07b03a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.135910 14823 ts_tablet_manager.cc:1905] T 4d6b5a30f27e47a3a515fb34691b9b15 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.137156 14165 catalog_manager.cc:5526] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:13.137348 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 338592cc70ad4e61ba90905a424a3871 (table default.Table_ee5faa8cd90f40199372322d90c5a204 [id=ff4681cab8ce44ceb45ed76c70c1132e]). Expected version 1 got 0
I20250115 03:06:13.139647 14812 tablet_bootstrap.cc:492] T fc056b9dbc374d26bf3feaf524f4d834 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.139729 14812 ts_tablet_manager.cc:1397] T fc056b9dbc374d26bf3feaf524f4d834 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20250115 03:06:13.140129 14812 raft_consensus.cc:357] T fc056b9dbc374d26bf3feaf524f4d834 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.140220 14812 raft_consensus.cc:383] T fc056b9dbc374d26bf3feaf524f4d834 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.140251 14812 raft_consensus.cc:738] T fc056b9dbc374d26bf3feaf524f4d834 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.140357 14812 consensus_queue.cc:260] T fc056b9dbc374d26bf3feaf524f4d834 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.140625 14812 ts_tablet_manager.cc:1428] T fc056b9dbc374d26bf3feaf524f4d834 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.140810 14812 tablet_bootstrap.cc:492] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.141680 14812 tablet_bootstrap.cc:654] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.145085 14819 ts_tablet_manager.cc:1918] T 8a75594237ce49edbc5ba410038bbecd P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.145166 14819 log.cc:1198] T 8a75594237ce49edbc5ba410038bbecd P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/8a75594237ce49edbc5ba410038bbecd
I20250115 03:06:13.145355 14819 ts_tablet_manager.cc:1939] T 8a75594237ce49edbc5ba410038bbecd P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.145767 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet 99abc1c7a20a4475921597f93c2db56c from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:13.146059 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet c6cd63ac1c084a0d8f166e63b0ed85bc from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:13.146677 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8a1be82e3d5648a9b4b5399d2f56e72f" } modify_external_catalogs: true
I20250115 03:06:13.146787 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8a1be82e3d5648a9b4b5399d2f56e72f" } modify_external_catalogs: true
I20250115 03:06:13.146843 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 8a75594237ce49edbc5ba410038bbecd (table default.Table_6d37c9d2d26e4b73ba4c230e471aeef7 [id=b3cf5d084f3348c091c6e5be7be4808f]) successfully deleted
I20250115 03:06:13.147670 14823 ts_tablet_manager.cc:1918] T 4d6b5a30f27e47a3a515fb34691b9b15 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.147734 14823 log.cc:1198] T 4d6b5a30f27e47a3a515fb34691b9b15 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/4d6b5a30f27e47a3a515fb34691b9b15
I20250115 03:06:13.147902 14823 ts_tablet_manager.cc:1939] T 4d6b5a30f27e47a3a515fb34691b9b15 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.148094 14819 tablet_replica.cc:331] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.148208 14819 raft_consensus.cc:2238] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.148296 14819 raft_consensus.cc:2267] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.148715 14819 ts_tablet_manager.cc:1905] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.149385 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 4d6b5a30f27e47a3a515fb34691b9b15 (table default.Table_fadcbb64abcd4a1b812defb8ed9d99f5 [id=e3077355979a401a89f0558104eb53cf]) successfully deleted
I20250115 03:06:13.150369 14819 ts_tablet_manager.cc:1918] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.150797 14162 catalog_manager.cc:5526] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:13.150990 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 9adb11e56c394461a91c924dc1eb404b (table default.Table_ea2592c9eb974e6ab84a4e2f799b5cae [id=d0fc70d8af9944099f8e7ee9a416399d]). Expected version 1 got 0
I20250115 03:06:13.151155 14162 catalog_manager.cc:5526] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:13.151311 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 99abc1c7a20a4475921597f93c2db56c (table default.Table_35d1dd748a6f4e79adced77fe7b2d08d [id=f94088b0776f4be990ddcec04afbca43]). Expected version 1 got 0
I20250115 03:06:13.151402 14162 catalog_manager.cc:5526] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:13.152019 14167 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_3bd81cb8bc2d4b74ae3dbcbdb74aae8b [id=05f9fd2ebec34d2299ea59c47a74c828] alter complete (version 4)
I20250115 03:06:13.152695 14812 tablet_bootstrap.cc:492] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.152756 14812 ts_tablet_manager.cc:1397] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20250115 03:06:13.153137 14812 raft_consensus.cc:357] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.153244 14812 raft_consensus.cc:383] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.153270 14812 raft_consensus.cc:738] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.153371 14823 tablet_replica.cc:331] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.153359 14812 consensus_queue.cc:260] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.153496 14823 raft_consensus.cc:2238] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.153585 14812 ts_tablet_manager.cc:1428] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20250115 03:06:13.153604 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 7b13f8b40c2f40b8a93ce8030845665c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7b13f8b40c2f40b8a93ce8030845665c already in progress: creating tablet
I20250115 03:06:13.153762 14812 tablet_bootstrap.cc:492] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.153880 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a75594237ce49edbc5ba410038bbecd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.150429 14819 log.cc:1198] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/0ecd4741aacb45e2ae6404675f885125
I20250115 03:06:13.154211 14819 ts_tablet_manager.cc:1939] T 0ecd4741aacb45e2ae6404675f885125 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.154599 14812 tablet_bootstrap.cc:654] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.154739 14819 tablet_replica.cc:331] T 876e28105fbb47bdb722e5d3f8467341 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.154729 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 0ecd4741aacb45e2ae6404675f885125 (table default.Table_3e8022382eb84ab9bae6fa5bc222cd1d [id=6fa83879f044445482db1ae6fde9f37f]) successfully deleted
I20250115 03:06:13.154840 14819 raft_consensus.cc:2238] T 876e28105fbb47bdb722e5d3f8467341 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.154909 14819 raft_consensus.cc:2267] T 876e28105fbb47bdb722e5d3f8467341 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.153597 14823 raft_consensus.cc:2267] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.155970 14823 ts_tablet_manager.cc:1905] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.156657 14819 ts_tablet_manager.cc:1905] T 876e28105fbb47bdb722e5d3f8467341 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.157838 14823 ts_tablet_manager.cc:1918] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:13.157900 14823 log.cc:1198] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/aa88f604011741f2b3eb2699b9328eae
I20250115 03:06:13.158115 14823 ts_tablet_manager.cc:1939] T aa88f604011741f2b3eb2699b9328eae P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.158573 14823 tablet_replica.cc:331] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.158648 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet aa88f604011741f2b3eb2699b9328eae (table default.Table_fa83f65f02b04ad491cee3b2a617927f [id=46f096547b3a4de9a46522b978ac4b8c]) successfully deleted
I20250115 03:06:13.158684 14823 raft_consensus.cc:2238] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.158779 14823 raft_consensus.cc:2267] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.159222 14819 ts_tablet_manager.cc:1918] T 876e28105fbb47bdb722e5d3f8467341 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.159333 14819 log.cc:1198] T 876e28105fbb47bdb722e5d3f8467341 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/876e28105fbb47bdb722e5d3f8467341
I20250115 03:06:13.159560 14819 ts_tablet_manager.cc:1939] T 876e28105fbb47bdb722e5d3f8467341 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.160446 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 876e28105fbb47bdb722e5d3f8467341 (table default.Table_94ee8224b6cd468bb2a1c15b2c2af35a [id=63bd4ca58023436eb2b453f6fa0ea0c4]) successfully deleted
W20250115 03:06:13.160588 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 8a75594237ce49edbc5ba410038bbecd because the tablet was not found. No further retry: Not found: Tablet not found: 8a75594237ce49edbc5ba410038bbecd
I20250115 03:06:13.160669 14165 master_service.cc:946] ReplaceTablet: received request to replace tablet 9c54a5df35e7422b84acc91e3a10aac3 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:13.161633 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_28b265c831a944c7bff1a3ee55a86c64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:13.161818 15153 raft_consensus.cc:491] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.161888 15153 raft_consensus.cc:513] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:13.162068 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_28b265c831a944c7bff1a3ee55a86c64 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.162173 15153 leader_election.cc:290] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:13.162521 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd2c39e3cf14fb8bcf2db6b10fcbb35" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:13.162899 14626 leader_election.cc:343] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.163139 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd2c39e3cf14fb8bcf2db6b10fcbb35" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:13.163400 14628 leader_election.cc:343] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.163461 14628 leader_election.cc:304] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:13.163612 15153 raft_consensus.cc:2743] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:13.164290 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet f4a0952cb8d1412081f3852ce9f60843
I20250115 03:06:13.164963 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet f4a0952cb8d1412081f3852ce9f60843 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.166167 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet f4a0952cb8d1412081f3852ce9f60843 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.166679 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_884bdba667784d59bc4fa65b5b437fcb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.166997 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_884bdba667784d59bc4fa65b5b437fcb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.168177 15153 raft_consensus.cc:491] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.168251 15153 raft_consensus.cc:513] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.168534 15153 leader_election.cc:290] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:13.168852 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a75594237ce49edbc5ba410038bbecd" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.168975 14557 raft_consensus.cc:2463] T 8a75594237ce49edbc5ba410038bbecd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:13.169190 14823 ts_tablet_manager.cc:1905] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.169211 14626 leader_election.cc:304] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
W20250115 03:06:13.169821 14628 leader_election.cc:343] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 8a75594237ce49edbc5ba410038bbecd
I20250115 03:06:13.169857 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet 876e28105fbb47bdb722e5d3f8467341 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.169986 15153 raft_consensus.cc:2798] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.170032 15153 raft_consensus.cc:491] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.170061 15153 raft_consensus.cc:3054] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.170122 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d9c5e89ac34af29a406e79d2ace824 with delete_type TABLET_DATA_DELETED (replaced by tablet de2fcf7797684ef78e2c4f40b7298afd) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.170842 15153 raft_consensus.cc:513] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.170961 14823 ts_tablet_manager.cc:1918] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:13.171032 14823 log.cc:1198] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/36bda89e64d44e708bccbcab847634a1
I20250115 03:06:13.171171 15153 leader_election.cc:290] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:13.171267 14823 ts_tablet_manager.cc:1939] T 36bda89e64d44e708bccbcab847634a1 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.171707 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet f4a0952cb8d1412081f3852ce9f60843 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.171716 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a75594237ce49edbc5ba410038bbecd" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:13.171891 14557 raft_consensus.cc:3054] T 8a75594237ce49edbc5ba410038bbecd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.171921 14823 tablet_replica.cc:331] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.172027 14823 raft_consensus.cc:2238] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:13.172194 14823 raft_consensus.cc:2267] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:13.171926 14628 leader_election.cc:343] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 8a75594237ce49edbc5ba410038bbecd
I20250115 03:06:13.172509 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 36bda89e64d44e708bccbcab847634a1 (table default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52]) successfully deleted
I20250115 03:06:13.172756 14557 raft_consensus.cc:2463] T 8a75594237ce49edbc5ba410038bbecd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:13.173375 14823 ts_tablet_manager.cc:1905] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.173405 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.173404 14626 leader_election.cc:304] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:13.173715 15153 raft_consensus.cc:2798] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.173789 15153 raft_consensus.cc:695] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:13.173888 15153 consensus_queue.cc:237] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.175060 14823 ts_tablet_manager.cc:1918] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:13.175119 14823 log.cc:1198] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/3ed567d482aa43ee9e24f992c346ebc3
I20250115 03:06:13.175328 14823 ts_tablet_manager.cc:1939] T 3ed567d482aa43ee9e24f992c346ebc3 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.175876 14823 tablet_replica.cc:331] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.175972 14823 raft_consensus.cc:2238] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:13.176115 14823 raft_consensus.cc:2267] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.176201 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 3ed567d482aa43ee9e24f992c346ebc3 (table default.Table_2635fdd75c304cb1a3fd33bf84b66d27 [id=e34781ee2864498bbc532144911630da]) successfully deleted
I20250115 03:06:13.176571 14823 ts_tablet_manager.cc:1905] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.178287 14823 ts_tablet_manager.cc:1918] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:13.178406 14823 log.cc:1198] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/b41d9a854bac49599d3390c15f31e0fa
I20250115 03:06:13.178669 14823 ts_tablet_manager.cc:1939] T b41d9a854bac49599d3390c15f31e0fa P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.179113 14823 tablet_replica.cc:331] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.179231 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet b41d9a854bac49599d3390c15f31e0fa (table default.Table_0bdf66c6040b4709b0a6be57188b9d86 [id=c8fdb0f9f9064f3eb8d19f4adf3c439f]) successfully deleted
I20250115 03:06:13.179255 14823 raft_consensus.cc:2238] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:13.179428 14823 pending_rounds.cc:62] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 1 pending ops.
I20250115 03:06:13.179466 14823 pending_rounds.cc:69] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384888521461760 op_type: NO_OP noop_request { }
I20250115 03:06:13.179515 14823 raft_consensus.cc:2883] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:13.179563 14823 raft_consensus.cc:2267] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.180059 14823 ts_tablet_manager.cc:1905] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.181723 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet d4f76d0b3a394a5f8aa33ad17b11da34 (table default.Table_0dd17013f6334efe97618f0b520512c5 [id=488a9a6268f64669ae17e1a67caa73a5]). Expected version 2 got 0
I20250115 03:06:13.181795 14823 ts_tablet_manager.cc:1918] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:13.181864 14823 log.cc:1198] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/8a75594237ce49edbc5ba410038bbecd
I20250115 03:06:13.182063 14823 ts_tablet_manager.cc:1939] T 8a75594237ce49edbc5ba410038bbecd P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
W20250115 03:06:13.182423 14162 catalog_manager.cc:5718] Failed to send AlterTable RPC for tablet d4f76d0b3a394a5f8aa33ad17b11da34 (table default.Table_0dd17013f6334efe97618f0b520512c5 [id=488a9a6268f64669ae17e1a67caa73a5], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet d4f76d0b3a394a5f8aa33ad17b11da34 consensus state has no leader
I20250115 03:06:13.182590 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 8a75594237ce49edbc5ba410038bbecd (table default.Table_6d37c9d2d26e4b73ba4c230e471aeef7 [id=b3cf5d084f3348c091c6e5be7be4808f]) successfully deleted
I20250115 03:06:13.182626 14823 tablet_replica.cc:331] T 0ecd4741aacb45e2ae6404675f885125 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.182718 14823 raft_consensus.cc:2238] T 0ecd4741aacb45e2ae6404675f885125 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.182798 14823 raft_consensus.cc:2267] T 0ecd4741aacb45e2ae6404675f885125 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.183192 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet b6a8a8da56274eaa8fef29fff38a7dae with delete_type TABLET_DATA_DELETED (replaced by tablet 1ef14443e7fd464a8ab82eb33819dac3) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.183346 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 876e28105fbb47bdb722e5d3f8467341 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.183465 14823 ts_tablet_manager.cc:1905] T 0ecd4741aacb45e2ae6404675f885125 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.183557 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d9c5e89ac34af29a406e79d2ace824 with delete_type TABLET_DATA_DELETED (replaced by tablet de2fcf7797684ef78e2c4f40b7298afd) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.185868 14823 ts_tablet_manager.cc:1918] T 0ecd4741aacb45e2ae6404675f885125 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.185909 14811 tablet_bootstrap.cc:492] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.185942 14823 log.cc:1198] T 0ecd4741aacb45e2ae6404675f885125 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/0ecd4741aacb45e2ae6404675f885125
I20250115 03:06:13.185992 14811 ts_tablet_manager.cc:1397] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.138s user 0.001s sys 0.000s
I20250115 03:06:13.186208 14823 ts_tablet_manager.cc:1939] T 0ecd4741aacb45e2ae6404675f885125 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.186376 14820 tablet_replica.cc:331] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.186525 14820 raft_consensus.cc:2238] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.186825 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 0ecd4741aacb45e2ae6404675f885125 (table default.Table_3e8022382eb84ab9bae6fa5bc222cd1d [id=6fa83879f044445482db1ae6fde9f37f]) successfully deleted
I20250115 03:06:13.187012 14823 tablet_replica.cc:331] T 7479bfc96d634011ba15069b5cf6794c P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.187109 14823 raft_consensus.cc:2238] T 7479bfc96d634011ba15069b5cf6794c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.187188 14823 raft_consensus.cc:2267] T 7479bfc96d634011ba15069b5cf6794c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.187611 14823 ts_tablet_manager.cc:1905] T 7479bfc96d634011ba15069b5cf6794c P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.189285 14823 ts_tablet_manager.cc:1918] T 7479bfc96d634011ba15069b5cf6794c P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.189339 14823 log.cc:1198] T 7479bfc96d634011ba15069b5cf6794c P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/7479bfc96d634011ba15069b5cf6794c
I20250115 03:06:13.189843 14823 ts_tablet_manager.cc:1939] T 7479bfc96d634011ba15069b5cf6794c P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.190335 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 7479bfc96d634011ba15069b5cf6794c (table default.Table_dffdb65871dc41b0ba48073022408925 [id=f8ea91c4841b470db1b9f05fa47244fc]) successfully deleted
W20250115 03:06:13.190704 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet f4a0952cb8d1412081f3852ce9f60843 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f4a0952cb8d1412081f3852ce9f60843 already in progress: creating tablet
I20250115 03:06:13.190807 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 9c54a5df35e7422b84acc91e3a10aac3
I20250115 03:06:13.190913 14823 tablet_replica.cc:331] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.190997 14823 raft_consensus.cc:2238] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.191046 14165 catalog_manager.cc:6596] ReplaceTablet: tablet 9c54a5df35e7422b84acc91e3a10aac3 deleted and replaced by tablet ef002211ee1c4a149232c8e433277add
I20250115 03:06:13.191067 14823 raft_consensus.cc:2267] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.191519 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c54a5df35e7422b84acc91e3a10aac3 with delete_type TABLET_DATA_DELETED (replaced by tablet ef002211ee1c4a149232c8e433277add) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.192020 14823 ts_tablet_manager.cc:1905] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.193699 14823 ts_tablet_manager.cc:1918] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.193804 14823 log.cc:1198] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/6fd2c39e3cf14fb8bcf2db6b10fcbb35
I20250115 03:06:13.193944 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c54a5df35e7422b84acc91e3a10aac3 with delete_type TABLET_DATA_DELETED (replaced by tablet ef002211ee1c4a149232c8e433277add) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.194100 14823 ts_tablet_manager.cc:1939] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.186506 14811 raft_consensus.cc:357] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.194437 14811 raft_consensus.cc:383] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.194483 14820 raft_consensus.cc:2267] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.194494 14811 raft_consensus.cc:738] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.194626 14811 consensus_queue.cc:260] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.194897 14811 ts_tablet_manager.cc:1428] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:13.195117 14811 tablet_bootstrap.cc:492] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.195242 14809 tablet_bootstrap.cc:492] T 99abc1c7a20a4475921597f93c2db56c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.195314 14809 ts_tablet_manager.cc:1397] T 99abc1c7a20a4475921597f93c2db56c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.127s user 0.002s sys 0.000s
I20250115 03:06:13.196022 14811 tablet_bootstrap.cc:654] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.196776 14809 raft_consensus.cc:357] T 99abc1c7a20a4475921597f93c2db56c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.196893 14809 raft_consensus.cc:383] T 99abc1c7a20a4475921597f93c2db56c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.196960 14809 raft_consensus.cc:738] T 99abc1c7a20a4475921597f93c2db56c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.196969 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35 (table default.Table_80b3b9aaffe94c7a9fef7e98b4fefb53 [id=32678ca965cc458daceb1ca31e5214af]) successfully deleted
I20250115 03:06:13.197203 14809 consensus_queue.cc:260] T 99abc1c7a20a4475921597f93c2db56c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.197484 14809 ts_tablet_manager.cc:1428] T 99abc1c7a20a4475921597f93c2db56c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:13.198704 14823 tablet_replica.cc:331] T b6a8a8da56274eaa8fef29fff38a7dae P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.198935 14823 raft_consensus.cc:2238] T b6a8a8da56274eaa8fef29fff38a7dae P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.199004 14809 tablet_bootstrap.cc:492] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.199100 14823 raft_consensus.cc:2267] T b6a8a8da56274eaa8fef29fff38a7dae P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.199534 14823 ts_tablet_manager.cc:1905] T b6a8a8da56274eaa8fef29fff38a7dae P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.201732 14809 tablet_bootstrap.cc:654] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.203083 14820 ts_tablet_manager.cc:1905] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.203758 14823 ts_tablet_manager.cc:1918] T b6a8a8da56274eaa8fef29fff38a7dae P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.203876 14823 log.cc:1198] T b6a8a8da56274eaa8fef29fff38a7dae P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/b6a8a8da56274eaa8fef29fff38a7dae
I20250115 03:06:13.204115 14823 ts_tablet_manager.cc:1939] T b6a8a8da56274eaa8fef29fff38a7dae P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.204728 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet b6a8a8da56274eaa8fef29fff38a7dae (table default.Table_f590b49e96644f96934046c984688857 [id=fd7ff6d50c944f3a80089030d9fc895d]) successfully deleted
I20250115 03:06:13.205046 14823 tablet_replica.cc:331] T 876e28105fbb47bdb722e5d3f8467341 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.205193 14823 raft_consensus.cc:2238] T 876e28105fbb47bdb722e5d3f8467341 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.205569 14823 raft_consensus.cc:2267] T 876e28105fbb47bdb722e5d3f8467341 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.205976 14823 ts_tablet_manager.cc:1905] T 876e28105fbb47bdb722e5d3f8467341 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.206209 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 99abc1c7a20a4475921597f93c2db56c
I20250115 03:06:13.206306 14809 tablet_bootstrap.cc:492] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.206373 14809 ts_tablet_manager.cc:1397] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:13.206462 14171 catalog_manager.cc:6596] ReplaceTablet: tablet 99abc1c7a20a4475921597f93c2db56c deleted and replaced by tablet 6629f2415b554c1c95927c721532fcfc
I20250115 03:06:13.206656 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet c6cd63ac1c084a0d8f166e63b0ed85bc
I20250115 03:06:13.206797 14809 raft_consensus.cc:357] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.206882 14164 catalog_manager.cc:6596] ReplaceTablet: tablet c6cd63ac1c084a0d8f166e63b0ed85bc deleted and replaced by tablet dca04d171deb488690ea58bd75995021
I20250115 03:06:13.206884 14809 raft_consensus.cc:383] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.206916 14809 raft_consensus.cc:738] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.207042 14809 consensus_queue.cc:260] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.207554 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 99abc1c7a20a4475921597f93c2db56c with delete_type TABLET_DATA_DELETED (replaced by tablet 6629f2415b554c1c95927c721532fcfc) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.207686 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet c6cd63ac1c084a0d8f166e63b0ed85bc with delete_type TABLET_DATA_DELETED (replaced by tablet dca04d171deb488690ea58bd75995021) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.208786 14823 ts_tablet_manager.cc:1918] T 876e28105fbb47bdb722e5d3f8467341 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.208895 14823 log.cc:1198] T 876e28105fbb47bdb722e5d3f8467341 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/876e28105fbb47bdb722e5d3f8467341
I20250115 03:06:13.209117 14809 ts_tablet_manager.cc:1428] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:13.209138 14823 ts_tablet_manager.cc:1939] T 876e28105fbb47bdb722e5d3f8467341 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.209693 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 876e28105fbb47bdb722e5d3f8467341 (table default.Table_94ee8224b6cd468bb2a1c15b2c2af35a [id=63bd4ca58023436eb2b453f6fa0ea0c4]) successfully deleted
W20250115 03:06:13.210076 14158 catalog_manager.cc:4839] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 07d9c5e89ac34af29a406e79d2ace824 because the tablet was not found. No further retry: Not found: Tablet not found: 07d9c5e89ac34af29a406e79d2ace824
I20250115 03:06:13.210287 14823 tablet_replica.cc:331] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.210397 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet c6cd63ac1c084a0d8f166e63b0ed85bc with delete_type TABLET_DATA_DELETED (replaced by tablet dca04d171deb488690ea58bd75995021) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.210533 14823 raft_consensus.cc:2238] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.210644 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 99abc1c7a20a4475921597f93c2db56c with delete_type TABLET_DATA_DELETED (replaced by tablet 6629f2415b554c1c95927c721532fcfc) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.210645 14823 raft_consensus.cc:2267] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.211138 14809 tablet_bootstrap.cc:492] T 07d9c5e89ac34af29a406e79d2ace824 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.212064 14823 ts_tablet_manager.cc:1905] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.212193 14809 tablet_bootstrap.cc:654] T 07d9c5e89ac34af29a406e79d2ace824 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.213711 14823 ts_tablet_manager.cc:1918] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:13.213776 14823 log.cc:1198] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/9c54a5df35e7422b84acc91e3a10aac3
I20250115 03:06:13.213975 14823 ts_tablet_manager.cc:1939] T 9c54a5df35e7422b84acc91e3a10aac3 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.214483 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 9c54a5df35e7422b84acc91e3a10aac3 (table default.Table_f957fe3487ec4546a46a8392108003f0 [id=fb3a764b51e74e08a281d128e21466e3]) successfully deleted
I20250115 03:06:13.214650 14823 tablet_replica.cc:331] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.214740 14823 raft_consensus.cc:2238] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.214818 14823 raft_consensus.cc:2267] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.215242 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c54a5df35e7422b84acc91e3a10aac3 with delete_type TABLET_DATA_DELETED (replaced by tablet ef002211ee1c4a149232c8e433277add) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.215330 14823 ts_tablet_manager.cc:1905] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.215914 14811 tablet_bootstrap.cc:492] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.215999 14811 ts_tablet_manager.cc:1397] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20250115 03:06:13.216207 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet c6cd63ac1c084a0d8f166e63b0ed85bc with delete_type TABLET_DATA_DELETED (replaced by tablet dca04d171deb488690ea58bd75995021) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.216421 14811 raft_consensus.cc:357] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.216518 14811 raft_consensus.cc:383] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.216548 14811 raft_consensus.cc:738] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.216666 14811 consensus_queue.cc:260] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.216966 14811 ts_tablet_manager.cc:1428] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.216871 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_06ad841d9f6e4ae9af7639a562f42036"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:13.217146 14811 tablet_bootstrap.cc:492] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
W20250115 03:06:13.217294 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_06ad841d9f6e4ae9af7639a562f42036 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.217855 14823 ts_tablet_manager.cc:1918] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20250115 03:06:13.217914 14823 log.cc:1198] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/c6cd63ac1c084a0d8f166e63b0ed85bc
I20250115 03:06:13.217926 14811 tablet_bootstrap.cc:654] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.218171 14823 ts_tablet_manager.cc:1939] T c6cd63ac1c084a0d8f166e63b0ed85bc P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.218406 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet 99abc1c7a20a4475921597f93c2db56c with delete_type TABLET_DATA_DELETED (replaced by tablet 6629f2415b554c1c95927c721532fcfc) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.218771 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet c6cd63ac1c084a0d8f166e63b0ed85bc (table default.Table_3bd81cb8bc2d4b74ae3dbcbdb74aae8b [id=05f9fd2ebec34d2299ea59c47a74c828]) successfully deleted
I20250115 03:06:13.218946 14823 tablet_replica.cc:331] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.220728 14823 raft_consensus.cc:2238] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:13.220863 14823 pending_rounds.cc:62] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 1 pending ops.
I20250115 03:06:13.220896 14823 pending_rounds.cc:69] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384887966212096 op_type: NO_OP noop_request { }
I20250115 03:06:13.220989 14823 raft_consensus.cc:2883] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:13.221046 14823 raft_consensus.cc:2267] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.221680 14823 ts_tablet_manager.cc:1905] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.223353 14823 ts_tablet_manager.cc:1918] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:13.223418 14823 log.cc:1198] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/99abc1c7a20a4475921597f93c2db56c
I20250115 03:06:13.223631 14823 ts_tablet_manager.cc:1939] T 99abc1c7a20a4475921597f93c2db56c P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.224069 14811 tablet_bootstrap.cc:492] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.224155 14811 ts_tablet_manager.cc:1397] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20250115 03:06:13.224552 14811 raft_consensus.cc:357] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.224645 14811 raft_consensus.cc:383] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.224678 14811 raft_consensus.cc:738] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.224802 14811 consensus_queue.cc:260] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.225098 14811 ts_tablet_manager.cc:1428] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.225095 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_aeb59441e2604aa189d7b95ea5e98ed2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:13.225257 14811 tablet_bootstrap.cc:492] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
W20250115 03:06:13.225423 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aeb59441e2604aa189d7b95ea5e98ed2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.226095 14811 tablet_bootstrap.cc:654] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.228062 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 99abc1c7a20a4475921597f93c2db56c (table default.Table_35d1dd748a6f4e79adced77fe7b2d08d [id=f94088b0776f4be990ddcec04afbca43]) successfully deleted
I20250115 03:06:13.230705 14811 tablet_bootstrap.cc:492] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.230779 14811 ts_tablet_manager.cc:1397] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:13.230895 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_65b9d218ca9743d7a7eaf7cec1e16c80" } modify_external_catalogs: true
I20250115 03:06:13.230989 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_65b9d218ca9743d7a7eaf7cec1e16c80" } modify_external_catalogs: true
I20250115 03:06:13.231137 14811 raft_consensus.cc:357] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.231213 14811 raft_consensus.cc:383] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.231238 14811 raft_consensus.cc:738] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.231346 14811 consensus_queue.cc:260] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.231606 14811 ts_tablet_manager.cc:1428] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.231832 15153 raft_consensus.cc:491] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.231911 15153 raft_consensus.cc:513] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.232259 15153 leader_election.cc:290] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:13.233132 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feefb2be7f1440b98d7893dbbf9db46c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:13.234519 14628 leader_election.cc:343] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.234846 14811 tablet_bootstrap.cc:492] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.234938 14809 tablet_bootstrap.cc:492] T 07d9c5e89ac34af29a406e79d2ace824 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.235006 14809 ts_tablet_manager.cc:1397] T 07d9c5e89ac34af29a406e79d2ace824 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20250115 03:06:13.235406 14809 raft_consensus.cc:357] T 07d9c5e89ac34af29a406e79d2ace824 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.235493 14809 raft_consensus.cc:383] T 07d9c5e89ac34af29a406e79d2ace824 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.235524 14809 raft_consensus.cc:738] T 07d9c5e89ac34af29a406e79d2ace824 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.235675 14811 tablet_bootstrap.cc:654] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.235630 14809 consensus_queue.cc:260] T 07d9c5e89ac34af29a406e79d2ace824 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.235692 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 35b90c4b0ef341c8a7c1bc517a1231ed
I20250115 03:06:13.235893 14809 ts_tablet_manager.cc:1428] T 07d9c5e89ac34af29a406e79d2ace824 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.236073 14809 tablet_bootstrap.cc:492] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.236359 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 35b90c4b0ef341c8a7c1bc517a1231ed with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.236414 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feefb2be7f1440b98d7893dbbf9db46c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.236536 14557 raft_consensus.cc:2463] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:13.236979 14823 tablet_replica.cc:331] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.236982 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet 35b90c4b0ef341c8a7c1bc517a1231ed from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:13.237085 14823 raft_consensus.cc:2238] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.237180 14823 raft_consensus.cc:2267] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.237641 14809 tablet_bootstrap.cc:654] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.237699 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 35b90c4b0ef341c8a7c1bc517a1231ed with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.237808 14823 ts_tablet_manager.cc:1905] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.238436 14820 ts_tablet_manager.cc:1918] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:13.238498 14820 log.cc:1198] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/36bda89e64d44e708bccbcab847634a1
I20250115 03:06:13.238711 14820 ts_tablet_manager.cc:1939] T 36bda89e64d44e708bccbcab847634a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.238948 14626 leader_election.cc:304] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:13.239079 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2af92748648345c0903a0f99da0d47c5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.239377 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2af92748648345c0903a0f99da0d47c5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.239557 14823 ts_tablet_manager.cc:1918] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:13.239612 14823 log.cc:1198] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/35b90c4b0ef341c8a7c1bc517a1231ed
I20250115 03:06:13.239823 14823 ts_tablet_manager.cc:1939] T 35b90c4b0ef341c8a7c1bc517a1231ed P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
W20250115 03:06:13.240110 14493 consensus_peers.cc:487] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 -> Peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Couldn't send request to peer 5c0dae7aff4b44c699275b95ba639770. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0ecd4741aacb45e2ae6404675f885125. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:13.240374 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 35b90c4b0ef341c8a7c1bc517a1231ed (table default.Table_65b9d218ca9743d7a7eaf7cec1e16c80 [id=1b4a3ef0246a4341869bf5163487cac6]) successfully deleted
I20250115 03:06:13.240782 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 3ed567d482aa43ee9e24f992c346ebc3 (table default.Table_2635fdd75c304cb1a3fd33bf84b66d27 [id=e34781ee2864498bbc532144911630da]) successfully deleted
I20250115 03:06:13.241300 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_42a1977fa70d4f16b43bf861b66524a7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.241590 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_42a1977fa70d4f16b43bf861b66524a7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.242830 15153 raft_consensus.cc:2798] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.242883 15153 raft_consensus.cc:491] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.242916 15153 raft_consensus.cc:3054] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.243654 15153 raft_consensus.cc:513] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:13.243906 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet f4a0952cb8d1412081f3852ce9f60843 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f4a0952cb8d1412081f3852ce9f60843 already in progress: creating tablet
I20250115 03:06:13.243948 15153 leader_election.cc:290] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:13.244437 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet 35b90c4b0ef341c8a7c1bc517a1231ed with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.244462 14819 tablet_replica.cc:331] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.244594 14819 raft_consensus.cc:2238] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:13.244660 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feefb2be7f1440b98d7893dbbf9db46c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:13.244781 14819 raft_consensus.cc:2267] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:13.244972 14628 leader_election.cc:343] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.245134 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feefb2be7f1440b98d7893dbbf9db46c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:13.245222 14557 raft_consensus.cc:3054] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.246028 14557 raft_consensus.cc:2463] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:13.246354 14626 leader_election.cc:304] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:13.251868 15153 raft_consensus.cc:2798] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.251950 15153 raft_consensus.cc:695] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:13.252061 15153 consensus_queue.cc:237] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.252382 14809 tablet_bootstrap.cc:492] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.252466 14809 ts_tablet_manager.cc:1397] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20250115 03:06:13.252882 14809 raft_consensus.cc:357] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.253008 14809 raft_consensus.cc:383] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.253043 14809 raft_consensus.cc:738] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.253142 14809 consensus_queue.cc:260] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.253378 14809 ts_tablet_manager.cc:1428] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.253551 14809 tablet_bootstrap.cc:492] T b6a8a8da56274eaa8fef29fff38a7dae P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.254442 14809 tablet_bootstrap.cc:654] T b6a8a8da56274eaa8fef29fff38a7dae P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.260229 14811 tablet_bootstrap.cc:492] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.260305 14811 ts_tablet_manager.cc:1397] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20250115 03:06:13.260672 14811 raft_consensus.cc:357] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.260761 14811 raft_consensus.cc:383] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.260785 14811 raft_consensus.cc:738] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.260881 14811 consensus_queue.cc:260] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.261134 14811 ts_tablet_manager.cc:1428] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.261220 14167 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 9adb11e56c394461a91c924dc1eb404b (table default.Table_ea2592c9eb974e6ab84a4e2f799b5cae [id=d0fc70d8af9944099f8e7ee9a416399d]). Expected version 1 got 0
I20250115 03:06:13.261284 14811 tablet_bootstrap.cc:492] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.262149 14165 catalog_manager.cc:5526] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:13.263561 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 0c969b652a564ed18fd41bebee47533c (DEFAULT_TABLE table=default.Table_938d94ee23e648b8ba1fbfcbec658c41 [id=e31e6fb5a7e941c494ef416ef4762fb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.263833 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c969b652a564ed18fd41bebee47533c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.263801 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 0c969b652a564ed18fd41bebee47533c (DEFAULT_TABLE table=default.Table_938d94ee23e648b8ba1fbfcbec658c41 [id=e31e6fb5a7e941c494ef416ef4762fb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.264005 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c969b652a564ed18fd41bebee47533c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.266553 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ea55848527c6480fbe8b8fa219480faa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.266908 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ea55848527c6480fbe8b8fa219480faa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:13.267422 14493 consensus_peers.cc:487] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 -> Peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Couldn't send request to peer e8df7247ee9c43e0aa18216da2661f75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0ecd4741aacb45e2ae6404675f885125. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:13.267683 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f5eb08f6d7974c8895385279b9b47805" } new_table_name: "default.Table_705abe6f3ef84a19911b5ee291bd84cc" modify_external_catalogs: true
I20250115 03:06:13.268484 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_52ccd0b580b74eb8a28800769ed69007" } new_table_name: "default.Table_117c4a2ba1cb4369881c9bc30c21781b" modify_external_catalogs: true
I20250115 03:06:13.269311 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5965fe6c20c84944b7697734b1dba127" } new_table_name: "default.Table_b44636b0d90d4972aceaa7d169a3a477" modify_external_catalogs: true
I20250115 03:06:13.270330 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_74d1acd4f65e41bdb046d79c090a111b" } modify_external_catalogs: true
I20250115 03:06:13.270349 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e9f7a3afca894ff18b614971924bbd50" } new_table_name: "default.Table_a7ef00d45deb49578042744e5f66f14e" modify_external_catalogs: true
I20250115 03:06:13.270421 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_74d1acd4f65e41bdb046d79c090a111b" } modify_external_catalogs: true
I20250115 03:06:13.271449 14532 tablet_service.cc:1467] Processing CreateTablet for tablet 0c969b652a564ed18fd41bebee47533c (DEFAULT_TABLE table=default.Table_938d94ee23e648b8ba1fbfcbec658c41 [id=e31e6fb5a7e941c494ef416ef4762fb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.271685 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c969b652a564ed18fd41bebee47533c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.273437 14689 raft_consensus.cc:1270] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:13.273815 14885 consensus_queue.cc:1035] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:13.274263 14861 raft_consensus.cc:491] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.274327 14861 raft_consensus.cc:513] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.274593 14861 leader_election.cc:290] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.274979 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9adb11e56c394461a91c924dc1eb404b" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:13.275827 14819 ts_tablet_manager.cc:1905] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.277498 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9adb11e56c394461a91c924dc1eb404b" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.277617 14556 raft_consensus.cc:2388] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8df7247ee9c43e0aa18216da2661f75 in current term 1: Already voted for candidate 5c0dae7aff4b44c699275b95ba639770 in this term.
I20250115 03:06:13.277807 14557 raft_consensus.cc:1270] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:13.279414 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 5ad5197958ce45de8b9d78174aaae689 (DEFAULT_TABLE table=default.Table_e8e168dcba994883bb43ad269eac0c91 [id=d8084de64a264d28a3370971617a694c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.279654 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ad5197958ce45de8b9d78174aaae689. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.279639 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 5883306d8b23451bad367b1809f38bcd (DEFAULT_TABLE table=default.Table_0bdf66c6040b4709b0a6be57188b9d86 [id=c8fdb0f9f9064f3eb8d19f4adf3c439f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.279842 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5883306d8b23451bad367b1809f38bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.281667 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 543bbb3ae87c46a6982e6fa1ae325db6 (DEFAULT_TABLE table=default.Table_9432efe75c694779976428f3dea16679 [id=98a21757d0884730ae2a1bbc97359df6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.281669 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 5ad5197958ce45de8b9d78174aaae689 (DEFAULT_TABLE table=default.Table_e8e168dcba994883bb43ad269eac0c91 [id=d8084de64a264d28a3370971617a694c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.281879 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ad5197958ce45de8b9d78174aaae689. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.281876 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 543bbb3ae87c46a6982e6fa1ae325db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.283567 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 543bbb3ae87c46a6982e6fa1ae325db6 (DEFAULT_TABLE table=default.Table_9432efe75c694779976428f3dea16679 [id=98a21757d0884730ae2a1bbc97359df6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.283751 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 543bbb3ae87c46a6982e6fa1ae325db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.285257 14356 leader_election.cc:304] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:13.285511 14885 raft_consensus.cc:3054] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.286271 14885 raft_consensus.cc:2743] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:13.286235 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 5883306d8b23451bad367b1809f38bcd (DEFAULT_TABLE table=default.Table_0bdf66c6040b4709b0a6be57188b9d86 [id=c8fdb0f9f9064f3eb8d19f4adf3c439f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.286375 14885 consensus_queue.cc:1035] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:13.286446 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5883306d8b23451bad367b1809f38bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.288443 14660 tablet_service.cc:1467] Processing CreateTablet for tablet d4f770f63a7348afb71904b93c68c379 (DEFAULT_TABLE table=default.Table_52ccd0b580b74eb8a28800769ed69007 [id=50c73c26b4ed47a4961348db433b7dd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.288707 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4f770f63a7348afb71904b93c68c379. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.291143 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d8f964febee84681a352ac34900c1e6c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.291472 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d8f964febee84681a352ac34900c1e6c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.292769 14400 tablet_service.cc:1467] Processing CreateTablet for tablet d4f770f63a7348afb71904b93c68c379 (DEFAULT_TABLE table=default.Table_52ccd0b580b74eb8a28800769ed69007 [id=50c73c26b4ed47a4961348db433b7dd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.293012 14851 raft_consensus.cc:491] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.293076 14851 raft_consensus.cc:513] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.293224 14812 tablet_bootstrap.cc:492] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.293298 14812 ts_tablet_manager.cc:1397] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.140s user 0.001s sys 0.000s
I20250115 03:06:13.293340 14851 leader_election.cc:290] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.293690 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4f76d0b3a394a5f8aa33ad17b11da34" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:13.293721 14812 raft_consensus.cc:357] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.293792 14691 raft_consensus.cc:2463] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:13.293812 14812 raft_consensus.cc:383] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.293840 14812 raft_consensus.cc:738] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.293946 14812 consensus_queue.cc:260] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.294024 14493 leader_election.cc:304] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:13.294183 14812 ts_tablet_manager.cc:1428] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:13.294350 14812 tablet_bootstrap.cc:492] T f4a0952cb8d1412081f3852ce9f60843 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.295380 14812 tablet_bootstrap.cc:654] T f4a0952cb8d1412081f3852ce9f60843 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.295817 14851 raft_consensus.cc:2798] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.295871 14851 raft_consensus.cc:491] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.295902 14851 raft_consensus.cc:3054] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.296722 14851 raft_consensus.cc:513] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.297034 14851 leader_election.cc:290] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.297401 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4f76d0b3a394a5f8aa33ad17b11da34" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:13.297488 14691 raft_consensus.cc:3054] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.298225 14691 raft_consensus.cc:2463] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:13.298367 14820 tablet_replica.cc:331] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.298483 14820 raft_consensus.cc:2238] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:13.298609 14820 pending_rounds.cc:62] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1: Trying to abort 1 pending ops.
I20250115 03:06:13.298645 14820 pending_rounds.cc:69] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384887862099968 op_type: NO_OP noop_request { }
I20250115 03:06:13.298667 14493 leader_election.cc:304] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:13.299175 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4f76d0b3a394a5f8aa33ad17b11da34" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:13.299227 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4f76d0b3a394a5f8aa33ad17b11da34" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
W20250115 03:06:13.299512 14493 leader_election.cc:343] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:13.299669 14493 leader_election.cc:343] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.301134 14817 raft_consensus.cc:2798] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.301216 14817 raft_consensus.cc:695] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:13.301322 14817 consensus_queue.cc:237] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.301858 14817 raft_consensus.cc:491] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.301872 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 20f86f93812849c4b7852a884f677c04 (table default.Table_5965fe6c20c84944b7697734b1dba127 [id=f8f2a7ff6fdc40468de6ae9ba28c1427]). Expected version 1 got 0
I20250115 03:06:13.301923 14817 raft_consensus.cc:513] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.302189 14817 leader_election.cc:290] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.303417 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d9c5e89ac34af29a406e79d2ace824 with delete_type TABLET_DATA_DELETED (replaced by tablet de2fcf7797684ef78e2c4f40b7298afd) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.304010 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc056b9dbc374d26bf3feaf524f4d834" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:13.304024 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc056b9dbc374d26bf3feaf524f4d834" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:13.304122 14691 raft_consensus.cc:2463] T fc056b9dbc374d26bf3feaf524f4d834 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:13.304585 14493 leader_election.cc:304] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:13.304749 14853 raft_consensus.cc:2798] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.304791 14853 raft_consensus.cc:491] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.304824 14853 raft_consensus.cc:3054] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.305644 14853 raft_consensus.cc:513] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.305958 14853 leader_election.cc:290] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:13.306690 14493 leader_election.cc:343] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.307142 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc056b9dbc374d26bf3feaf524f4d834" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:13.307157 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc056b9dbc374d26bf3feaf524f4d834" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:13.307242 14691 raft_consensus.cc:3054] T fc056b9dbc374d26bf3feaf524f4d834 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.307998 14691 raft_consensus.cc:2463] T fc056b9dbc374d26bf3feaf524f4d834 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
W20250115 03:06:13.308382 14493 leader_election.cc:343] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.308490 14493 leader_election.cc:304] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:13.308678 14853 raft_consensus.cc:2798] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.308750 14853 raft_consensus.cc:695] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:13.308861 14853 consensus_queue.cc:237] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.310395 14532 tablet_service.cc:1467] Processing CreateTablet for tablet d4f770f63a7348afb71904b93c68c379 (DEFAULT_TABLE table=default.Table_52ccd0b580b74eb8a28800769ed69007 [id=50c73c26b4ed47a4961348db433b7dd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.310626 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4f770f63a7348afb71904b93c68c379. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.310643 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 5883306d8b23451bad367b1809f38bcd (DEFAULT_TABLE table=default.Table_0bdf66c6040b4709b0a6be57188b9d86 [id=c8fdb0f9f9064f3eb8d19f4adf3c439f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.310878 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5883306d8b23451bad367b1809f38bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.311601 14811 tablet_bootstrap.cc:654] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.312172 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4f770f63a7348afb71904b93c68c379. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.313158 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 5ad5197958ce45de8b9d78174aaae689 (DEFAULT_TABLE table=default.Table_e8e168dcba994883bb43ad269eac0c91 [id=d8084de64a264d28a3370971617a694c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.313511 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ad5197958ce45de8b9d78174aaae689. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.315619 14532 tablet_service.cc:1467] Processing CreateTablet for tablet 543bbb3ae87c46a6982e6fa1ae325db6 (DEFAULT_TABLE table=default.Table_9432efe75c694779976428f3dea16679 [id=98a21757d0884730ae2a1bbc97359df6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.315814 14851 consensus_queue.cc:1035] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:13.315915 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 543bbb3ae87c46a6982e6fa1ae325db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.317106 14167 master_service.cc:946] ReplaceTablet: received request to replace tablet 7479bfc96d634011ba15069b5cf6794c from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:13.320624 14422 raft_consensus.cc:1270] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 5c0dae7aff4b44c699275b95ba639770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:13.321023 15151 consensus_queue.cc:1035] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:13.321355 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 28324608e3764b7e8a550bc71a4bad58 consensus state has no leader
I20250115 03:06:13.322147 14556 raft_consensus.cc:1270] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer 5c0dae7aff4b44c699275b95ba639770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:13.322739 15151 consensus_queue.cc:1035] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:13.298699 14820 raft_consensus.cc:2883] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:13.323355 14820 raft_consensus.cc:2267] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.323932 14820 ts_tablet_manager.cc:1905] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.325907 14851 consensus_queue.cc:1035] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:13.327107 14820 ts_tablet_manager.cc:1918] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:13.327173 14820 log.cc:1198] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/4d6b5a30f27e47a3a515fb34691b9b15
I20250115 03:06:13.327397 14820 ts_tablet_manager.cc:1939] T 4d6b5a30f27e47a3a515fb34691b9b15 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
W20250115 03:06:13.329334 14158 catalog_manager.cc:4945] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): alter failed for tablet 99abc1c7a20a4475921597f93c2db56c (table default.Table_35d1dd748a6f4e79adced77fe7b2d08d [id=f94088b0776f4be990ddcec04afbca43]),no further retry: Not found: Tablet not found: 99abc1c7a20a4475921597f93c2db56c
I20250115 03:06:13.329982 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 36bda89e64d44e708bccbcab847634a1 (table default.Table_d68e408815964b6e93e2ed412c1f82e4 [id=9dcc0154ed914304a15cf96049afaf52]) successfully deleted
W20250115 03:06:13.330106 14156 catalog_manager.cc:4839] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 594f755feed947afba1ba8b0853d5788 because the tablet was not found. No further retry: Not found: Tablet not found: 594f755feed947afba1ba8b0853d5788
I20250115 03:06:13.330371 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 4d6b5a30f27e47a3a515fb34691b9b15 (table default.Table_fadcbb64abcd4a1b812defb8ed9d99f5 [id=e3077355979a401a89f0558104eb53cf]) successfully deleted
I20250115 03:06:13.334131 14969 tablet.cc:1721] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:13.335016 14891 tablet.cc:1721] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:13.336170 15157 tablet.cc:1721] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:13.336320 14819 ts_tablet_manager.cc:1918] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:13.336383 14819 log.cc:1198] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/9c54a5df35e7422b84acc91e3a10aac3
I20250115 03:06:13.336620 14819 ts_tablet_manager.cc:1939] T 9c54a5df35e7422b84acc91e3a10aac3 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.337239 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_46f14a41d55b4e4783139238fbd0ba49"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.337783 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_46f14a41d55b4e4783139238fbd0ba49 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.338927 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d75a227f24d644ceb2bda90d1f6ae998"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.339282 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d75a227f24d644ceb2bda90d1f6ae998 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.340497 14164 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 9adb11e56c394461a91c924dc1eb404b (table default.Table_ea2592c9eb974e6ab84a4e2f799b5cae [id=d0fc70d8af9944099f8e7ee9a416399d]). Expected version 1 got 0
I20250115 03:06:13.342295 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_937e6e7e4b014731b5945f00775aa14c" } modify_external_catalogs: true
I20250115 03:06:13.342775 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_937e6e7e4b014731b5945f00775aa14c" } modify_external_catalogs: true
I20250115 03:06:13.345342 14858 consensus_queue.cc:1035] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:13.352090 14162 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d4f770f63a7348afb71904b93c68c379 consensus state has no leader
I20250115 03:06:13.352346 14858 raft_consensus.cc:491] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.352417 14858 raft_consensus.cc:513] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.352730 14858 leader_election.cc:290] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.353348 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f86f93812849c4b7852a884f677c04" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:13.353453 14687 raft_consensus.cc:2463] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:13.353737 14493 leader_election.cc:304] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:13.353890 14858 raft_consensus.cc:2798] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.353933 14858 raft_consensus.cc:491] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.353964 14858 raft_consensus.cc:3054] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.354844 14858 raft_consensus.cc:513] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.355125 14858 leader_election.cc:290] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.355502 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f86f93812849c4b7852a884f677c04" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:13.355593 14687 raft_consensus.cc:3054] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.356484 14687 raft_consensus.cc:2463] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:13.356788 14493 leader_election.cc:304] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:13.357118 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f86f93812849c4b7852a884f677c04" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:13.357182 14858 raft_consensus.cc:2798] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.357237 14420 raft_consensus.cc:2463] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:13.357252 14858 raft_consensus.cc:695] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:13.357362 14858 consensus_queue.cc:237] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.357524 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f86f93812849c4b7852a884f677c04" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:13.357626 14421 raft_consensus.cc:3054] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.358481 14421 raft_consensus.cc:2463] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
W20250115 03:06:13.359252 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bdc2e882dc7e469e857d74e41ee2f435 consensus state has no leader
I20250115 03:06:13.359424 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a07d634b43f842438f4990cd7ff748b5" } modify_external_catalogs: true
I20250115 03:06:13.359500 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a07d634b43f842438f4990cd7ff748b5" } modify_external_catalogs: true
W20250115 03:06:13.360060 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 20f86f93812849c4b7852a884f677c04 consensus state has no leader
I20250115 03:06:13.360268 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 108aafc0c034498bbd300be9c6b1f8c4
I20250115 03:06:13.361142 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 108aafc0c034498bbd300be9c6b1f8c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.363678 14811 tablet_bootstrap.cc:492] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.363772 14811 ts_tablet_manager.cc:1397] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.103s user 0.001s sys 0.000s
I20250115 03:06:13.364187 14811 raft_consensus.cc:357] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.364280 14811 raft_consensus.cc:383] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.364306 14811 raft_consensus.cc:738] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.364396 14811 consensus_queue.cc:260] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.364640 14811 ts_tablet_manager.cc:1428] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.364812 14811 tablet_bootstrap.cc:492] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.365479 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 108aafc0c034498bbd300be9c6b1f8c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.365700 14823 tablet_replica.cc:331] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.365789 14811 tablet_bootstrap.cc:654] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.365839 14823 raft_consensus.cc:2238] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.365953 14823 raft_consensus.cc:2267] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.366315 14823 ts_tablet_manager.cc:1905] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.366806 14812 tablet_bootstrap.cc:492] T f4a0952cb8d1412081f3852ce9f60843 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.366883 14812 ts_tablet_manager.cc:1397] T f4a0952cb8d1412081f3852ce9f60843 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.073s user 0.002s sys 0.000s
I20250115 03:06:13.367301 14812 raft_consensus.cc:357] T f4a0952cb8d1412081f3852ce9f60843 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.367389 14812 raft_consensus.cc:383] T f4a0952cb8d1412081f3852ce9f60843 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.367419 14812 raft_consensus.cc:738] T f4a0952cb8d1412081f3852ce9f60843 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.367532 14812 consensus_queue.cc:260] T f4a0952cb8d1412081f3852ce9f60843 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.367790 14812 ts_tablet_manager.cc:1428] T f4a0952cb8d1412081f3852ce9f60843 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.367951 14812 tablet_bootstrap.cc:492] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.368410 14823 ts_tablet_manager.cc:1918] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:13.368479 14823 log.cc:1198] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/108aafc0c034498bbd300be9c6b1f8c4
I20250115 03:06:13.368690 14823 ts_tablet_manager.cc:1939] T 108aafc0c034498bbd300be9c6b1f8c4 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.368786 14812 tablet_bootstrap.cc:654] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.369367 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 108aafc0c034498bbd300be9c6b1f8c4 (table default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd]) successfully deleted
I20250115 03:06:13.369694 15151 consensus_queue.cc:1035] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:13.373266 14809 tablet_bootstrap.cc:492] T b6a8a8da56274eaa8fef29fff38a7dae P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.373356 14809 ts_tablet_manager.cc:1397] T b6a8a8da56274eaa8fef29fff38a7dae P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.120s user 0.002s sys 0.000s
I20250115 03:06:13.373757 14809 raft_consensus.cc:357] T b6a8a8da56274eaa8fef29fff38a7dae P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.373850 14809 raft_consensus.cc:383] T b6a8a8da56274eaa8fef29fff38a7dae P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.373878 14809 raft_consensus.cc:738] T b6a8a8da56274eaa8fef29fff38a7dae P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.373992 14809 consensus_queue.cc:260] T b6a8a8da56274eaa8fef29fff38a7dae P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.374229 14809 ts_tablet_manager.cc:1428] T b6a8a8da56274eaa8fef29fff38a7dae P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.374370 14809 tablet_bootstrap.cc:492] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.375125 14809 tablet_bootstrap.cc:654] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.376039 14809 tablet_bootstrap.cc:492] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.376117 14809 ts_tablet_manager.cc:1397] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:13.376531 14809 raft_consensus.cc:357] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.376624 14809 raft_consensus.cc:383] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.376657 14809 raft_consensus.cc:738] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.376745 14809 consensus_queue.cc:260] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.377197 14809 ts_tablet_manager.cc:1428] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.377353 14809 tablet_bootstrap.cc:492] T fc056b9dbc374d26bf3feaf524f4d834 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.377663 14853 raft_consensus.cc:491] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.377746 14853 raft_consensus.cc:513] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.378055 14853 leader_election.cc:290] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.378106 14809 tablet_bootstrap.cc:654] T fc056b9dbc374d26bf3feaf524f4d834 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:13.378752 14493 leader_election.cc:343] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: b6a8a8da56274eaa8fef29fff38a7dae
I20250115 03:06:13.378975 14809 tablet_bootstrap.cc:492] T fc056b9dbc374d26bf3feaf524f4d834 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.379047 14809 ts_tablet_manager.cc:1397] T fc056b9dbc374d26bf3feaf524f4d834 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:13.379470 14809 raft_consensus.cc:357] T fc056b9dbc374d26bf3feaf524f4d834 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.379567 14809 raft_consensus.cc:383] T fc056b9dbc374d26bf3feaf524f4d834 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.379603 14809 raft_consensus.cc:738] T fc056b9dbc374d26bf3feaf524f4d834 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.379712 14809 consensus_queue.cc:260] T fc056b9dbc374d26bf3feaf524f4d834 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.379956 14809 ts_tablet_manager.cc:1428] T fc056b9dbc374d26bf3feaf524f4d834 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.380111 14809 tablet_bootstrap.cc:492] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.381104 14809 tablet_bootstrap.cc:654] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:13.381635 14156 catalog_manager.cc:4670] Async tablet task AlterTable RPC for tablet 83ac7b09047849da8f6770126c8dc033 (table default.Table_ee5faa8cd90f40199372322d90c5a204 [id=ff4681cab8ce44ceb45ed76c70c1132e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 83ac7b09047849da8f6770126c8dc033 consensus state has no leader
I20250115 03:06:13.381760 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6a8a8da56274eaa8fef29fff38a7dae" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:13.381858 14420 raft_consensus.cc:2463] T b6a8a8da56274eaa8fef29fff38a7dae P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:13.381904 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d0dde0a873df401c9502d832495fb2b1" } new_table_name: "default.Table_db8a2673977247e0b14b7995f7e09661" modify_external_catalogs: true
I20250115 03:06:13.382110 14493 leader_election.cc:304] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770
I20250115 03:06:13.382118 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet 108aafc0c034498bbd300be9c6b1f8c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.382320 14853 raft_consensus.cc:2798] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.382370 14853 raft_consensus.cc:491] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.382416 14853 raft_consensus.cc:3054] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.383301 14853 raft_consensus.cc:513] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.383479 14809 tablet_bootstrap.cc:492] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.383558 14809 ts_tablet_manager.cc:1397] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:13.383633 14853 leader_election.cc:290] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.383798 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 9c54a5df35e7422b84acc91e3a10aac3 (table default.Table_f957fe3487ec4546a46a8392108003f0 [id=fb3a764b51e74e08a281d128e21466e3]) successfully deleted
I20250115 03:06:13.383953 14809 raft_consensus.cc:357] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.384043 14809 raft_consensus.cc:383] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.384074 14809 raft_consensus.cc:738] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.384173 14809 consensus_queue.cc:260] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.384406 14809 ts_tablet_manager.cc:1428] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.384562 14809 tablet_bootstrap.cc:492] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.384639 14819 tablet_replica.cc:331] T 99abc1c7a20a4475921597f93c2db56c P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.384788 14819 raft_consensus.cc:2238] T 99abc1c7a20a4475921597f93c2db56c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
W20250115 03:06:13.384975 14493 leader_election.cc:343] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: b6a8a8da56274eaa8fef29fff38a7dae
I20250115 03:06:13.385434 14809 tablet_bootstrap.cc:654] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.385538 14819 raft_consensus.cc:2267] T 99abc1c7a20a4475921597f93c2db56c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.385969 14819 ts_tablet_manager.cc:1905] T 99abc1c7a20a4475921597f93c2db56c P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.386092 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6a8a8da56274eaa8fef29fff38a7dae" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:13.386185 14420 raft_consensus.cc:3054] T b6a8a8da56274eaa8fef29fff38a7dae P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.387004 14420 raft_consensus.cc:2463] T b6a8a8da56274eaa8fef29fff38a7dae P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:13.387667 14809 tablet_bootstrap.cc:492] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.388280 14809 ts_tablet_manager.cc:1397] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250115 03:06:13.388674 14809 raft_consensus.cc:357] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.388760 14809 raft_consensus.cc:383] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.387835 14493 leader_election.cc:304] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770
I20250115 03:06:13.388792 14809 raft_consensus.cc:738] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.388888 14809 consensus_queue.cc:260] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.388965 14853 raft_consensus.cc:2798] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.389045 14853 raft_consensus.cc:695] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:13.389160 14809 ts_tablet_manager.cc:1428] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.389147 14853 consensus_queue.cc:237] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.389293 14809 tablet_bootstrap.cc:492] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.390127 14809 tablet_bootstrap.cc:654] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.393299 14969 tablet.cc:1721] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:13.394511 14166 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_43170518d7504d1d8d17374e281f2728" } modify_external_catalogs: true
I20250115 03:06:13.394604 14166 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_43170518d7504d1d8d17374e281f2728" } modify_external_catalogs: true
I20250115 03:06:13.396234 14809 tablet_bootstrap.cc:492] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.396317 14809 ts_tablet_manager.cc:1397] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:13.396833 14809 raft_consensus.cc:357] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.396950 14809 raft_consensus.cc:383] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.396986 14809 raft_consensus.cc:738] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.397085 14809 consensus_queue.cc:260] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.397318 14809 ts_tablet_manager.cc:1428] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:13.397470 14809 tablet_bootstrap.cc:492] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.398257 14809 tablet_bootstrap.cc:654] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.398582 14891 tablet.cc:1721] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:13.400978 14809 tablet_bootstrap.cc:492] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.401054 14809 ts_tablet_manager.cc:1397] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20250115 03:06:13.401443 14809 raft_consensus.cc:357] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.401543 14809 raft_consensus.cc:383] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.401572 14809 raft_consensus.cc:738] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.401665 14809 consensus_queue.cc:260] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.401896 14809 ts_tablet_manager.cc:1428] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.402063 14809 tablet_bootstrap.cc:492] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.402673 14853 consensus_queue.cc:1035] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:13.402882 14809 tablet_bootstrap.cc:654] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.403556 14861 raft_consensus.cc:491] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.403633 14861 raft_consensus.cc:513] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.403940 14861 leader_election.cc:290] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.404278 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e78c18fd98f44bcbd08b747f2a0ffd0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:13.404390 14691 raft_consensus.cc:2463] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:13.404807 14358 leader_election.cc:304] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:13.404994 14861 raft_consensus.cc:2798] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.405041 14861 raft_consensus.cc:491] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.405074 14861 raft_consensus.cc:3054] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.405861 14861 raft_consensus.cc:513] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.406141 14861 leader_election.cc:290] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.406548 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e78c18fd98f44bcbd08b747f2a0ffd0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:13.406651 14691 raft_consensus.cc:3054] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.407007 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e78c18fd98f44bcbd08b747f2a0ffd0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:13.407310 14356 leader_election.cc:343] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.407298 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e78c18fd98f44bcbd08b747f2a0ffd0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.407336 14820 tablet_replica.cc:331] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.407447 14691 raft_consensus.cc:2463] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:13.407464 14820 raft_consensus.cc:2238] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:13.407598 14820 pending_rounds.cc:62] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1: Trying to abort 1 pending ops.
W20250115 03:06:13.407613 14356 leader_election.cc:343] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.407625 14820 pending_rounds.cc:69] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384887932149760 op_type: NO_OP noop_request { }
I20250115 03:06:13.407672 14820 raft_consensus.cc:2883] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:13.407720 14820 raft_consensus.cc:2267] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.407809 14358 leader_election.cc:304] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:13.407999 15276 raft_consensus.cc:2798] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.408120 15276 raft_consensus.cc:695] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:13.408248 15276 consensus_queue.cc:237] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.409168 14819 ts_tablet_manager.cc:1918] T 99abc1c7a20a4475921597f93c2db56c P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.409230 14819 log.cc:1198] T 99abc1c7a20a4475921597f93c2db56c P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/99abc1c7a20a4475921597f93c2db56c
I20250115 03:06:13.409417 14819 ts_tablet_manager.cc:1939] T 99abc1c7a20a4475921597f93c2db56c P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.410722 14812 tablet_bootstrap.cc:492] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.410796 14812 ts_tablet_manager.cc:1397] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.043s user 0.000s sys 0.002s
I20250115 03:06:13.410665 15151 consensus_queue.cc:1035] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:13.411227 14812 raft_consensus.cc:357] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.411334 14812 raft_consensus.cc:383] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.411363 14812 raft_consensus.cc:738] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.411473 14812 consensus_queue.cc:260] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.411742 14812 ts_tablet_manager.cc:1428] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:13.411743 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 99abc1c7a20a4475921597f93c2db56c (table default.Table_35d1dd748a6f4e79adced77fe7b2d08d [id=f94088b0776f4be990ddcec04afbca43]) successfully deleted
I20250115 03:06:13.411895 14812 tablet_bootstrap.cc:492] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.411926 14819 tablet_replica.cc:331] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.412014 14819 raft_consensus.cc:2238] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:13.412158 14819 raft_consensus.cc:2267] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.412598 14819 ts_tablet_manager.cc:1905] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.412685 14812 tablet_bootstrap.cc:654] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.417129 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e0faf1e67748430cb2b830441fd58344"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.417452 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e0faf1e67748430cb2b830441fd58344 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.421974 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 9aa9dbbbb8c548a9b3943f0305cb6e9f
I20250115 03:06:13.422546 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aa9dbbbb8c548a9b3943f0305cb6e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.422811 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aa9dbbbb8c548a9b3943f0305cb6e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.423336 14820 ts_tablet_manager.cc:1905] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.423342 15276 raft_consensus.cc:491] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.423413 15276 raft_consensus.cc:513] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.423746 15276 leader_election.cc:290] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.424153 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd2c39e3cf14fb8bcf2db6b10fcbb35" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.424261 14554 raft_consensus.cc:2463] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:13.424516 14356 leader_election.cc:304] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:13.424657 15276 raft_consensus.cc:2798] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.424705 15276 raft_consensus.cc:491] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.424736 15276 raft_consensus.cc:3054] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.425415 14820 ts_tablet_manager.cc:1918] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:13.425475 14820 log.cc:1198] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/7479bfc96d634011ba15069b5cf6794c
I20250115 03:06:13.425554 14819 ts_tablet_manager.cc:1918] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20250115 03:06:13.425554 14170 master_service.cc:946] ReplaceTablet: received request to replace tablet 876e28105fbb47bdb722e5d3f8467341 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:13.425616 14819 log.cc:1198] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/c6cd63ac1c084a0d8f166e63b0ed85bc
I20250115 03:06:13.425585 15276 raft_consensus.cc:513] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.425694 14820 ts_tablet_manager.cc:1939] T 7479bfc96d634011ba15069b5cf6794c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.425865 14819 ts_tablet_manager.cc:1939] T c6cd63ac1c084a0d8f166e63b0ed85bc P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.425855 14170 master_service.cc:946] ReplaceTablet: received request to replace tablet c6cd63ac1c084a0d8f166e63b0ed85bc from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:13.425897 15276 leader_election.cc:290] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.426105 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet 551d5ffdcd2d43fcb63e42c135fc2bd5 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:13.426205 14820 tablet_replica.cc:331] T b41d9a854bac49599d3390c15f31e0fa P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.426323 14820 raft_consensus.cc:2238] T b41d9a854bac49599d3390c15f31e0fa P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.426419 14820 raft_consensus.cc:2267] T b41d9a854bac49599d3390c15f31e0fa P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.426569 14819 tablet_replica.cc:331] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.426676 14819 raft_consensus.cc:2238] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.426767 14819 raft_consensus.cc:2267] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.426748 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d4a2862d217743a181c6139aeeda1f1a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.427045 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d4a2862d217743a181c6139aeeda1f1a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.427668 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet c6cd63ac1c084a0d8f166e63b0ed85bc (table default.Table_3bd81cb8bc2d4b74ae3dbcbdb74aae8b [id=05f9fd2ebec34d2299ea59c47a74c828]) successfully deleted
I20250115 03:06:13.427721 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aa9dbbbb8c548a9b3943f0305cb6e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.428247 14811 tablet_bootstrap.cc:492] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
W20250115 03:06:13.428256 14358 leader_election.cc:343] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 6fd2c39e3cf14fb8bcf2db6b10fcbb35
I20250115 03:06:13.428321 14811 ts_tablet_manager.cc:1397] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.064s user 0.002s sys 0.000s
W20250115 03:06:13.428431 14358 leader_election.cc:343] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 6fd2c39e3cf14fb8bcf2db6b10fcbb35
I20250115 03:06:13.428576 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 7479bfc96d634011ba15069b5cf6794c (table default.Table_dffdb65871dc41b0ba48073022408925 [id=f8ea91c4841b470db1b9f05fa47244fc]) successfully deleted
I20250115 03:06:13.428680 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fd2c39e3cf14fb8bcf2db6b10fcbb35" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:13.428712 14811 raft_consensus.cc:357] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.428789 14556 raft_consensus.cc:3054] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.428812 14811 raft_consensus.cc:383] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.428843 14811 raft_consensus.cc:738] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.428966 14811 consensus_queue.cc:260] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.429170 14819 ts_tablet_manager.cc:1905] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.429216 14811 ts_tablet_manager.cc:1428] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.429375 14811 tablet_bootstrap.cc:492] T 28324608e3764b7e8a550bc71a4bad58 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.430193 14811 tablet_bootstrap.cc:654] T 28324608e3764b7e8a550bc71a4bad58 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:13.430655 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 9aa9dbbbb8c548a9b3943f0305cb6e9f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9aa9dbbbb8c548a9b3943f0305cb6e9f already in progress: creating tablet
I20250115 03:06:13.431520 14556 raft_consensus.cc:2463] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:13.431665 14819 ts_tablet_manager.cc:1918] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:13.431720 14819 log.cc:1198] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/35b90c4b0ef341c8a7c1bc517a1231ed
I20250115 03:06:13.431844 14356 leader_election.cc:304] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770
I20250115 03:06:13.431959 14819 ts_tablet_manager.cc:1939] T 35b90c4b0ef341c8a7c1bc517a1231ed P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.432042 15276 raft_consensus.cc:2798] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.432121 15276 raft_consensus.cc:695] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:13.432221 15276 consensus_queue.cc:237] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.432515 14819 tablet_replica.cc:331] T 07d9c5e89ac34af29a406e79d2ace824 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.432514 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 35b90c4b0ef341c8a7c1bc517a1231ed (table default.Table_65b9d218ca9743d7a7eaf7cec1e16c80 [id=1b4a3ef0246a4341869bf5163487cac6]) successfully deleted
I20250115 03:06:13.432621 14819 raft_consensus.cc:2238] T 07d9c5e89ac34af29a406e79d2ace824 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.432708 14819 raft_consensus.cc:2267] T 07d9c5e89ac34af29a406e79d2ace824 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.434818 14819 ts_tablet_manager.cc:1905] T 07d9c5e89ac34af29a406e79d2ace824 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.435493 15157 tablet.cc:1721] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:13.435871 14811 tablet_bootstrap.cc:492] T 28324608e3764b7e8a550bc71a4bad58 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.435949 14811 ts_tablet_manager.cc:1397] T 28324608e3764b7e8a550bc71a4bad58 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:13.436349 14811 raft_consensus.cc:357] T 28324608e3764b7e8a550bc71a4bad58 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.436452 14811 raft_consensus.cc:383] T 28324608e3764b7e8a550bc71a4bad58 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.436484 14811 raft_consensus.cc:738] T 28324608e3764b7e8a550bc71a4bad58 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.436594 14811 consensus_queue.cc:260] T 28324608e3764b7e8a550bc71a4bad58 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.436941 14811 ts_tablet_manager.cc:1428] T 28324608e3764b7e8a550bc71a4bad58 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.437098 14811 tablet_bootstrap.cc:492] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.438565 14811 tablet_bootstrap.cc:654] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.439208 14820 ts_tablet_manager.cc:1905] T b41d9a854bac49599d3390c15f31e0fa P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.440104 14809 tablet_bootstrap.cc:492] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.440165 14819 ts_tablet_manager.cc:1918] T 07d9c5e89ac34af29a406e79d2ace824 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.440243 14819 log.cc:1198] T 07d9c5e89ac34af29a406e79d2ace824 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/07d9c5e89ac34af29a406e79d2ace824
I20250115 03:06:13.440179 14809 ts_tablet_manager.cc:1397] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.000s
I20250115 03:06:13.440418 14819 ts_tablet_manager.cc:1939] T 07d9c5e89ac34af29a406e79d2ace824 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.440677 14809 raft_consensus.cc:357] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.440778 14809 raft_consensus.cc:383] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.440809 14809 raft_consensus.cc:738] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.440913 14809 consensus_queue.cc:260] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.442242 14809 ts_tablet_manager.cc:1428] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:13.442412 14809 tablet_bootstrap.cc:492] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.442741 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 07d9c5e89ac34af29a406e79d2ace824 (table default.Table_64c4e8668b494feca761d65318286040 [id=265c248361624c3f9e5ece27144d8e04]) successfully deleted
I20250115 03:06:13.442926 14819 tablet_replica.cc:331] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.443027 14819 raft_consensus.cc:2238] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:13.443178 14819 raft_consensus.cc:2267] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.443413 14809 tablet_bootstrap.cc:654] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.443953 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 2fcce3505b744f668c5355aa982c8485 (DEFAULT_TABLE table=default.Table_2af92748648345c0903a0f99da0d47c5 [id=19efb3734d824699a088ad62e69095f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.444164 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fcce3505b744f668c5355aa982c8485. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.444554 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 338592cc70ad4e61ba90905a424a3871 (table default.Table_ee5faa8cd90f40199372322d90c5a204 [id=ff4681cab8ce44ceb45ed76c70c1132e]). Expected version 1 got 0
I20250115 03:06:13.446653 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 2fcce3505b744f668c5355aa982c8485 (DEFAULT_TABLE table=default.Table_2af92748648345c0903a0f99da0d47c5 [id=19efb3734d824699a088ad62e69095f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.446893 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fcce3505b744f668c5355aa982c8485. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.447985 14820 ts_tablet_manager.cc:1918] T b41d9a854bac49599d3390c15f31e0fa P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:13.448042 14820 log.cc:1198] T b41d9a854bac49599d3390c15f31e0fa P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/b41d9a854bac49599d3390c15f31e0fa
I20250115 03:06:13.448241 14820 ts_tablet_manager.cc:1939] T b41d9a854bac49599d3390c15f31e0fa P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.448526 14809 tablet_bootstrap.cc:492] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.448593 14809 ts_tablet_manager.cc:1397] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20250115 03:06:13.449002 14812 tablet_bootstrap.cc:492] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.449025 14809 raft_consensus.cc:357] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.449086 14812 ts_tablet_manager.cc:1397] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.037s user 0.000s sys 0.002s
I20250115 03:06:13.449111 14809 raft_consensus.cc:383] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.449141 14809 raft_consensus.cc:738] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.449242 14809 consensus_queue.cc:260] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.449499 14809 ts_tablet_manager.cc:1428] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20250115 03:06:13.449502 14812 raft_consensus.cc:357] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.449597 14812 raft_consensus.cc:383] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.449628 14812 raft_consensus.cc:738] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.449649 14809 tablet_bootstrap.cc:492] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.449734 14812 consensus_queue.cc:260] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.450026 14812 ts_tablet_manager.cc:1428] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:13.450100 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 39245099377549dd81238eabd690b6da (DEFAULT_TABLE table=default.Table_aeb59441e2604aa189d7b95ea5e98ed2 [id=c121e0ff11a141d6b1c10f66e38b1006]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.450189 14812 tablet_bootstrap.cc:492] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.450294 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39245099377549dd81238eabd690b6da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.450471 14809 tablet_bootstrap.cc:654] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.451155 14812 tablet_bootstrap.cc:654] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.451851 14396 tablet_service.cc:1467] Processing CreateTablet for tablet bcf6745e549d47cdbdf97c2dff6997ad (DEFAULT_TABLE table=default.Table_28b265c831a944c7bff1a3ee55a86c64 [id=65f79614672344d88c2ec541c43da990]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.452059 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf6745e549d47cdbdf97c2dff6997ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.452693 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 0580a8f7b2984162b9275d74334b1905 (DEFAULT_TABLE table=default.Table_06ad841d9f6e4ae9af7639a562f42036 [id=0790b08615854d3688f940aba20b6520]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.452864 14669 tablet_service.cc:1467] Processing CreateTablet for tablet dca04d171deb488690ea58bd75995021 (DEFAULT_TABLE table=default.Table_3bd81cb8bc2d4b74ae3dbcbdb74aae8b [id=05f9fd2ebec34d2299ea59c47a74c828]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.452944 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0580a8f7b2984162b9275d74334b1905. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.453069 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dca04d171deb488690ea58bd75995021. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.453698 14400 tablet_service.cc:1467] Processing CreateTablet for tablet bda0e0238da246feb36450f40dd061e5 (DEFAULT_TABLE table=default.Table_884bdba667784d59bc4fa65b5b437fcb [id=111fc67094164d3f99965c246ae12b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.453889 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bda0e0238da246feb36450f40dd061e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.454119 14399 tablet_service.cc:1467] Processing CreateTablet for tablet dca04d171deb488690ea58bd75995021 (DEFAULT_TABLE table=default.Table_3bd81cb8bc2d4b74ae3dbcbdb74aae8b [id=05f9fd2ebec34d2299ea59c47a74c828]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.454300 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dca04d171deb488690ea58bd75995021. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.454787 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 6629f2415b554c1c95927c721532fcfc (DEFAULT_TABLE table=default.Table_35d1dd748a6f4e79adced77fe7b2d08d [id=f94088b0776f4be990ddcec04afbca43]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.454989 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6629f2415b554c1c95927c721532fcfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.455106 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 39245099377549dd81238eabd690b6da (DEFAULT_TABLE table=default.Table_aeb59441e2604aa189d7b95ea5e98ed2 [id=c121e0ff11a141d6b1c10f66e38b1006]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.455278 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39245099377549dd81238eabd690b6da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.456045 14401 tablet_service.cc:1467] Processing CreateTablet for tablet ef002211ee1c4a149232c8e433277add (DEFAULT_TABLE table=default.Table_f957fe3487ec4546a46a8392108003f0 [id=fb3a764b51e74e08a281d128e21466e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.456187 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 0580a8f7b2984162b9275d74334b1905 (DEFAULT_TABLE table=default.Table_06ad841d9f6e4ae9af7639a562f42036 [id=0790b08615854d3688f940aba20b6520]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.456238 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef002211ee1c4a149232c8e433277add. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.456353 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0580a8f7b2984162b9275d74334b1905. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.456959 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet b41d9a854bac49599d3390c15f31e0fa (table default.Table_0bdf66c6040b4709b0a6be57188b9d86 [id=c8fdb0f9f9064f3eb8d19f4adf3c439f]) successfully deleted
I20250115 03:06:13.457057 14672 tablet_service.cc:1467] Processing CreateTablet for tablet ef002211ee1c4a149232c8e433277add (DEFAULT_TABLE table=default.Table_f957fe3487ec4546a46a8392108003f0 [id=fb3a764b51e74e08a281d128e21466e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.457262 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef002211ee1c4a149232c8e433277add. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.457451 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_bf26dc2911ac4ad496564d8a941c0d9f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.457762 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bf26dc2911ac4ad496564d8a941c0d9f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.458245 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 6629f2415b554c1c95927c721532fcfc (DEFAULT_TABLE table=default.Table_35d1dd748a6f4e79adced77fe7b2d08d [id=f94088b0776f4be990ddcec04afbca43]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.458437 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6629f2415b554c1c95927c721532fcfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.458477 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f6b447ccf7e4426484d99557dd1591d7" } modify_external_catalogs: true
I20250115 03:06:13.458568 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f6b447ccf7e4426484d99557dd1591d7" } modify_external_catalogs: true
I20250115 03:06:13.459225 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_7e53fa73287c43e3a9a0e65908b42593"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.459510 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e53fa73287c43e3a9a0e65908b42593 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.459647 14660 tablet_service.cc:1467] Processing CreateTablet for tablet bda0e0238da246feb36450f40dd061e5 (DEFAULT_TABLE table=default.Table_884bdba667784d59bc4fa65b5b437fcb [id=111fc67094164d3f99965c246ae12b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.459846 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bda0e0238da246feb36450f40dd061e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.452697 14663 tablet_service.cc:1467] Processing CreateTablet for tablet bcf6745e549d47cdbdf97c2dff6997ad (DEFAULT_TABLE table=default.Table_28b265c831a944c7bff1a3ee55a86c64 [id=65f79614672344d88c2ec541c43da990]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.460693 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf6745e549d47cdbdf97c2dff6997ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.460947 14853 raft_consensus.cc:491] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.461035 14853 raft_consensus.cc:513] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.461378 14853 leader_election.cc:290] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.461805 14532 tablet_service.cc:1467] Processing CreateTablet for tablet 2fcce3505b744f668c5355aa982c8485 (DEFAULT_TABLE table=default.Table_2af92748648345c0903a0f99da0d47c5 [id=19efb3734d824699a088ad62e69095f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.461897 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044651a15c8346a6a0d11713a82a3641" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:13.462038 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fcce3505b744f668c5355aa982c8485. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.462289 14536 tablet_service.cc:1467] Processing CreateTablet for tablet dca04d171deb488690ea58bd75995021 (DEFAULT_TABLE table=default.Table_3bd81cb8bc2d4b74ae3dbcbdb74aae8b [id=05f9fd2ebec34d2299ea59c47a74c828]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.462496 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dca04d171deb488690ea58bd75995021. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.464030 14537 tablet_service.cc:1467] Processing CreateTablet for tablet bda0e0238da246feb36450f40dd061e5 (DEFAULT_TABLE table=default.Table_884bdba667784d59bc4fa65b5b437fcb [id=111fc67094164d3f99965c246ae12b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.464102 14820 tablet_replica.cc:331] T 8a75594237ce49edbc5ba410038bbecd P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.464215 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bda0e0238da246feb36450f40dd061e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.464394 14529 tablet_service.cc:1467] Processing CreateTablet for tablet 39245099377549dd81238eabd690b6da (DEFAULT_TABLE table=default.Table_aeb59441e2604aa189d7b95ea5e98ed2 [id=c121e0ff11a141d6b1c10f66e38b1006]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.464594 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39245099377549dd81238eabd690b6da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.464744 14811 tablet_bootstrap.cc:492] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.464823 14811 ts_tablet_manager.cc:1397] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.028s user 0.001s sys 0.000s
I20250115 03:06:13.465107 14527 tablet_service.cc:1467] Processing CreateTablet for tablet bcf6745e549d47cdbdf97c2dff6997ad (DEFAULT_TABLE table=default.Table_28b265c831a944c7bff1a3ee55a86c64 [id=65f79614672344d88c2ec541c43da990]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.465299 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcf6745e549d47cdbdf97c2dff6997ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.465350 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 6629f2415b554c1c95927c721532fcfc (DEFAULT_TABLE table=default.Table_35d1dd748a6f4e79adced77fe7b2d08d [id=f94088b0776f4be990ddcec04afbca43]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.465498 14811 raft_consensus.cc:357] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.465574 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6629f2415b554c1c95927c721532fcfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.465584 14811 raft_consensus.cc:383] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.465643 14811 raft_consensus.cc:738] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.465765 14811 consensus_queue.cc:260] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.467088 14528 tablet_service.cc:1467] Processing CreateTablet for tablet 0580a8f7b2984162b9275d74334b1905 (DEFAULT_TABLE table=default.Table_06ad841d9f6e4ae9af7639a562f42036 [id=0790b08615854d3688f940aba20b6520]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.467128 14535 tablet_service.cc:1467] Processing CreateTablet for tablet ef002211ee1c4a149232c8e433277add (DEFAULT_TABLE table=default.Table_f957fe3487ec4546a46a8392108003f0 [id=fb3a764b51e74e08a281d128e21466e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.467267 14811 ts_tablet_manager.cc:1428] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:13.467506 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "044651a15c8346a6a0d11713a82a3641" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:13.467576 14493 leader_election.cc:343] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:13.467830 14493 leader_election.cc:343] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.467864 14493 leader_election.cc:304] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:13.468043 14853 raft_consensus.cc:2743] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:13.467422 14811 tablet_bootstrap.cc:492] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.467284 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0580a8f7b2984162b9275d74334b1905. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.469293 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef002211ee1c4a149232c8e433277add. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.471941 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 551d5ffdcd2d43fcb63e42c135fc2bd5
I20250115 03:06:13.472201 14168 catalog_manager.cc:6596] ReplaceTablet: tablet 551d5ffdcd2d43fcb63e42c135fc2bd5 deleted and replaced by tablet 4a56ae8a3e3e4f57b13e762cef21f753
I20250115 03:06:13.472419 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_929dc98c434d495da9e1e46de64f5ef9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.472720 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_929dc98c434d495da9e1e46de64f5ef9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.473258 14811 tablet_bootstrap.cc:654] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.474839 14171 catalog_manager.cc:5526] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:13.474964 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet afa7cecc384f495c9ff4723f553acc24
I20250115 03:06:13.475092 14171 catalog_manager.cc:5526] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:13.475235 14171 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 20f86f93812849c4b7852a884f677c04 (table default.Table_b44636b0d90d4972aceaa7d169a3a477 [id=f8f2a7ff6fdc40468de6ae9ba28c1427]). Expected version 2 got 0
I20250115 03:06:13.475284 14165 master_service.cc:946] ReplaceTablet: received request to replace tablet c6cd63ac1c084a0d8f166e63b0ed85bc from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:13.475345 14171 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 9adb11e56c394461a91c924dc1eb404b (table default.Table_ea2592c9eb974e6ab84a4e2f799b5cae [id=d0fc70d8af9944099f8e7ee9a416399d]). Expected version 1 got 0
I20250115 03:06:13.475484 14171 catalog_manager.cc:5526] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:13.475620 14171 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet d4f76d0b3a394a5f8aa33ad17b11da34 (table default.Table_0dd17013f6334efe97618f0b520512c5 [id=488a9a6268f64669ae17e1a67caa73a5]). Expected version 2 got 0
I20250115 03:06:13.475744 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 551d5ffdcd2d43fcb63e42c135fc2bd5 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a56ae8a3e3e4f57b13e762cef21f753) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.464203 14820 raft_consensus.cc:2238] T 8a75594237ce49edbc5ba410038bbecd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.475888 14823 tablet_replica.cc:331] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.475958 14820 raft_consensus.cc:2267] T 8a75594237ce49edbc5ba410038bbecd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.475991 14823 raft_consensus.cc:2238] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.476073 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 9ffc6d1d09e04cc99a5e7a1e1af7bef8
I20250115 03:06:13.476083 14823 raft_consensus.cc:2267] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.476604 14823 ts_tablet_manager.cc:1905] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.476637 14162 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_ee5faa8cd90f40199372322d90c5a204 [id=ff4681cab8ce44ceb45ed76c70c1132e] alter complete (version 1)
I20250115 03:06:13.476861 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ffc6d1d09e04cc99a5e7a1e1af7bef8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.477015 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 551d5ffdcd2d43fcb63e42c135fc2bd5 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a56ae8a3e3e4f57b13e762cef21f753) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.477385 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 044651a15c8346a6a0d11713a82a3641
I20250115 03:06:13.477826 14812 tablet_bootstrap.cc:492] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.477900 14812 ts_tablet_manager.cc:1397] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.002s
I20250115 03:06:13.478305 14812 raft_consensus.cc:357] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.478394 14812 raft_consensus.cc:383] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.478421 14812 raft_consensus.cc:738] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.478494 14861 raft_consensus.cc:491] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.478523 14812 consensus_queue.cc:260] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.478571 14861 raft_consensus.cc:513] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.478679 14823 ts_tablet_manager.cc:1918] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.478741 14823 log.cc:1198] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/551d5ffdcd2d43fcb63e42c135fc2bd5
I20250115 03:06:13.478771 14812 ts_tablet_manager.cc:1428] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.478879 14861 leader_election.cc:290] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.478929 14823 ts_tablet_manager.cc:1939] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.478942 14812 tablet_bootstrap.cc:492] T 7b13f8b40c2f40b8a93ce8030845665c P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.479257 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa9dbbbb8c548a9b3943f0305cb6e9f" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.479630 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9aa9dbbbb8c548a9b3943f0305cb6e9f" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:13.479813 14812 tablet_bootstrap.cc:654] T 7b13f8b40c2f40b8a93ce8030845665c P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:13.479930 14356 leader_election.cc:343] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:13.480027 14358 leader_election.cc:343] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.480068 14358 leader_election.cc:304] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:13.480243 14861 raft_consensus.cc:2743] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:13.480518 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet afa7cecc384f495c9ff4723f553acc24 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.480643 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ffc6d1d09e04cc99a5e7a1e1af7bef8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.481729 14812 tablet_bootstrap.cc:492] T 7b13f8b40c2f40b8a93ce8030845665c P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.481814 14812 ts_tablet_manager.cc:1397] T 7b13f8b40c2f40b8a93ce8030845665c P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:13.481987 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet afa7cecc384f495c9ff4723f553acc24 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.482214 14812 raft_consensus.cc:357] T 7b13f8b40c2f40b8a93ce8030845665c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.482311 14812 raft_consensus.cc:383] T 7b13f8b40c2f40b8a93ce8030845665c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.482342 14812 raft_consensus.cc:738] T 7b13f8b40c2f40b8a93ce8030845665c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.482442 14812 consensus_queue.cc:260] T 7b13f8b40c2f40b8a93ce8030845665c P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.482672 14812 ts_tablet_manager.cc:1428] T 7b13f8b40c2f40b8a93ce8030845665c P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.482824 14812 tablet_bootstrap.cc:492] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.482893 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ffc6d1d09e04cc99a5e7a1e1af7bef8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.483021 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet afa7cecc384f495c9ff4723f553acc24 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.483253 14823 tablet_replica.cc:331] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.483364 14823 raft_consensus.cc:2238] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:13.483538 14823 raft_consensus.cc:2267] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.483728 14812 tablet_bootstrap.cc:654] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.483830 14823 ts_tablet_manager.cc:1905] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.483937 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 044651a15c8346a6a0d11713a82a3641 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.484277 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 551d5ffdcd2d43fcb63e42c135fc2bd5 (table default.Table_05065fcbba614582b828f0b729c8c07c [id=1b7d3a80c5424e5692c2e6c556919342]) successfully deleted
I20250115 03:06:13.485170 14812 tablet_bootstrap.cc:492] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.485301 14812 ts_tablet_manager.cc:1397] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20250115 03:06:13.485563 14823 ts_tablet_manager.cc:1918] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:13.485594 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 044651a15c8346a6a0d11713a82a3641 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.485631 14823 log.cc:1198] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/afa7cecc384f495c9ff4723f553acc24
I20250115 03:06:13.485857 14823 ts_tablet_manager.cc:1939] T afa7cecc384f495c9ff4723f553acc24 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.485949 14812 raft_consensus.cc:357] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.486024 14812 raft_consensus.cc:383] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.486052 14812 raft_consensus.cc:738] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.486174 14812 consensus_queue.cc:260] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.486320 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 551d5ffdcd2d43fcb63e42c135fc2bd5 with delete_type TABLET_DATA_DELETED (replaced by tablet 4a56ae8a3e3e4f57b13e762cef21f753) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.486413 14812 ts_tablet_manager.cc:1428] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:13.486461 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 044651a15c8346a6a0d11713a82a3641 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.486557 14812 tablet_bootstrap.cc:492] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:13.487282 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 9ffc6d1d09e04cc99a5e7a1e1af7bef8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9ffc6d1d09e04cc99a5e7a1e1af7bef8 already in progress: creating tablet
I20250115 03:06:13.487361 14812 tablet_bootstrap.cc:654] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.488395 15274 tablet.cc:1721] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:13.488445 14819 ts_tablet_manager.cc:1905] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.489462 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet afa7cecc384f495c9ff4723f553acc24 (table default.Table_f6b447ccf7e4426484d99557dd1591d7 [id=f4597c64de65413a9885be2bb4301d44]) successfully deleted
W20250115 03:06:13.489719 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 044651a15c8346a6a0d11713a82a3641 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 044651a15c8346a6a0d11713a82a3641 already in progress: creating tablet
I20250115 03:06:13.490638 14820 ts_tablet_manager.cc:1905] T 8a75594237ce49edbc5ba410038bbecd P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.492203 14969 tablet.cc:1721] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:13.492799 14820 ts_tablet_manager.cc:1918] T 8a75594237ce49edbc5ba410038bbecd P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.492835 14819 ts_tablet_manager.cc:1918] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:13.492866 14820 log.cc:1198] T 8a75594237ce49edbc5ba410038bbecd P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/8a75594237ce49edbc5ba410038bbecd
I20250115 03:06:13.492890 14819 log.cc:1198] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/108aafc0c034498bbd300be9c6b1f8c4
I20250115 03:06:13.493114 14820 ts_tablet_manager.cc:1939] T 8a75594237ce49edbc5ba410038bbecd P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.493160 14819 ts_tablet_manager.cc:1939] T 108aafc0c034498bbd300be9c6b1f8c4 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.493626 14820 tablet_replica.cc:331] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.493714 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 108aafc0c034498bbd300be9c6b1f8c4 (table default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd]) successfully deleted
I20250115 03:06:13.493741 14820 raft_consensus.cc:2238] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:13.493886 14820 pending_rounds.cc:62] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1: Trying to abort 1 pending ops.
I20250115 03:06:13.493916 14820 pending_rounds.cc:69] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384887191818240 op_type: NO_OP noop_request { }
I20250115 03:06:13.493963 14820 raft_consensus.cc:2883] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:13.494010 14820 raft_consensus.cc:2267] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.494344 14811 tablet_bootstrap.cc:492] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.494412 14811 ts_tablet_manager.cc:1397] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20250115 03:06:13.494830 14811 raft_consensus.cc:357] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.494936 15157 tablet.cc:1721] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:13.495451 14819 tablet_replica.cc:331] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.495584 14819 raft_consensus.cc:2238] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.495666 14819 raft_consensus.cc:2267] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.495715 14166 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet e4994733ae904c99986cabe4c6efc503 (table default.Table_db8a2673977247e0b14b7995f7e09661 [id=f7e4534c1f6f48f09e10b84c1d5c458c]). Expected version 1 got 0
I20250115 03:06:13.495805 14166 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet bdc2e882dc7e469e857d74e41ee2f435 (table default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]). Expected version 1 got 0
I20250115 03:06:13.495879 14166 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet f21761ff87ac43d5b9b47b5905fa5867 (table default.Table_53c9a73ae8ef459c987964efac552c96 [id=37d965d8a5eb408ea3f33313bd22c152]). Expected version 2 got 0
I20250115 03:06:13.496073 14819 ts_tablet_manager.cc:1905] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.494953 14811 raft_consensus.cc:383] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.497009 14811 raft_consensus.cc:738] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.494834 14820 ts_tablet_manager.cc:1905] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.497673 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet e4994733ae904c99986cabe4c6efc503 (table default.Table_db8a2673977247e0b14b7995f7e09661 [id=f7e4534c1f6f48f09e10b84c1d5c458c]). Expected version 1 got 0
I20250115 03:06:13.497838 14163 catalog_manager.cc:5526] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:13.498111 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet bdc2e882dc7e469e857d74e41ee2f435 (table default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]). Expected version 1 got 0
I20250115 03:06:13.498239 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet d4f76d0b3a394a5f8aa33ad17b11da34 (table default.Table_0dd17013f6334efe97618f0b520512c5 [id=488a9a6268f64669ae17e1a67caa73a5]). Expected version 2 got 0
I20250115 03:06:13.498229 14811 consensus_queue.cc:260] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.498692 14819 ts_tablet_manager.cc:1918] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.498762 14819 log.cc:1198] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/9aa9dbbbb8c548a9b3943f0305cb6e9f
I20250115 03:06:13.499792 14820 ts_tablet_manager.cc:1918] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:13.499898 14820 log.cc:1198] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/0ecd4741aacb45e2ae6404675f885125
I20250115 03:06:13.500185 14820 ts_tablet_manager.cc:1939] T 0ecd4741aacb45e2ae6404675f885125 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.499426 14819 ts_tablet_manager.cc:1939] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.500587 14861 raft_consensus.cc:491] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.500659 14861 raft_consensus.cc:513] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.500731 14811 ts_tablet_manager.cc:1428] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:13.500885 14820 tablet_replica.cc:331] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.501001 14861 leader_election.cc:290] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.501011 14820 raft_consensus.cc:2238] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:13.501142 14820 pending_rounds.cc:62] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1: Trying to abort 1 pending ops.
I20250115 03:06:13.501262 14820 pending_rounds.cc:69] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384888080281600 op_type: NO_OP noop_request { }
I20250115 03:06:13.501385 14820 raft_consensus.cc:2883] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:13.501451 14820 raft_consensus.cc:2267] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.501587 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet b6a8a8da56274eaa8fef29fff38a7dae with delete_type TABLET_DATA_DELETED (replaced by tablet 1ef14443e7fd464a8ab82eb33819dac3) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.501636 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdc2e882dc7e469e857d74e41ee2f435" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:13.501736 14689 raft_consensus.cc:2463] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:13.501834 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.502895 14811 tablet_bootstrap.cc:492] T f4a0952cb8d1412081f3852ce9f60843 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.502952 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdc2e882dc7e469e857d74e41ee2f435" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.503053 14557 raft_consensus.cc:2463] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:13.503298 14358 leader_election.cc:304] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:13.503814 14811 tablet_bootstrap.cc:654] T f4a0952cb8d1412081f3852ce9f60843 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.504041 14861 raft_consensus.cc:2798] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.504098 14861 raft_consensus.cc:491] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.504137 14861 raft_consensus.cc:3054] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.501223 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 8a75594237ce49edbc5ba410038bbecd (table default.Table_6d37c9d2d26e4b73ba4c230e471aeef7 [id=b3cf5d084f3348c091c6e5be7be4808f]) successfully deleted
I20250115 03:06:13.503376 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_ea2592c9eb974e6ab84a4e2f799b5cae [id=d0fc70d8af9944099f8e7ee9a416399d] alter complete (version 1)
I20250115 03:06:13.504877 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 0ecd4741aacb45e2ae6404675f885125 (table default.Table_3e8022382eb84ab9bae6fa5bc222cd1d [id=6fa83879f044445482db1ae6fde9f37f]) successfully deleted
I20250115 03:06:13.504860 14861 raft_consensus.cc:513] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.505164 14861 leader_election.cc:290] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.505317 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet f4a0952cb8d1412081f3852ce9f60843 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.505465 14823 tablet_replica.cc:331] T f4a0952cb8d1412081f3852ce9f60843 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.505582 14823 raft_consensus.cc:2238] T f4a0952cb8d1412081f3852ce9f60843 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.505662 14823 raft_consensus.cc:2267] T f4a0952cb8d1412081f3852ce9f60843 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.505901 14819 tablet_replica.cc:331] T afa7cecc384f495c9ff4723f553acc24 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.505895 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdc2e882dc7e469e857d74e41ee2f435" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:13.505981 14557 raft_consensus.cc:3054] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.505998 14819 raft_consensus.cc:2238] T afa7cecc384f495c9ff4723f553acc24 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.506078 14819 raft_consensus.cc:2267] T afa7cecc384f495c9ff4723f553acc24 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.506362 14819 ts_tablet_manager.cc:1905] T afa7cecc384f495c9ff4723f553acc24 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.506781 14557 raft_consensus.cc:2463] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:13.506970 14820 ts_tablet_manager.cc:1905] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.507988 14819 ts_tablet_manager.cc:1918] T afa7cecc384f495c9ff4723f553acc24 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:13.508045 14819 log.cc:1198] T afa7cecc384f495c9ff4723f553acc24 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/afa7cecc384f495c9ff4723f553acc24
I20250115 03:06:13.508261 14819 ts_tablet_manager.cc:1939] T afa7cecc384f495c9ff4723f553acc24 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.508715 14819 tablet_replica.cc:331] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.508891 14823 ts_tablet_manager.cc:1905] T f4a0952cb8d1412081f3852ce9f60843 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.509341 14812 tablet_bootstrap.cc:492] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.509405 14812 ts_tablet_manager.cc:1397] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.001s
I20250115 03:06:13.509856 14812 raft_consensus.cc:357] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.509943 14812 raft_consensus.cc:383] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.509971 14812 raft_consensus.cc:738] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.510098 14812 consensus_queue.cc:260] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.510535 14820 ts_tablet_manager.cc:1918] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:13.510593 14820 log.cc:1198] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/876e28105fbb47bdb722e5d3f8467341
I20250115 03:06:13.510792 14823 ts_tablet_manager.cc:1918] T f4a0952cb8d1412081f3852ce9f60843 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.510834 14820 ts_tablet_manager.cc:1939] T 876e28105fbb47bdb722e5d3f8467341 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.510849 14823 log.cc:1198] T f4a0952cb8d1412081f3852ce9f60843 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/f4a0952cb8d1412081f3852ce9f60843
I20250115 03:06:13.511023 14823 ts_tablet_manager.cc:1939] T f4a0952cb8d1412081f3852ce9f60843 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.511468 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 876e28105fbb47bdb722e5d3f8467341 (table default.Table_94ee8224b6cd468bb2a1c15b2c2af35a [id=63bd4ca58023436eb2b453f6fa0ea0c4]) successfully deleted
I20250115 03:06:13.511576 14812 ts_tablet_manager.cc:1428] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:13.511595 14820 tablet_replica.cc:331] T 07d9c5e89ac34af29a406e79d2ace824 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.511727 14812 tablet_bootstrap.cc:492] T 044651a15c8346a6a0d11713a82a3641 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.511691 14820 raft_consensus.cc:2238] T 07d9c5e89ac34af29a406e79d2ace824 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.511893 14820 raft_consensus.cc:2267] T 07d9c5e89ac34af29a406e79d2ace824 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.512404 14820 ts_tablet_manager.cc:1905] T 07d9c5e89ac34af29a406e79d2ace824 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.512482 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdc2e882dc7e469e857d74e41ee2f435" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:13.512590 14689 raft_consensus.cc:3054] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.513181 14171 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet bdc2e882dc7e469e857d74e41ee2f435 (table default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]). Expected version 1 got 0
I20250115 03:06:13.513384 14689 raft_consensus.cc:2463] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:13.513572 14812 tablet_bootstrap.cc:654] T 044651a15c8346a6a0d11713a82a3641 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.514101 14820 ts_tablet_manager.cc:1918] T 07d9c5e89ac34af29a406e79d2ace824 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.510020 14809 tablet_bootstrap.cc:492] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.514166 14820 log.cc:1198] T 07d9c5e89ac34af29a406e79d2ace824 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/07d9c5e89ac34af29a406e79d2ace824
I20250115 03:06:13.514220 14809 ts_tablet_manager.cc:1397] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.065s user 0.001s sys 0.000s
I20250115 03:06:13.514324 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet f4a0952cb8d1412081f3852ce9f60843 (table default.Table_8a1be82e3d5648a9b4b5399d2f56e72f [id=72bddfc9455d4d60bd138b4218f77013]) successfully deleted
I20250115 03:06:13.514355 14820 ts_tablet_manager.cc:1939] T 07d9c5e89ac34af29a406e79d2ace824 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.514434 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 9aa9dbbbb8c548a9b3943f0305cb6e9f (table default.Table_937e6e7e4b014731b5945f00775aa14c [id=93321e5844114a63bb53decd1f836d98]) successfully deleted
I20250115 03:06:13.514515 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet afa7cecc384f495c9ff4723f553acc24 (table default.Table_f6b447ccf7e4426484d99557dd1591d7 [id=f4597c64de65413a9885be2bb4301d44]) successfully deleted
I20250115 03:06:13.514640 14809 raft_consensus.cc:357] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.514724 14809 raft_consensus.cc:383] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.514753 14809 raft_consensus.cc:738] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.514883 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet b6a8a8da56274eaa8fef29fff38a7dae with delete_type TABLET_DATA_DELETED (replaced by tablet 1ef14443e7fd464a8ab82eb33819dac3) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.514910 14820 tablet_replica.cc:331] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.515018 14820 raft_consensus.cc:2238] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.515121 14820 raft_consensus.cc:2267] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.515202 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.513665 14358 leader_election.cc:304] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:13.515342 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 07d9c5e89ac34af29a406e79d2ace824 (table default.Table_64c4e8668b494feca761d65318286040 [id=265c248361624c3f9e5ece27144d8e04]) successfully deleted
W20250115 03:06:13.515496 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet f4a0952cb8d1412081f3852ce9f60843 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f4a0952cb8d1412081f3852ce9f60843 already in progress: creating tablet
I20250115 03:06:13.515576 14820 ts_tablet_manager.cc:1905] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.508826 14819 raft_consensus.cc:2238] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.515750 14819 raft_consensus.cc:2267] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.515911 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet bdc2e882dc7e469e857d74e41ee2f435 (table default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]). Expected version 1 got 0
I20250115 03:06:13.516708 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 28324608e3764b7e8a550bc71a4bad58 (table default.Table_705abe6f3ef84a19911b5ee291bd84cc [id=39c58ec19ba2491b93975df1e0690559]). Expected version 1 got 0
I20250115 03:06:13.516741 14861 raft_consensus.cc:2798] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.516829 14861 raft_consensus.cc:695] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:13.516986 14861 consensus_queue.cc:237] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.517331 14820 ts_tablet_manager.cc:1918] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:13.517405 14820 log.cc:1198] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/9c54a5df35e7422b84acc91e3a10aac3
I20250115 03:06:13.517635 14820 ts_tablet_manager.cc:1939] T 9c54a5df35e7422b84acc91e3a10aac3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.518092 14820 tablet_replica.cc:331] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.514855 14809 consensus_queue.cc:260] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.518195 14820 raft_consensus.cc:2238] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.518294 14820 raft_consensus.cc:2267] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.518432 14809 ts_tablet_manager.cc:1428] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:13.518525 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 9c54a5df35e7422b84acc91e3a10aac3 (table default.Table_f957fe3487ec4546a46a8392108003f0 [id=fb3a764b51e74e08a281d128e21466e3]) successfully deleted
I20250115 03:06:13.518596 14809 tablet_bootstrap.cc:492] T 044651a15c8346a6a0d11713a82a3641 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.518725 14171 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_db8a2673977247e0b14b7995f7e09661 [id=f7e4534c1f6f48f09e10b84c1d5c458c] alter complete (version 1)
I20250115 03:06:13.519198 14820 ts_tablet_manager.cc:1905] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.519273 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_490cb72d634f46eaa4130e320f581469"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:13.519446 14809 tablet_bootstrap.cc:654] T 044651a15c8346a6a0d11713a82a3641 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:13.519587 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_490cb72d634f46eaa4130e320f581469 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.521726 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 9aa9dbbbb8c548a9b3943f0305cb6e9f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.522344 14820 ts_tablet_manager.cc:1918] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20250115 03:06:13.522338 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b13f8b40c2f40b8a93ce8030845665c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.522408 14820 log.cc:1198] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/c6cd63ac1c084a0d8f166e63b0ed85bc
I20250115 03:06:13.522927 14823 tablet_replica.cc:331] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.523924 14819 ts_tablet_manager.cc:1905] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.524538 14820 ts_tablet_manager.cc:1939] T c6cd63ac1c084a0d8f166e63b0ed85bc P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.525064 14820 tablet_replica.cc:331] T 99abc1c7a20a4475921597f93c2db56c P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.525173 14820 raft_consensus.cc:2238] T 99abc1c7a20a4475921597f93c2db56c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.525250 14820 raft_consensus.cc:2267] T 99abc1c7a20a4475921597f93c2db56c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.525357 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet c6cd63ac1c084a0d8f166e63b0ed85bc (table default.Table_3bd81cb8bc2d4b74ae3dbcbdb74aae8b [id=05f9fd2ebec34d2299ea59c47a74c828]) successfully deleted
I20250115 03:06:13.525521 14820 ts_tablet_manager.cc:1905] T 99abc1c7a20a4475921597f93c2db56c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.526785 14422 raft_consensus.cc:3054] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.526880 14823 raft_consensus.cc:2238] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.526875 14819 ts_tablet_manager.cc:1918] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.526939 14819 log.cc:1198] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/9ffc6d1d09e04cc99a5e7a1e1af7bef8
I20250115 03:06:13.526963 14823 raft_consensus.cc:2267] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.527103 14819 ts_tablet_manager.cc:1939] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.527231 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_438b02457c07420fa9969504491d9c31"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.527520 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_438b02457c07420fa9969504491d9c31 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.527642 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 9ffc6d1d09e04cc99a5e7a1e1af7bef8 (table default.Table_a07d634b43f842438f4990cd7ff748b5 [id=8d5d65560792413ea180b35bb9319f04]) successfully deleted
I20250115 03:06:13.527684 14422 raft_consensus.cc:1270] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:13.527824 14820 ts_tablet_manager.cc:1918] T 99abc1c7a20a4475921597f93c2db56c P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.527880 14820 log.cc:1198] T 99abc1c7a20a4475921597f93c2db56c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/99abc1c7a20a4475921597f93c2db56c
I20250115 03:06:13.526530 14689 raft_consensus.cc:1270] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250115 03:06:13.528053 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 044651a15c8346a6a0d11713a82a3641 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 044651a15c8346a6a0d11713a82a3641 already in progress: creating tablet
I20250115 03:06:13.528159 14820 ts_tablet_manager.cc:1939] T 99abc1c7a20a4475921597f93c2db56c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.528227 14819 tablet_replica.cc:331] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.528318 14819 raft_consensus.cc:2238] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.528398 14819 raft_consensus.cc:2267] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.528353 14817 consensus_queue.cc:1035] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:13.528612 14820 tablet_replica.cc:331] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.528713 14820 raft_consensus.cc:2238] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:13.528766 14819 ts_tablet_manager.cc:1905] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.528829 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 99abc1c7a20a4475921597f93c2db56c (table default.Table_35d1dd748a6f4e79adced77fe7b2d08d [id=f94088b0776f4be990ddcec04afbca43]) successfully deleted
I20250115 03:06:13.528882 14820 raft_consensus.cc:2267] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.529201 14820 ts_tablet_manager.cc:1905] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.529822 14823 ts_tablet_manager.cc:1905] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.530239 14817 consensus_queue.cc:1035] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:13.531088 14820 ts_tablet_manager.cc:1918] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:13.531154 14820 log.cc:1198] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/35b90c4b0ef341c8a7c1bc517a1231ed
I20250115 03:06:13.531387 14820 ts_tablet_manager.cc:1939] T 35b90c4b0ef341c8a7c1bc517a1231ed P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.531626 14819 ts_tablet_manager.cc:1918] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.531648 14820 tablet_replica.cc:331] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.531683 14819 log.cc:1198] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/551d5ffdcd2d43fcb63e42c135fc2bd5
I20250115 03:06:13.531741 14820 raft_consensus.cc:2238] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.531837 14820 raft_consensus.cc:2267] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.531848 14819 ts_tablet_manager.cc:1939] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.532125 14820 ts_tablet_manager.cc:1905] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.532318 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 35b90c4b0ef341c8a7c1bc517a1231ed (table default.Table_65b9d218ca9743d7a7eaf7cec1e16c80 [id=1b4a3ef0246a4341869bf5163487cac6]) successfully deleted
I20250115 03:06:13.532508 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_44e943822f744731aaee3411219a0765" } new_table_name: "default.Table_3db38338e5b04298954aded6d5482169" modify_external_catalogs: true
I20250115 03:06:13.533767 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 551d5ffdcd2d43fcb63e42c135fc2bd5 (table default.Table_05065fcbba614582b828f0b729c8c07c [id=1b7d3a80c5424e5692c2e6c556919342]) successfully deleted
I20250115 03:06:13.533768 14820 ts_tablet_manager.cc:1918] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:13.533843 14820 log.cc:1198] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/108aafc0c034498bbd300be9c6b1f8c4
I20250115 03:06:13.533957 14819 tablet_replica.cc:331] T b6a8a8da56274eaa8fef29fff38a7dae P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.534049 14820 ts_tablet_manager.cc:1939] T 108aafc0c034498bbd300be9c6b1f8c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.534060 14819 raft_consensus.cc:2238] T b6a8a8da56274eaa8fef29fff38a7dae P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.534137 14819 raft_consensus.cc:2267] T b6a8a8da56274eaa8fef29fff38a7dae P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.534305 14820 tablet_replica.cc:331] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.534407 14820 raft_consensus.cc:2238] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.534487 14820 raft_consensus.cc:2267] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.534515 14819 ts_tablet_manager.cc:1905] T b6a8a8da56274eaa8fef29fff38a7dae P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.534854 14820 ts_tablet_manager.cc:1905] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.535117 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 108aafc0c034498bbd300be9c6b1f8c4 (table default.Table_74d1acd4f65e41bdb046d79c090a111b [id=96a7774dedae4e7fb54cb70b3e8a85fd]) successfully deleted
I20250115 03:06:13.535169 14812 tablet_bootstrap.cc:492] T 044651a15c8346a6a0d11713a82a3641 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.535241 14812 ts_tablet_manager.cc:1397] T 044651a15c8346a6a0d11713a82a3641 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.000s
I20250115 03:06:13.535701 14812 raft_consensus.cc:357] T 044651a15c8346a6a0d11713a82a3641 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.535804 14812 raft_consensus.cc:383] T 044651a15c8346a6a0d11713a82a3641 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.535841 14812 raft_consensus.cc:738] T 044651a15c8346a6a0d11713a82a3641 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.535959 14812 consensus_queue.cc:260] T 044651a15c8346a6a0d11713a82a3641 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.536208 14812 ts_tablet_manager.cc:1428] T 044651a15c8346a6a0d11713a82a3641 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.536372 14812 tablet_bootstrap.cc:492] T 28324608e3764b7e8a550bc71a4bad58 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.536437 14820 ts_tablet_manager.cc:1918] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.536501 14820 log.cc:1198] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/9aa9dbbbb8c548a9b3943f0305cb6e9f
I20250115 03:06:13.536680 14820 ts_tablet_manager.cc:1939] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.537190 14820 tablet_replica.cc:331] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.537297 14820 raft_consensus.cc:2238] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.537323 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 9aa9dbbbb8c548a9b3943f0305cb6e9f (table default.Table_937e6e7e4b014731b5945f00775aa14c [id=93321e5844114a63bb53decd1f836d98]) successfully deleted
I20250115 03:06:13.537384 14820 raft_consensus.cc:2267] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.537392 14812 tablet_bootstrap.cc:654] T 28324608e3764b7e8a550bc71a4bad58 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.537606 14823 ts_tablet_manager.cc:1918] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.537724 14823 log.cc:1198] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/9aa9dbbbb8c548a9b3943f0305cb6e9f
I20250115 03:06:13.537809 14820 ts_tablet_manager.cc:1905] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.538004 14823 ts_tablet_manager.cc:1939] T 9aa9dbbbb8c548a9b3943f0305cb6e9f P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.538305 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet de2fcf7797684ef78e2c4f40b7298afd from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:13.539841 14820 ts_tablet_manager.cc:1918] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.539908 14820 log.cc:1198] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/9ffc6d1d09e04cc99a5e7a1e1af7bef8
I20250115 03:06:13.540073 14820 ts_tablet_manager.cc:1939] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.541616 14819 ts_tablet_manager.cc:1918] T b6a8a8da56274eaa8fef29fff38a7dae P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.541694 14819 log.cc:1198] T b6a8a8da56274eaa8fef29fff38a7dae P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/b6a8a8da56274eaa8fef29fff38a7dae
I20250115 03:06:13.541862 14819 ts_tablet_manager.cc:1939] T b6a8a8da56274eaa8fef29fff38a7dae P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.542497 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 9aa9dbbbb8c548a9b3943f0305cb6e9f (table default.Table_937e6e7e4b014731b5945f00775aa14c [id=93321e5844114a63bb53decd1f836d98]) successfully deleted
I20250115 03:06:13.542547 14823 tablet_replica.cc:331] T 7b13f8b40c2f40b8a93ce8030845665c P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.542750 14823 raft_consensus.cc:2238] T 7b13f8b40c2f40b8a93ce8030845665c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.542805 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet b6a8a8da56274eaa8fef29fff38a7dae (table default.Table_f590b49e96644f96934046c984688857 [id=fd7ff6d50c944f3a80089030d9fc895d]) successfully deleted
I20250115 03:06:13.542917 14823 raft_consensus.cc:2267] T 7b13f8b40c2f40b8a93ce8030845665c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.543001 14819 tablet_replica.cc:331] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.543107 14819 raft_consensus.cc:2238] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:13.543231 14819 pending_rounds.cc:62] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75: Trying to abort 1 pending ops.
I20250115 03:06:13.543273 14819 pending_rounds.cc:69] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384889579454464 op_type: NO_OP noop_request { }
I20250115 03:06:13.543330 14819 raft_consensus.cc:2883] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:13.543386 14819 raft_consensus.cc:2267] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.543669 14823 ts_tablet_manager.cc:1905] T 7b13f8b40c2f40b8a93ce8030845665c P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.543927 14969 tablet.cc:1721] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:13.544445 14819 ts_tablet_manager.cc:1905] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.545382 15274 tablet.cc:1721] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:13.546236 14823 ts_tablet_manager.cc:1918] T 7b13f8b40c2f40b8a93ce8030845665c P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.546308 14823 log.cc:1198] T 7b13f8b40c2f40b8a93ce8030845665c P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/7b13f8b40c2f40b8a93ce8030845665c
I20250115 03:06:13.546265 14819 ts_tablet_manager.cc:1918] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:13.546444 14819 log.cc:1198] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/6fd2c39e3cf14fb8bcf2db6b10fcbb35
I20250115 03:06:13.546494 14823 ts_tablet_manager.cc:1939] T 7b13f8b40c2f40b8a93ce8030845665c P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.546833 14819 ts_tablet_manager.cc:1939] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.547108 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 7b13f8b40c2f40b8a93ce8030845665c (table default.Table_92483ac55d3d40249e8b468ee1511225 [id=bd1ddc98267f4afe9bcd4dd0c0951ff7]) successfully deleted
I20250115 03:06:13.548308 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35 (table default.Table_80b3b9aaffe94c7a9fef7e98b4fefb53 [id=32678ca965cc458daceb1ca31e5214af]) successfully deleted
I20250115 03:06:13.550518 15157 tablet.cc:1721] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:13.551796 14812 tablet_bootstrap.cc:492] T 28324608e3764b7e8a550bc71a4bad58 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.551872 14812 ts_tablet_manager.cc:1397] T 28324608e3764b7e8a550bc71a4bad58 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20250115 03:06:13.552182 15274 tablet.cc:1721] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:13.552287 14812 raft_consensus.cc:357] T 28324608e3764b7e8a550bc71a4bad58 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.552366 14812 raft_consensus.cc:383] T 28324608e3764b7e8a550bc71a4bad58 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.552393 14812 raft_consensus.cc:738] T 28324608e3764b7e8a550bc71a4bad58 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.552495 14812 consensus_queue.cc:260] T 28324608e3764b7e8a550bc71a4bad58 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.552726 14812 ts_tablet_manager.cc:1428] T 28324608e3764b7e8a550bc71a4bad58 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.552875 14812 tablet_bootstrap.cc:492] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.554064 15157 tablet.cc:1721] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:13.555979 14820 tablet_replica.cc:331] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.556138 14820 raft_consensus.cc:2238] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.556207 14969 tablet.cc:1721] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:13.556255 14820 raft_consensus.cc:2267] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.557360 14820 ts_tablet_manager.cc:1905] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.557475 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 9ffc6d1d09e04cc99a5e7a1e1af7bef8 (table default.Table_a07d634b43f842438f4990cd7ff748b5 [id=8d5d65560792413ea180b35bb9319f04]) successfully deleted
I20250115 03:06:13.558337 14165 catalog_manager.cc:5526] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:13.558534 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet bdc2e882dc7e469e857d74e41ee2f435 (table default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]). Expected version 1 got 0
I20250115 03:06:13.558657 14809 tablet_bootstrap.cc:492] T 044651a15c8346a6a0d11713a82a3641 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.558730 14809 ts_tablet_manager.cc:1397] T 044651a15c8346a6a0d11713a82a3641 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.040s user 0.001s sys 0.000s
I20250115 03:06:13.559130 14820 ts_tablet_manager.cc:1918] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.559139 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet d4f76d0b3a394a5f8aa33ad17b11da34 (table default.Table_0dd17013f6334efe97618f0b520512c5 [id=488a9a6268f64669ae17e1a67caa73a5]). Expected version 2 got 0
I20250115 03:06:13.559206 14820 log.cc:1198] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/551d5ffdcd2d43fcb63e42c135fc2bd5
I20250115 03:06:13.559147 14809 raft_consensus.cc:357] T 044651a15c8346a6a0d11713a82a3641 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.559239 14809 raft_consensus.cc:383] T 044651a15c8346a6a0d11713a82a3641 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.559276 14809 raft_consensus.cc:738] T 044651a15c8346a6a0d11713a82a3641 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.559399 14820 ts_tablet_manager.cc:1939] T 551d5ffdcd2d43fcb63e42c135fc2bd5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.559383 14809 consensus_queue.cc:260] T 044651a15c8346a6a0d11713a82a3641 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.559636 14809 ts_tablet_manager.cc:1428] T 044651a15c8346a6a0d11713a82a3641 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.559794 14809 tablet_bootstrap.cc:492] T f4a0952cb8d1412081f3852ce9f60843 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.559885 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 551d5ffdcd2d43fcb63e42c135fc2bd5 (table default.Table_05065fcbba614582b828f0b729c8c07c [id=1b7d3a80c5424e5692c2e6c556919342]) successfully deleted
I20250115 03:06:13.560040 14820 tablet_replica.cc:331] T afa7cecc384f495c9ff4723f553acc24 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.560146 14820 raft_consensus.cc:2238] T afa7cecc384f495c9ff4723f553acc24 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.560237 14820 raft_consensus.cc:2267] T afa7cecc384f495c9ff4723f553acc24 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.560508 14820 ts_tablet_manager.cc:1905] T afa7cecc384f495c9ff4723f553acc24 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.560652 14809 tablet_bootstrap.cc:654] T f4a0952cb8d1412081f3852ce9f60843 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.561805 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 29ef42586d164315b5fd54ba91227bad (DEFAULT_TABLE table=default.Table_d4a2862d217743a181c6139aeeda1f1a [id=c7ebf406fe8c4b4f91ab34cae586b47a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.561846 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 29ef42586d164315b5fd54ba91227bad (DEFAULT_TABLE table=default.Table_d4a2862d217743a181c6139aeeda1f1a [id=c7ebf406fe8c4b4f91ab34cae586b47a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.561939 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 1c76ab180b26487896f4360efe47b2c7 (DEFAULT_TABLE table=default.Table_d8f964febee84681a352ac34900c1e6c [id=dde72643e9ec41e290d4896e58742c70]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.562299 15151 consensus_queue.cc:1035] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:13.562529 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29ef42586d164315b5fd54ba91227bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.562831 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c76ab180b26487896f4360efe47b2c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.563290 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 29ef42586d164315b5fd54ba91227bad (DEFAULT_TABLE table=default.Table_d4a2862d217743a181c6139aeeda1f1a [id=c7ebf406fe8c4b4f91ab34cae586b47a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.563520 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29ef42586d164315b5fd54ba91227bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.566431 14171 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 28324608e3764b7e8a550bc71a4bad58 (table default.Table_705abe6f3ef84a19911b5ee291bd84cc [id=39c58ec19ba2491b93975df1e0690559]). Expected version 1 got 0
I20250115 03:06:13.566826 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29ef42586d164315b5fd54ba91227bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.567874 14812 tablet_bootstrap.cc:654] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.568197 14820 ts_tablet_manager.cc:1918] T afa7cecc384f495c9ff4723f553acc24 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:13.568251 14820 log.cc:1198] T afa7cecc384f495c9ff4723f553acc24 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/afa7cecc384f495c9ff4723f553acc24
I20250115 03:06:13.568513 14820 ts_tablet_manager.cc:1939] T afa7cecc384f495c9ff4723f553acc24 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.570037 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 1c76ab180b26487896f4360efe47b2c7 (DEFAULT_TABLE table=default.Table_d8f964febee84681a352ac34900c1e6c [id=dde72643e9ec41e290d4896e58742c70]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.570247 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c76ab180b26487896f4360efe47b2c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.570755 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet afa7cecc384f495c9ff4723f553acc24 (table default.Table_f6b447ccf7e4426484d99557dd1591d7 [id=f4597c64de65413a9885be2bb4301d44]) successfully deleted
I20250115 03:06:13.572086 14820 tablet_replica.cc:331] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.572192 14820 raft_consensus.cc:2238] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.572260 14820 raft_consensus.cc:2267] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.572625 14820 ts_tablet_manager.cc:1905] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.572881 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_0dd17013f6334efe97618f0b520512c5 [id=488a9a6268f64669ae17e1a67caa73a5] alter complete (version 2)
I20250115 03:06:13.573428 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 1c76ab180b26487896f4360efe47b2c7 (DEFAULT_TABLE table=default.Table_d8f964febee84681a352ac34900c1e6c [id=dde72643e9ec41e290d4896e58742c70]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.573585 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 55dcc7ac82ac44ab9d85055720760d3d (DEFAULT_TABLE table=default.Table_46f14a41d55b4e4783139238fbd0ba49 [id=c9645f75dd5e4011974d87019ff11435]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.573649 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c76ab180b26487896f4360efe47b2c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.573796 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55dcc7ac82ac44ab9d85055720760d3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.574396 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet de2fcf7797684ef78e2c4f40b7298afd
I20250115 03:06:13.574656 14820 ts_tablet_manager.cc:1918] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.574714 14820 log.cc:1198] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/044651a15c8346a6a0d11713a82a3641
I20250115 03:06:13.574890 14820 ts_tablet_manager.cc:1939] T 044651a15c8346a6a0d11713a82a3641 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.575708 14163 catalog_manager.cc:6596] ReplaceTablet: tablet de2fcf7797684ef78e2c4f40b7298afd deleted and replaced by tablet 2d3157859e684da98bb473d5799b24a6
I20250115 03:06:13.575711 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet de2fcf7797684ef78e2c4f40b7298afd with delete_type TABLET_DATA_DELETED (replaced by tablet 2d3157859e684da98bb473d5799b24a6) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.576090 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet de2fcf7797684ef78e2c4f40b7298afd with delete_type TABLET_DATA_DELETED (replaced by tablet 2d3157859e684da98bb473d5799b24a6) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.576160 14528 tablet_service.cc:1467] Processing CreateTablet for tablet 55dcc7ac82ac44ab9d85055720760d3d (DEFAULT_TABLE table=default.Table_46f14a41d55b4e4783139238fbd0ba49 [id=c9645f75dd5e4011974d87019ff11435]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.576429 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 50df3dd199bc4283bc1836d1917988a8 (DEFAULT_TABLE table=default.Table_d75a227f24d644ceb2bda90d1f6ae998 [id=c72398a7988c48cc99ae0d244be1ac1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.576634 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50df3dd199bc4283bc1836d1917988a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.576606 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 50df3dd199bc4283bc1836d1917988a8 (DEFAULT_TABLE table=default.Table_d75a227f24d644ceb2bda90d1f6ae998 [id=c72398a7988c48cc99ae0d244be1ac1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.577358 14809 tablet_bootstrap.cc:492] T f4a0952cb8d1412081f3852ce9f60843 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.577427 14809 ts_tablet_manager.cc:1397] T f4a0952cb8d1412081f3852ce9f60843 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20250115 03:06:13.577849 14809 raft_consensus.cc:357] T f4a0952cb8d1412081f3852ce9f60843 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.577934 14809 raft_consensus.cc:383] T f4a0952cb8d1412081f3852ce9f60843 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.577960 14809 raft_consensus.cc:738] T f4a0952cb8d1412081f3852ce9f60843 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.578372 14812 tablet_bootstrap.cc:492] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.578442 14812 ts_tablet_manager.cc:1397] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20250115 03:06:13.578441 14529 tablet_service.cc:1467] Processing CreateTablet for tablet 6f83dd9a2d13420daefdacdb0f565a7d (DEFAULT_TABLE table=default.Table_ea55848527c6480fbe8b8fa219480faa [id=44488be7335b488a9ff7cf913ce4273a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.578682 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 0d25aa6a292b454f8f451d6493d940c3 (DEFAULT_TABLE table=default.Table_e0faf1e67748430cb2b830441fd58344 [id=8698f51a556d4fcc8ed5ebe0bbc9feb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.578859 14812 raft_consensus.cc:357] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.578948 14812 raft_consensus.cc:383] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.578979 14812 raft_consensus.cc:738] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.579088 14812 consensus_queue.cc:260] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.579351 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 673f9e6cabdd4320b777181c787d15c7 (DEFAULT_TABLE table=default.Table_42a1977fa70d4f16b43bf861b66524a7 [id=c9b28587ba934521bd4158dfaf416f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.579128 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 55dcc7ac82ac44ab9d85055720760d3d (DEFAULT_TABLE table=default.Table_46f14a41d55b4e4783139238fbd0ba49 [id=c9645f75dd5e4011974d87019ff11435]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.580210 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 6f83dd9a2d13420daefdacdb0f565a7d (DEFAULT_TABLE table=default.Table_ea55848527c6480fbe8b8fa219480faa [id=44488be7335b488a9ff7cf913ce4273a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.580415 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f83dd9a2d13420daefdacdb0f565a7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.580998 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 044651a15c8346a6a0d11713a82a3641 (table default.Table_43170518d7504d1d8d17374e281f2728 [id=540df72d30804bdf8595bc7df61e1f46]) successfully deleted
I20250115 03:06:13.581192 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0bdf66c6040b4709b0a6be57188b9d86" } modify_external_catalogs: true
I20250115 03:06:13.581281 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0bdf66c6040b4709b0a6be57188b9d86" } modify_external_catalogs: true
I20250115 03:06:13.581421 14811 tablet_bootstrap.cc:492] T f4a0952cb8d1412081f3852ce9f60843 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.581501 14811 ts_tablet_manager.cc:1397] T f4a0952cb8d1412081f3852ce9f60843 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.079s user 0.001s sys 0.000s
I20250115 03:06:13.581516 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 673f9e6cabdd4320b777181c787d15c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.581750 14422 raft_consensus.cc:1270] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:13.581900 14811 raft_consensus.cc:357] T f4a0952cb8d1412081f3852ce9f60843 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.581987 14811 raft_consensus.cc:383] T f4a0952cb8d1412081f3852ce9f60843 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.582017 14811 raft_consensus.cc:738] T f4a0952cb8d1412081f3852ce9f60843 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.582124 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_3db38338e5b04298954aded6d5482169 [id=f56a0de6e56441bfa620902f175feb4a] alter complete (version 2)
I20250115 03:06:13.582114 14811 consensus_queue.cc:260] T f4a0952cb8d1412081f3852ce9f60843 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.582370 14811 ts_tablet_manager.cc:1428] T f4a0952cb8d1412081f3852ce9f60843 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.582538 14811 tablet_bootstrap.cc:492] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.582621 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 0d25aa6a292b454f8f451d6493d940c3 (DEFAULT_TABLE table=default.Table_e0faf1e67748430cb2b830441fd58344 [id=8698f51a556d4fcc8ed5ebe0bbc9feb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.582818 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d25aa6a292b454f8f451d6493d940c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.583386 14811 tablet_bootstrap.cc:654] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.583817 14858 consensus_queue.cc:1035] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:13.584447 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55dcc7ac82ac44ab9d85055720760d3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.584529 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d68e408815964b6e93e2ed412c1f82e4" } new_table_name: "default.Table_64d7a4f0d2694a65a7fb86e55e1255e0" modify_external_catalogs: true
I20250115 03:06:13.584576 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_48970d0e864f42beaa17e0c0ce6d1786"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.584877 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_48970d0e864f42beaa17e0c0ce6d1786 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.585070 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d25aa6a292b454f8f451d6493d940c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.586865 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f83dd9a2d13420daefdacdb0f565a7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.587338 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 6f83dd9a2d13420daefdacdb0f565a7d (DEFAULT_TABLE table=default.Table_ea55848527c6480fbe8b8fa219480faa [id=44488be7335b488a9ff7cf913ce4273a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.587430 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 50df3dd199bc4283bc1836d1917988a8 (DEFAULT_TABLE table=default.Table_d75a227f24d644ceb2bda90d1f6ae998 [id=c72398a7988c48cc99ae0d244be1ac1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.587586 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 0d25aa6a292b454f8f451d6493d940c3 (DEFAULT_TABLE table=default.Table_e0faf1e67748430cb2b830441fd58344 [id=8698f51a556d4fcc8ed5ebe0bbc9feb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.587586 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 673f9e6cabdd4320b777181c787d15c7 (DEFAULT_TABLE table=default.Table_42a1977fa70d4f16b43bf861b66524a7 [id=c9b28587ba934521bd4158dfaf416f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.588393 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 044651a15c8346a6a0d11713a82a3641 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.588501 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 673f9e6cabdd4320b777181c787d15c7 (DEFAULT_TABLE table=default.Table_42a1977fa70d4f16b43bf861b66524a7 [id=c9b28587ba934521bd4158dfaf416f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.588701 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 673f9e6cabdd4320b777181c787d15c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.588735 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50df3dd199bc4283bc1836d1917988a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.588708 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_bf26dc2911ac4ad496564d8a941c0d9f" } new_table_name: "default.Table_4bfe9fa1f7a143d684bc948b47deb8d8" modify_external_catalogs: true
I20250115 03:06:13.589416 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_6faeecc09aca49888fb4ccd071352ab7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.589725 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6faeecc09aca49888fb4ccd071352ab7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.590597 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_06ad841d9f6e4ae9af7639a562f42036" } new_table_name: "default.Table_3e704f1e03e34a2ca2377db61214c5a0" modify_external_catalogs: true
I20250115 03:06:13.590957 14812 ts_tablet_manager.cc:1428] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.012s user 0.001s sys 0.000s
I20250115 03:06:13.591804 14689 raft_consensus.cc:1270] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:13.592149 14851 consensus_queue.cc:1035] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:13.592434 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55dcc7ac82ac44ab9d85055720760d3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.594363 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 673f9e6cabdd4320b777181c787d15c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.595994 14820 tablet_replica.cc:331] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.596067 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet f4a0952cb8d1412081f3852ce9f60843 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.596104 14820 raft_consensus.cc:2238] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:13.596225 14820 pending_rounds.cc:62] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1: Trying to abort 1 pending ops.
I20250115 03:06:13.596251 14820 pending_rounds.cc:69] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384889403174912 op_type: NO_OP noop_request { }
I20250115 03:06:13.596295 14820 raft_consensus.cc:2883] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:13.596340 14820 raft_consensus.cc:2267] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.596863 14820 ts_tablet_manager.cc:1905] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.597821 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 20f86f93812849c4b7852a884f677c04 (table default.Table_b44636b0d90d4972aceaa7d169a3a477 [id=f8f2a7ff6fdc40468de6ae9ba28c1427]). Expected version 2 got 0
W20250115 03:06:13.597999 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet de2fcf7797684ef78e2c4f40b7298afd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de2fcf7797684ef78e2c4f40b7298afd already in progress: creating tablet
I20250115 03:06:13.598408 15153 consensus_queue.cc:1035] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:13.598506 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ffc6d1d09e04cc99a5e7a1e1af7bef8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.598515 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 044651a15c8346a6a0d11713a82a3641 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.598927 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 20f86f93812849c4b7852a884f677c04 (table default.Table_b44636b0d90d4972aceaa7d169a3a477 [id=f8f2a7ff6fdc40468de6ae9ba28c1427]). Expected version 2 got 0
I20250115 03:06:13.599282 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f83dd9a2d13420daefdacdb0f565a7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.600450 14809 consensus_queue.cc:260] T f4a0952cb8d1412081f3852ce9f60843 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.600733 14809 ts_tablet_manager.cc:1428] T f4a0952cb8d1412081f3852ce9f60843 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.023s user 0.001s sys 0.000s
I20250115 03:06:13.600885 14809 tablet_bootstrap.cc:492] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.601285 14861 raft_consensus.cc:491] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.601353 14861 raft_consensus.cc:513] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.601652 14861 leader_election.cc:290] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.601792 14809 tablet_bootstrap.cc:654] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.601986 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feefb2be7f1440b98d7893dbbf9db46c" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:13.602816 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7b2210a8bd9946508870844ee2e88fb1 consensus state has no leader
I20250115 03:06:13.603145 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feefb2be7f1440b98d7893dbbf9db46c" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.603260 14557 raft_consensus.cc:2388] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8df7247ee9c43e0aa18216da2661f75 in current term 1: Already voted for candidate 5c0dae7aff4b44c699275b95ba639770 in this term.
I20250115 03:06:13.603312 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet de2fcf7797684ef78e2c4f40b7298afd with delete_type TABLET_DATA_DELETED (replaced by tablet 2d3157859e684da98bb473d5799b24a6) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.603837 14820 ts_tablet_manager.cc:1918] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:13.604156 14820 log.cc:1198] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/b6a8a8da56274eaa8fef29fff38a7dae
I20250115 03:06:13.604311 14356 leader_election.cc:304] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:13.604574 14861 raft_consensus.cc:3054] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.605427 14861 raft_consensus.cc:2743] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:13.605904 14820 ts_tablet_manager.cc:1939] T b6a8a8da56274eaa8fef29fff38a7dae P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.607936 14819 tablet_replica.cc:331] T 044651a15c8346a6a0d11713a82a3641 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.608047 14819 raft_consensus.cc:2238] T 044651a15c8346a6a0d11713a82a3641 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.608125 14819 raft_consensus.cc:2267] T 044651a15c8346a6a0d11713a82a3641 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.608871 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_67d007791e734b749420eb72d54c0c69"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:13.609130 14819 ts_tablet_manager.cc:1905] T 044651a15c8346a6a0d11713a82a3641 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:13.609337 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_67d007791e734b749420eb72d54c0c69 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.610003 14823 tablet_replica.cc:331] T 044651a15c8346a6a0d11713a82a3641 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.610121 14823 raft_consensus.cc:2238] T 044651a15c8346a6a0d11713a82a3641 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.610206 14823 raft_consensus.cc:2267] T 044651a15c8346a6a0d11713a82a3641 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.610487 14812 tablet_bootstrap.cc:492] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.611403 14812 tablet_bootstrap.cc:654] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.612738 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d25aa6a292b454f8f451d6493d940c3. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:13.613086 14163 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0580a8f7b2984162b9275d74334b1905 consensus state has no leader
W20250115 03:06:13.614750 14165 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c consensus state has no leader
I20250115 03:06:13.614797 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50df3dd199bc4283bc1836d1917988a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.615670 14528 tablet_service.cc:1467] Processing CreateTablet for tablet f54942abdfb748f2a939993a3c2d5db0 (DEFAULT_TABLE table=default.Table_438b02457c07420fa9969504491d9c31 [id=a5c946319e8a4be5ba4a01a40a9ad0b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.615761 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 01aa12e3d8fa463cbaee1287e7b74dd4 (DEFAULT_TABLE table=default.Table_490cb72d634f46eaa4130e320f581469 [id=fe70549b0af7478ea3ddd2b658fb42fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.615890 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f54942abdfb748f2a939993a3c2d5db0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.616092 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 080224910d554389b646e2c250fada2d (DEFAULT_TABLE table=default.Table_929dc98c434d495da9e1e46de64f5ef9 [id=ade8ca55065d405b994a155664fa3eb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.616286 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 080224910d554389b646e2c250fada2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.617712 14820 tablet_replica.cc:331] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.617815 14820 raft_consensus.cc:2238] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.617826 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 080224910d554389b646e2c250fada2d (DEFAULT_TABLE table=default.Table_929dc98c434d495da9e1e46de64f5ef9 [id=ade8ca55065d405b994a155664fa3eb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.617853 14401 tablet_service.cc:1467] Processing CreateTablet for tablet f54942abdfb748f2a939993a3c2d5db0 (DEFAULT_TABLE table=default.Table_438b02457c07420fa9969504491d9c31 [id=a5c946319e8a4be5ba4a01a40a9ad0b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.617902 14820 raft_consensus.cc:2267] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.612027 14819 ts_tablet_manager.cc:1918] T 044651a15c8346a6a0d11713a82a3641 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.618016 14819 log.cc:1198] T 044651a15c8346a6a0d11713a82a3641 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/044651a15c8346a6a0d11713a82a3641
I20250115 03:06:13.618041 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 080224910d554389b646e2c250fada2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.618091 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet b6a8a8da56274eaa8fef29fff38a7dae (table default.Table_f590b49e96644f96934046c984688857 [id=fd7ff6d50c944f3a80089030d9fc895d]) successfully deleted
I20250115 03:06:13.618211 14819 ts_tablet_manager.cc:1939] T 044651a15c8346a6a0d11713a82a3641 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.618211 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01aa12e3d8fa463cbaee1287e7b74dd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.618371 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f54942abdfb748f2a939993a3c2d5db0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.620067 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 01aa12e3d8fa463cbaee1287e7b74dd4 (DEFAULT_TABLE table=default.Table_490cb72d634f46eaa4130e320f581469 [id=fe70549b0af7478ea3ddd2b658fb42fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.620273 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01aa12e3d8fa463cbaee1287e7b74dd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.620734 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 2dffa87697284dca8f07719f1c861bd4 (DEFAULT_TABLE table=default.Table_7e53fa73287c43e3a9a0e65908b42593 [id=1f983a10a8704e22b8a2b96ef4e06fbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.620812 14528 tablet_service.cc:1467] Processing CreateTablet for tablet 2d3157859e684da98bb473d5799b24a6 (DEFAULT_TABLE table=default.Table_64c4e8668b494feca761d65318286040 [id=265c248361624c3f9e5ece27144d8e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.620920 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dffa87697284dca8f07719f1c861bd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.621042 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d3157859e684da98bb473d5799b24a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.621255 14529 tablet_service.cc:1467] Processing CreateTablet for tablet 4a56ae8a3e3e4f57b13e762cef21f753 (DEFAULT_TABLE table=default.Table_05065fcbba614582b828f0b729c8c07c [id=1b7d3a80c5424e5692c2e6c556919342]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.621439 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a56ae8a3e3e4f57b13e762cef21f753. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.622598 14667 tablet_service.cc:1467] Processing CreateTablet for tablet f54942abdfb748f2a939993a3c2d5db0 (DEFAULT_TABLE table=default.Table_438b02457c07420fa9969504491d9c31 [id=a5c946319e8a4be5ba4a01a40a9ad0b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.623175 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c (DEFAULT_TABLE table=default.Table_4bfe9fa1f7a143d684bc948b47deb8d8 [id=339b61af27cb486aaaadf8730ab21ce7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.623268 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 5883306d8b23451bad367b1809f38bcd
I20250115 03:06:13.623371 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.624429 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 01aa12e3d8fa463cbaee1287e7b74dd4 (DEFAULT_TABLE table=default.Table_490cb72d634f46eaa4130e320f581469 [id=fe70549b0af7478ea3ddd2b658fb42fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.625566 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 2d3157859e684da98bb473d5799b24a6 (DEFAULT_TABLE table=default.Table_64c4e8668b494feca761d65318286040 [id=265c248361624c3f9e5ece27144d8e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.625640 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 2dffa87697284dca8f07719f1c861bd4 (DEFAULT_TABLE table=default.Table_7e53fa73287c43e3a9a0e65908b42593 [id=1f983a10a8704e22b8a2b96ef4e06fbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.625564 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c (DEFAULT_TABLE table=default.Table_4bfe9fa1f7a143d684bc948b47deb8d8 [id=339b61af27cb486aaaadf8730ab21ce7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.625896 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 080224910d554389b646e2c250fada2d (DEFAULT_TABLE table=default.Table_929dc98c434d495da9e1e46de64f5ef9 [id=ade8ca55065d405b994a155664fa3eb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.626313 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c (DEFAULT_TABLE table=default.Table_4bfe9fa1f7a143d684bc948b47deb8d8 [id=339b61af27cb486aaaadf8730ab21ce7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.626590 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.627303 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 044651a15c8346a6a0d11713a82a3641 (table default.Table_43170518d7504d1d8d17374e281f2728 [id=540df72d30804bdf8595bc7df61e1f46]) successfully deleted
I20250115 03:06:13.627401 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 5883306d8b23451bad367b1809f38bcd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:13.627453 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet de2fcf7797684ef78e2c4f40b7298afd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de2fcf7797684ef78e2c4f40b7298afd already in progress: creating tablet
I20250115 03:06:13.627600 14823 ts_tablet_manager.cc:1905] T 044651a15c8346a6a0d11713a82a3641 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.627645 14812 tablet_bootstrap.cc:492] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.627723 14812 ts_tablet_manager.cc:1397] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20250115 03:06:13.628028 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 4a56ae8a3e3e4f57b13e762cef21f753 (DEFAULT_TABLE table=default.Table_05065fcbba614582b828f0b729c8c07c [id=1b7d3a80c5424e5692c2e6c556919342]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.628032 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_aeb59441e2604aa189d7b95ea5e98ed2" } modify_external_catalogs: true
I20250115 03:06:13.628125 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_aeb59441e2604aa189d7b95ea5e98ed2" } modify_external_catalogs: true
I20250115 03:06:13.628156 14812 raft_consensus.cc:357] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.628232 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a56ae8a3e3e4f57b13e762cef21f753. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.628239 14812 raft_consensus.cc:383] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.628266 14812 raft_consensus.cc:738] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.628369 14812 consensus_queue.cc:260] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.630012 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 5883306d8b23451bad367b1809f38bcd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.631062 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f54942abdfb748f2a939993a3c2d5db0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.631214 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01aa12e3d8fa463cbaee1287e7b74dd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.631556 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet 5883306d8b23451bad367b1809f38bcd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.631661 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 4a56ae8a3e3e4f57b13e762cef21f753 (DEFAULT_TABLE table=default.Table_05065fcbba614582b828f0b729c8c07c [id=1b7d3a80c5424e5692c2e6c556919342]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.631749 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d3157859e684da98bb473d5799b24a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.631855 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a56ae8a3e3e4f57b13e762cef21f753. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.632119 15153 raft_consensus.cc:491] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.632189 15153 raft_consensus.cc:513] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.632493 15153 leader_election.cc:290] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:13.633266 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f21761ff87ac43d5b9b47b5905fa5867" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:13.633424 14812 ts_tablet_manager.cc:1428] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:13.633580 14812 tablet_bootstrap.cc:492] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:13.633589 14628 leader_election.cc:343] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.633724 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dffa87697284dca8f07719f1c861bd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.633838 14823 ts_tablet_manager.cc:1918] T 044651a15c8346a6a0d11713a82a3641 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.633890 14823 log.cc:1198] T 044651a15c8346a6a0d11713a82a3641 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/044651a15c8346a6a0d11713a82a3641
I20250115 03:06:13.634068 14823 ts_tablet_manager.cc:1939] T 044651a15c8346a6a0d11713a82a3641 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.634501 14823 tablet_replica.cc:331] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.634604 14823 raft_consensus.cc:2238] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.634676 14823 raft_consensus.cc:2267] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.635051 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.635136 14820 ts_tablet_manager.cc:1905] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.635478 14812 tablet_bootstrap.cc:654] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.636902 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 080224910d554389b646e2c250fada2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.636894 14820 ts_tablet_manager.cc:1918] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.636984 14820 log.cc:1198] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/6fd2c39e3cf14fb8bcf2db6b10fcbb35
I20250115 03:06:13.637167 14820 ts_tablet_manager.cc:1939] T 6fd2c39e3cf14fb8bcf2db6b10fcbb35 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.637740 14820 tablet_replica.cc:331] T f4a0952cb8d1412081f3852ce9f60843 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.637843 14820 raft_consensus.cc:2238] T f4a0952cb8d1412081f3852ce9f60843 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.637926 14820 raft_consensus.cc:2267] T f4a0952cb8d1412081f3852ce9f60843 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.638350 14820 ts_tablet_manager.cc:1905] T f4a0952cb8d1412081f3852ce9f60843 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.640136 14820 ts_tablet_manager.cc:1918] T f4a0952cb8d1412081f3852ce9f60843 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.640200 14820 log.cc:1198] T f4a0952cb8d1412081f3852ce9f60843 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/f4a0952cb8d1412081f3852ce9f60843
I20250115 03:06:13.640286 14861 raft_consensus.cc:491] T fc056b9dbc374d26bf3feaf524f4d834 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.640379 14820 ts_tablet_manager.cc:1939] T f4a0952cb8d1412081f3852ce9f60843 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.640359 14861 raft_consensus.cc:513] T fc056b9dbc374d26bf3feaf524f4d834 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.640641 14861 leader_election.cc:290] T fc056b9dbc374d26bf3feaf524f4d834 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.641477 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 044651a15c8346a6a0d11713a82a3641 (table default.Table_43170518d7504d1d8d17374e281f2728 [id=540df72d30804bdf8595bc7df61e1f46]) successfully deleted
I20250115 03:06:13.641966 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc056b9dbc374d26bf3feaf524f4d834" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:13.642051 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 5883306d8b23451bad367b1809f38bcd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5883306d8b23451bad367b1809f38bcd already in progress: creating tablet
I20250115 03:06:13.642309 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f21761ff87ac43d5b9b47b5905fa5867" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.642623 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc056b9dbc374d26bf3feaf524f4d834" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:13.642742 14691 raft_consensus.cc:2388] T fc056b9dbc374d26bf3feaf524f4d834 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e8df7247ee9c43e0aa18216da2661f75 in current term 1: Already voted for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in this term.
W20250115 03:06:13.642812 14626 leader_election.cc:343] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.642870 14626 leader_election.cc:304] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:13.643024 15153 raft_consensus.cc:2743] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:13.643024 14358 leader_election.cc:304] T fc056b9dbc374d26bf3feaf524f4d834 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:13.643240 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 6fd2c39e3cf14fb8bcf2db6b10fcbb35 (table default.Table_80b3b9aaffe94c7a9fef7e98b4fefb53 [id=32678ca965cc458daceb1ca31e5214af]) successfully deleted
I20250115 03:06:13.643303 14885 raft_consensus.cc:3054] T fc056b9dbc374d26bf3feaf524f4d834 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:13.643339 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet de2fcf7797684ef78e2c4f40b7298afd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de2fcf7797684ef78e2c4f40b7298afd already in progress: creating tablet
I20250115 03:06:13.643409 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet f4a0952cb8d1412081f3852ce9f60843 (table default.Table_8a1be82e3d5648a9b4b5399d2f56e72f [id=72bddfc9455d4d60bd138b4218f77013]) successfully deleted
W20250115 03:06:13.643503 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 5883306d8b23451bad367b1809f38bcd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5883306d8b23451bad367b1809f38bcd already in progress: creating tablet
I20250115 03:06:13.643831 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet f54942abdfb748f2a939993a3c2d5db0 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:13.643834 14691 raft_consensus.cc:1270] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:13.644018 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet 4a56ae8a3e3e4f57b13e762cef21f753 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:13.644099 14885 raft_consensus.cc:2743] T fc056b9dbc374d26bf3feaf524f4d834 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:13.644301 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_c35ae9375d8c41f988ccbd9403698a8a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:13.644345 14885 consensus_queue.cc:1035] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:13.644615 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c35ae9375d8c41f988ccbd9403698a8a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.645323 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e557ad0830f44aec955ce14c80b107e1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.645613 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e557ad0830f44aec955ce14c80b107e1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.646518 14557 raft_consensus.cc:1270] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:13.646448 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 2dffa87697284dca8f07719f1c861bd4 (DEFAULT_TABLE table=default.Table_7e53fa73287c43e3a9a0e65908b42593 [id=1f983a10a8704e22b8a2b96ef4e06fbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.646757 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2dffa87697284dca8f07719f1c861bd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.648679 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet f4a0952cb8d1412081f3852ce9f60843 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.648625 14861 consensus_queue.cc:1035] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:13.648631 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 2d3157859e684da98bb473d5799b24a6 (DEFAULT_TABLE table=default.Table_64c4e8668b494feca761d65318286040 [id=265c248361624c3f9e5ece27144d8e04]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.648810 14819 tablet_replica.cc:331] T f4a0952cb8d1412081f3852ce9f60843 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.648960 14819 raft_consensus.cc:2238] T f4a0952cb8d1412081f3852ce9f60843 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.648972 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d3157859e684da98bb473d5799b24a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.649080 14819 raft_consensus.cc:2267] T f4a0952cb8d1412081f3852ce9f60843 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.650913 14823 ts_tablet_manager.cc:1905] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.652630 14819 ts_tablet_manager.cc:1905] T f4a0952cb8d1412081f3852ce9f60843 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.652637 14823 ts_tablet_manager.cc:1918] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.652693 14823 log.cc:1198] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/9ffc6d1d09e04cc99a5e7a1e1af7bef8
I20250115 03:06:13.652865 14823 ts_tablet_manager.cc:1939] T 9ffc6d1d09e04cc99a5e7a1e1af7bef8 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.653218 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 4a56ae8a3e3e4f57b13e762cef21f753
I20250115 03:06:13.653471 14171 catalog_manager.cc:6596] ReplaceTablet: tablet 4a56ae8a3e3e4f57b13e762cef21f753 deleted and replaced by tablet 5dd115f83c544c4bba0d08fd82220952
I20250115 03:06:13.653645 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet f54942abdfb748f2a939993a3c2d5db0
I20250115 03:06:13.653875 14163 catalog_manager.cc:6596] ReplaceTablet: tablet f54942abdfb748f2a939993a3c2d5db0 deleted and replaced by tablet 382116b1fb6a4f888ab1d55c29917ec8
I20250115 03:06:13.653921 14399 tablet_service.cc:1467] Processing CreateTablet for tablet bb809ec664784386aa5f365bc27e0a1a (DEFAULT_TABLE table=default.Table_48970d0e864f42beaa17e0c0ce6d1786 [id=9391b8f0b8a144f592e4aeaef70fbeef]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.654110 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb809ec664784386aa5f365bc27e0a1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.654600 14819 ts_tablet_manager.cc:1918] T f4a0952cb8d1412081f3852ce9f60843 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.654654 14819 log.cc:1198] T f4a0952cb8d1412081f3852ce9f60843 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/f4a0952cb8d1412081f3852ce9f60843
I20250115 03:06:13.654837 14819 ts_tablet_manager.cc:1939] T f4a0952cb8d1412081f3852ce9f60843 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.655579 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 39245099377549dd81238eabd690b6da
I20250115 03:06:13.656277 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a56ae8a3e3e4f57b13e762cef21f753 with delete_type TABLET_DATA_DELETED (replaced by tablet 5dd115f83c544c4bba0d08fd82220952) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.656471 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet f54942abdfb748f2a939993a3c2d5db0 with delete_type TABLET_DATA_DELETED (replaced by tablet 382116b1fb6a4f888ab1d55c29917ec8) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.656710 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 99ef00eb188a4f628ecbf8fd8906b6b5 (DEFAULT_TABLE table=default.Table_6faeecc09aca49888fb4ccd071352ab7 [id=4d6cb6f5c7354beebee1b3fbe4384d54]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.656905 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99ef00eb188a4f628ecbf8fd8906b6b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.656905 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet f54942abdfb748f2a939993a3c2d5db0 with delete_type TABLET_DATA_DELETED (replaced by tablet 382116b1fb6a4f888ab1d55c29917ec8) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.657070 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 9ffc6d1d09e04cc99a5e7a1e1af7bef8 (table default.Table_a07d634b43f842438f4990cd7ff748b5 [id=8d5d65560792413ea180b35bb9319f04]) successfully deleted
W20250115 03:06:13.657215 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 5883306d8b23451bad367b1809f38bcd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5883306d8b23451bad367b1809f38bcd already in progress: creating tablet
I20250115 03:06:13.657331 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet f4a0952cb8d1412081f3852ce9f60843 (table default.Table_8a1be82e3d5648a9b4b5399d2f56e72f [id=72bddfc9455d4d60bd138b4218f77013]) successfully deleted
W20250115 03:06:13.657516 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet f54942abdfb748f2a939993a3c2d5db0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f54942abdfb748f2a939993a3c2d5db0 already in progress: creating tablet
W20250115 03:06:13.657637 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 4a56ae8a3e3e4f57b13e762cef21f753 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a56ae8a3e3e4f57b13e762cef21f753 already in progress: creating tablet
W20250115 03:06:13.657770 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet f54942abdfb748f2a939993a3c2d5db0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f54942abdfb748f2a939993a3c2d5db0 already in progress: creating tablet
I20250115 03:06:13.657773 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 39245099377549dd81238eabd690b6da with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.658008 14663 tablet_service.cc:1467] Processing CreateTablet for tablet bb809ec664784386aa5f365bc27e0a1a (DEFAULT_TABLE table=default.Table_48970d0e864f42beaa17e0c0ce6d1786 [id=9391b8f0b8a144f592e4aeaef70fbeef]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.658612 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb809ec664784386aa5f365bc27e0a1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.659209 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 39245099377549dd81238eabd690b6da with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:13.659137 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 39245099377549dd81238eabd690b6da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 39245099377549dd81238eabd690b6da already in progress: creating tablet
I20250115 03:06:13.659229 14969 tablet.cc:1721] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:13.659335 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a56ae8a3e3e4f57b13e762cef21f753 with delete_type TABLET_DATA_DELETED (replaced by tablet 5dd115f83c544c4bba0d08fd82220952) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.660789 14811 tablet_bootstrap.cc:492] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.660880 14811 ts_tablet_manager.cc:1397] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.078s user 0.001s sys 0.000s
I20250115 03:06:13.661293 14811 raft_consensus.cc:357] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.661392 14811 raft_consensus.cc:383] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.661422 14811 raft_consensus.cc:738] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.661521 14811 consensus_queue.cc:260] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.661803 14811 ts_tablet_manager.cc:1428] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.661967 14811 tablet_bootstrap.cc:492] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.663251 15299 tablet.cc:1721] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:13.663525 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 99ef00eb188a4f628ecbf8fd8906b6b5 (DEFAULT_TABLE table=default.Table_6faeecc09aca49888fb4ccd071352ab7 [id=4d6cb6f5c7354beebee1b3fbe4384d54]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.663751 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99ef00eb188a4f628ecbf8fd8906b6b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.664350 15157 tablet.cc:1721] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:13.665127 14811 tablet_bootstrap.cc:654] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:13.666260 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 39245099377549dd81238eabd690b6da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 39245099377549dd81238eabd690b6da already in progress: creating tablet
W20250115 03:06:13.666654 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 4a56ae8a3e3e4f57b13e762cef21f753 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a56ae8a3e3e4f57b13e762cef21f753 already in progress: creating tablet
I20250115 03:06:13.667577 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 99ef00eb188a4f628ecbf8fd8906b6b5 (DEFAULT_TABLE table=default.Table_6faeecc09aca49888fb4ccd071352ab7 [id=4d6cb6f5c7354beebee1b3fbe4384d54]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.667784 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99ef00eb188a4f628ecbf8fd8906b6b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.667846 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 41b22aa85ce34b38b0d6a5e2d38f0a1c (DEFAULT_TABLE table=default.Table_67d007791e734b749420eb72d54c0c69 [id=588a068527c446e9aa2d1910ac9dca64]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.667908 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet f54942abdfb748f2a939993a3c2d5db0 with delete_type TABLET_DATA_DELETED (replaced by tablet 382116b1fb6a4f888ab1d55c29917ec8) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.668045 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41b22aa85ce34b38b0d6a5e2d38f0a1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.668179 15274 tablet.cc:1721] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:13.669646 14535 tablet_service.cc:1467] Processing CreateTablet for tablet bb809ec664784386aa5f365bc27e0a1a (DEFAULT_TABLE table=default.Table_48970d0e864f42beaa17e0c0ce6d1786 [id=9391b8f0b8a144f592e4aeaef70fbeef]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.669687 14163 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet bdc2e882dc7e469e857d74e41ee2f435 (table default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]). Expected version 1 got 0
I20250115 03:06:13.669853 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb809ec664784386aa5f365bc27e0a1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.671620 14529 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a56ae8a3e3e4f57b13e762cef21f753 with delete_type TABLET_DATA_DELETED (replaced by tablet 5dd115f83c544c4bba0d08fd82220952) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.671736 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 39245099377549dd81238eabd690b6da with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.671927 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 41b22aa85ce34b38b0d6a5e2d38f0a1c (DEFAULT_TABLE table=default.Table_67d007791e734b749420eb72d54c0c69 [id=588a068527c446e9aa2d1910ac9dca64]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.672118 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41b22aa85ce34b38b0d6a5e2d38f0a1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.675935 14969 tablet.cc:1721] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:13.676540 14168 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516] alter complete (version 1)
I20250115 03:06:13.677611 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 41b22aa85ce34b38b0d6a5e2d38f0a1c (DEFAULT_TABLE table=default.Table_67d007791e734b749420eb72d54c0c69 [id=588a068527c446e9aa2d1910ac9dca64]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.677937 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41b22aa85ce34b38b0d6a5e2d38f0a1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.680881 14809 tablet_bootstrap.cc:492] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.680979 14809 ts_tablet_manager.cc:1397] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.080s user 0.002s sys 0.000s
I20250115 03:06:13.681399 14809 raft_consensus.cc:357] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.681489 14809 raft_consensus.cc:383] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.681548 14809 raft_consensus.cc:738] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.681639 14809 consensus_queue.cc:260] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.681879 14809 ts_tablet_manager.cc:1428] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.682025 14809 tablet_bootstrap.cc:492] T 7b13f8b40c2f40b8a93ce8030845665c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.682363 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 37351252b608492e9acbc818b2754731 (DEFAULT_TABLE table=default.Table_e557ad0830f44aec955ce14c80b107e1 [id=0b8e801e3c8f40a6a5e226fbd824379e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.682564 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37351252b608492e9acbc818b2754731. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.682694 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 5dd115f83c544c4bba0d08fd82220952 (DEFAULT_TABLE table=default.Table_05065fcbba614582b828f0b729c8c07c [id=1b7d3a80c5424e5692c2e6c556919342]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.682861 14809 tablet_bootstrap.cc:654] T 7b13f8b40c2f40b8a93ce8030845665c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.682897 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dd115f83c544c4bba0d08fd82220952. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.684723 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 382116b1fb6a4f888ab1d55c29917ec8 (DEFAULT_TABLE table=default.Table_438b02457c07420fa9969504491d9c31 [id=a5c946319e8a4be5ba4a01a40a9ad0b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.684965 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 382116b1fb6a4f888ab1d55c29917ec8. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:13.685629 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet f54942abdfb748f2a939993a3c2d5db0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f54942abdfb748f2a939993a3c2d5db0 already in progress: creating tablet
W20250115 03:06:13.685763 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 4a56ae8a3e3e4f57b13e762cef21f753 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a56ae8a3e3e4f57b13e762cef21f753 already in progress: creating tablet
I20250115 03:06:13.685720 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 481705dc9ae841659520a6c909879f9b (DEFAULT_TABLE table=default.Table_c35ae9375d8c41f988ccbd9403698a8a [id=fe32f8f8243b4803876ed2b371e9e602]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:13.685863 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 39245099377549dd81238eabd690b6da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 39245099377549dd81238eabd690b6da already in progress: creating tablet
I20250115 03:06:13.685927 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 481705dc9ae841659520a6c909879f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.686175 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 5dd115f83c544c4bba0d08fd82220952 (DEFAULT_TABLE table=default.Table_05065fcbba614582b828f0b729c8c07c [id=1b7d3a80c5424e5692c2e6c556919342]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.687043 14529 tablet_service.cc:1467] Processing CreateTablet for tablet 37351252b608492e9acbc818b2754731 (DEFAULT_TABLE table=default.Table_e557ad0830f44aec955ce14c80b107e1 [id=0b8e801e3c8f40a6a5e226fbd824379e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.687240 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37351252b608492e9acbc818b2754731. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.688575 14163 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_b44636b0d90d4972aceaa7d169a3a477 [id=f8f2a7ff6fdc40468de6ae9ba28c1427] alter complete (version 2)
I20250115 03:06:13.689206 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 481705dc9ae841659520a6c909879f9b (DEFAULT_TABLE table=default.Table_c35ae9375d8c41f988ccbd9403698a8a [id=fe32f8f8243b4803876ed2b371e9e602]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.689399 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 481705dc9ae841659520a6c909879f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.690155 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet f21761ff87ac43d5b9b47b5905fa5867 (table default.Table_53c9a73ae8ef459c987964efac552c96 [id=37d965d8a5eb408ea3f33313bd22c152]). Expected version 2 got 0
I20250115 03:06:13.690527 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_4bde77af131940a099eabda16fa2ccb2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.690821 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4bde77af131940a099eabda16fa2ccb2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.691313 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 5dd115f83c544c4bba0d08fd82220952 (DEFAULT_TABLE table=default.Table_05065fcbba614582b828f0b729c8c07c [id=1b7d3a80c5424e5692c2e6c556919342]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.691497 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dd115f83c544c4bba0d08fd82220952. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.693228 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 481705dc9ae841659520a6c909879f9b (DEFAULT_TABLE table=default.Table_c35ae9375d8c41f988ccbd9403698a8a [id=fe32f8f8243b4803876ed2b371e9e602]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.693480 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 481705dc9ae841659520a6c909879f9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.695133 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 37351252b608492e9acbc818b2754731 (DEFAULT_TABLE table=default.Table_e557ad0830f44aec955ce14c80b107e1 [id=0b8e801e3c8f40a6a5e226fbd824379e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.695324 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37351252b608492e9acbc818b2754731. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.696076 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dd115f83c544c4bba0d08fd82220952. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.696760 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 382116b1fb6a4f888ab1d55c29917ec8 (DEFAULT_TABLE table=default.Table_438b02457c07420fa9969504491d9c31 [id=a5c946319e8a4be5ba4a01a40a9ad0b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.697029 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 382116b1fb6a4f888ab1d55c29917ec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.697052 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_42a1977fa70d4f16b43bf861b66524a7" } modify_external_catalogs: true
I20250115 03:06:13.697146 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_42a1977fa70d4f16b43bf861b66524a7" } modify_external_catalogs: true
I20250115 03:06:13.698894 14812 tablet_bootstrap.cc:492] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.698978 14812 ts_tablet_manager.cc:1397] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.065s user 0.001s sys 0.000s
I20250115 03:06:13.699414 14812 raft_consensus.cc:357] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.699514 14812 raft_consensus.cc:383] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.699558 14812 raft_consensus.cc:738] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.699666 14812 consensus_queue.cc:260] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.699918 14812 ts_tablet_manager.cc:1428] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.700079 14812 tablet_bootstrap.cc:492] T de2fcf7797684ef78e2c4f40b7298afd P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.701192 14812 tablet_bootstrap.cc:654] T de2fcf7797684ef78e2c4f40b7298afd P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.701819 14399 tablet_service.cc:1467] Processing CreateTablet for tablet f978e85cf659450ca0039368d58b49e5 (DEFAULT_TABLE table=default.Table_4bde77af131940a099eabda16fa2ccb2 [id=4f952e57c86746f6bb1df72e13589832]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.702188 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f978e85cf659450ca0039368d58b49e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.703562 15153 raft_consensus.cc:491] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.703636 15153 raft_consensus.cc:513] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.703943 15153 leader_election.cc:290] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:13.704308 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "186b3bf9f6e14c9cb435a1094564f66a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:13.704416 14422 raft_consensus.cc:2463] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:13.704721 14809 tablet_bootstrap.cc:492] T 7b13f8b40c2f40b8a93ce8030845665c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.704789 14809 ts_tablet_manager.cc:1397] T 7b13f8b40c2f40b8a93ce8030845665c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s
I20250115 03:06:13.704969 14628 leader_election.cc:304] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:13.705039 14665 tablet_service.cc:1467] Processing CreateTablet for tablet f978e85cf659450ca0039368d58b49e5 (DEFAULT_TABLE table=default.Table_4bde77af131940a099eabda16fa2ccb2 [id=4f952e57c86746f6bb1df72e13589832]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.705111 15153 raft_consensus.cc:2798] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.705160 15153 raft_consensus.cc:491] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.705195 15153 raft_consensus.cc:3054] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.705241 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f978e85cf659450ca0039368d58b49e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.705226 14809 raft_consensus.cc:357] T 7b13f8b40c2f40b8a93ce8030845665c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.705314 14809 raft_consensus.cc:383] T 7b13f8b40c2f40b8a93ce8030845665c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.705343 14809 raft_consensus.cc:738] T 7b13f8b40c2f40b8a93ce8030845665c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.705446 14809 consensus_queue.cc:260] T 7b13f8b40c2f40b8a93ce8030845665c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.705686 14809 ts_tablet_manager.cc:1428] T 7b13f8b40c2f40b8a93ce8030845665c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.705711 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 673f9e6cabdd4320b777181c787d15c7
I20250115 03:06:13.705848 14809 tablet_bootstrap.cc:492] T 28324608e3764b7e8a550bc71a4bad58 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.706081 15153 raft_consensus.cc:513] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.706377 15153 leader_election.cc:290] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:13.706678 14809 tablet_bootstrap.cc:654] T 28324608e3764b7e8a550bc71a4bad58 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.706657 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 673f9e6cabdd4320b777181c787d15c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:13.707067 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 673f9e6cabdd4320b777181c787d15c7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 673f9e6cabdd4320b777181c787d15c7 already in progress: creating tablet
I20250115 03:06:13.707414 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "186b3bf9f6e14c9cb435a1094564f66a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:13.707504 14422 raft_consensus.cc:3054] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.708300 14422 raft_consensus.cc:2463] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:13.708602 14628 leader_election.cc:304] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:13.708626 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "186b3bf9f6e14c9cb435a1094564f66a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.708680 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e557ad0830f44aec955ce14c80b107e1" } modify_external_catalogs: true
I20250115 03:06:13.708782 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e557ad0830f44aec955ce14c80b107e1" } modify_external_catalogs: true
I20250115 03:06:13.708876 15153 raft_consensus.cc:2798] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.708884 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "186b3bf9f6e14c9cb435a1094564f66a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:13.708988 15153 raft_consensus.cc:695] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:13.709100 15153 consensus_queue.cc:237] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.709765 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 673f9e6cabdd4320b777181c787d15c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.710297 14811 tablet_bootstrap.cc:492] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.710378 14811 ts_tablet_manager.cc:1397] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.048s user 0.002s sys 0.000s
I20250115 03:06:13.710753 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ea2592c9eb974e6ab84a4e2f799b5cae" } new_table_name: "default.Table_605aff02064949c1b48d374a50c910b2" modify_external_catalogs: true
I20250115 03:06:13.710815 14811 raft_consensus.cc:357] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.710908 14811 raft_consensus.cc:383] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.710939 14811 raft_consensus.cc:738] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.711048 14811 consensus_queue.cc:260] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:13.711210 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 673f9e6cabdd4320b777181c787d15c7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 673f9e6cabdd4320b777181c787d15c7 already in progress: creating tablet
I20250115 03:06:13.711330 14811 ts_tablet_manager.cc:1428] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20250115 03:06:13.711419 14626 leader_election.cc:343] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.711494 14811 tablet_bootstrap.cc:492] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
W20250115 03:06:13.711591 14626 leader_election.cc:343] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.711766 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 382116b1fb6a4f888ab1d55c29917ec8 (DEFAULT_TABLE table=default.Table_438b02457c07420fa9969504491d9c31 [id=a5c946319e8a4be5ba4a01a40a9ad0b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.711966 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 382116b1fb6a4f888ab1d55c29917ec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.712361 14811 tablet_bootstrap.cc:654] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.713891 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b13f8b40c2f40b8a93ce8030845665c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.714076 14819 tablet_replica.cc:331] T 7b13f8b40c2f40b8a93ce8030845665c P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.714190 14819 raft_consensus.cc:2238] T 7b13f8b40c2f40b8a93ce8030845665c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.714285 14819 raft_consensus.cc:2267] T 7b13f8b40c2f40b8a93ce8030845665c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.715092 14812 tablet_bootstrap.cc:492] T de2fcf7797684ef78e2c4f40b7298afd P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.715170 14812 ts_tablet_manager.cc:1397] T de2fcf7797684ef78e2c4f40b7298afd P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.001s
I20250115 03:06:13.715575 14812 raft_consensus.cc:357] T de2fcf7797684ef78e2c4f40b7298afd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.715665 14812 raft_consensus.cc:383] T de2fcf7797684ef78e2c4f40b7298afd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.715694 14812 raft_consensus.cc:738] T de2fcf7797684ef78e2c4f40b7298afd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.715808 14812 consensus_queue.cc:260] T de2fcf7797684ef78e2c4f40b7298afd P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.715895 14819 ts_tablet_manager.cc:1905] T 7b13f8b40c2f40b8a93ce8030845665c P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.716045 14812 ts_tablet_manager.cc:1428] T de2fcf7797684ef78e2c4f40b7298afd P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:13.716073 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet b561abc942014fb79f22fd6a9fec38d0 (table default.Table_fa83f65f02b04ad491cee3b2a617927f [id=46f096547b3a4de9a46522b978ac4b8c]). Expected version 1 got 0
I20250115 03:06:13.716086 15274 tablet.cc:1721] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:13.716192 14812 tablet_bootstrap.cc:492] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:13.716363 14163 catalog_manager.cc:5718] Failed to send AlterTable RPC for tablet b561abc942014fb79f22fd6a9fec38d0 (table default.Table_fa83f65f02b04ad491cee3b2a617927f [id=46f096547b3a4de9a46522b978ac4b8c], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet b561abc942014fb79f22fd6a9fec38d0 consensus state has no leader
I20250115 03:06:13.717085 14812 tablet_bootstrap.cc:654] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.717813 14819 ts_tablet_manager.cc:1918] T 7b13f8b40c2f40b8a93ce8030845665c P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.717877 14819 log.cc:1198] T 7b13f8b40c2f40b8a93ce8030845665c P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/7b13f8b40c2f40b8a93ce8030845665c
I20250115 03:06:13.718063 14819 ts_tablet_manager.cc:1939] T 7b13f8b40c2f40b8a93ce8030845665c P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.718609 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 7b13f8b40c2f40b8a93ce8030845665c (table default.Table_92483ac55d3d40249e8b468ee1511225 [id=bd1ddc98267f4afe9bcd4dd0c0951ff7]) successfully deleted
I20250115 03:06:13.719575 14809 tablet_bootstrap.cc:492] T 28324608e3764b7e8a550bc71a4bad58 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.719650 14809 ts_tablet_manager.cc:1397] T 28324608e3764b7e8a550bc71a4bad58 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20250115 03:06:13.720031 14809 raft_consensus.cc:357] T 28324608e3764b7e8a550bc71a4bad58 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.720121 14809 raft_consensus.cc:383] T 28324608e3764b7e8a550bc71a4bad58 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.720152 14809 raft_consensus.cc:738] T 28324608e3764b7e8a550bc71a4bad58 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.720253 14809 consensus_queue.cc:260] T 28324608e3764b7e8a550bc71a4bad58 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.720492 14809 ts_tablet_manager.cc:1428] T 28324608e3764b7e8a550bc71a4bad58 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.720577 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 37351252b608492e9acbc818b2754731
I20250115 03:06:13.720669 14809 tablet_bootstrap.cc:492] T 1ef14443e7fd464a8ab82eb33819dac3 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.721729 14809 tablet_bootstrap.cc:654] T 1ef14443e7fd464a8ab82eb33819dac3 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.722406 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 37351252b608492e9acbc818b2754731 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.722499 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 37351252b608492e9acbc818b2754731 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:13.722909 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 37351252b608492e9acbc818b2754731 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37351252b608492e9acbc818b2754731 already in progress: creating tablet
I20250115 03:06:13.722877 14166 catalog_manager.cc:5526] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
W20250115 03:06:13.723414 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 37351252b608492e9acbc818b2754731 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37351252b608492e9acbc818b2754731 already in progress: creating tablet
I20250115 03:06:13.726173 14168 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 28324608e3764b7e8a550bc71a4bad58 (table default.Table_705abe6f3ef84a19911b5ee291bd84cc [id=39c58ec19ba2491b93975df1e0690559]). Expected version 1 got 0
I20250115 03:06:13.727105 14891 tablet.cc:1721] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:13.727818 14969 tablet.cc:1721] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:13.731390 14162 master_service.cc:946] ReplaceTablet: received request to replace tablet 7b2210a8bd9946508870844ee2e88fb1 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:13.732482 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet de2fcf7797684ef78e2c4f40b7298afd with delete_type TABLET_DATA_DELETED (replaced by tablet 2d3157859e684da98bb473d5799b24a6) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.732836 14166 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 28324608e3764b7e8a550bc71a4bad58 (table default.Table_705abe6f3ef84a19911b5ee291bd84cc [id=39c58ec19ba2491b93975df1e0690559]). Expected version 1 got 0
I20250115 03:06:13.733224 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_669098605d764723aba5170b1c1f12a1" } modify_external_catalogs: true
I20250115 03:06:13.733316 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_669098605d764723aba5170b1c1f12a1" } modify_external_catalogs: true
I20250115 03:06:13.733738 14823 tablet_replica.cc:331] T de2fcf7797684ef78e2c4f40b7298afd P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.733850 14823 raft_consensus.cc:2238] T de2fcf7797684ef78e2c4f40b7298afd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.733937 14823 raft_consensus.cc:2267] T de2fcf7797684ef78e2c4f40b7298afd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.735368 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 7b2210a8bd9946508870844ee2e88fb1
I20250115 03:06:13.735380 14823 ts_tablet_manager.cc:1905] T de2fcf7797684ef78e2c4f40b7298afd P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.735651 14162 catalog_manager.cc:6596] ReplaceTablet: tablet 7b2210a8bd9946508870844ee2e88fb1 deleted and replaced by tablet 55f01d704b174b2ea108ea6f1df763df
I20250115 03:06:13.735842 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b2210a8bd9946508870844ee2e88fb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 55f01d704b174b2ea108ea6f1df763df) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.735978 15299 tablet.cc:1721] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:13.736430 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b2210a8bd9946508870844ee2e88fb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 55f01d704b174b2ea108ea6f1df763df) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.737126 14823 ts_tablet_manager.cc:1918] T de2fcf7797684ef78e2c4f40b7298afd P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.737185 14823 log.cc:1198] T de2fcf7797684ef78e2c4f40b7298afd P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/de2fcf7797684ef78e2c4f40b7298afd
I20250115 03:06:13.737339 14823 ts_tablet_manager.cc:1939] T de2fcf7797684ef78e2c4f40b7298afd P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
W20250115 03:06:13.737679 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 7b2210a8bd9946508870844ee2e88fb1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7b2210a8bd9946508870844ee2e88fb1 already in progress: creating tablet
I20250115 03:06:13.738567 14812 tablet_bootstrap.cc:492] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.738622 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet de2fcf7797684ef78e2c4f40b7298afd (table default.Table_64c4e8668b494feca761d65318286040 [id=265c248361624c3f9e5ece27144d8e04]) successfully deleted
I20250115 03:06:13.738653 14812 ts_tablet_manager.cc:1397] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.001s
W20250115 03:06:13.738772 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 7b2210a8bd9946508870844ee2e88fb1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7b2210a8bd9946508870844ee2e88fb1 already in progress: creating tablet
I20250115 03:06:13.739262 14812 raft_consensus.cc:357] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.739359 14812 raft_consensus.cc:383] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.739394 14812 raft_consensus.cc:738] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.739509 14812 consensus_queue.cc:260] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.740494 14812 ts_tablet_manager.cc:1428] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:13.741434 14812 tablet_bootstrap.cc:492] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.741588 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 0e78c18fd98f44bcbd08b747f2a0ffd0
I20250115 03:06:13.742314 14812 tablet_bootstrap.cc:654] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.743911 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_8497f888af1440288983d17597a1499f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.744196 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8497f888af1440288983d17597a1499f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.744437 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e78c18fd98f44bcbd08b747f2a0ffd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.745018 14809 tablet_bootstrap.cc:492] T 1ef14443e7fd464a8ab82eb33819dac3 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.745085 14809 ts_tablet_manager.cc:1397] T 1ef14443e7fd464a8ab82eb33819dac3 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20250115 03:06:13.745477 14809 raft_consensus.cc:357] T 1ef14443e7fd464a8ab82eb33819dac3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.745563 14809 raft_consensus.cc:383] T 1ef14443e7fd464a8ab82eb33819dac3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.745589 14809 raft_consensus.cc:738] T 1ef14443e7fd464a8ab82eb33819dac3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.745714 14809 consensus_queue.cc:260] T 1ef14443e7fd464a8ab82eb33819dac3 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.746556 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_85ea53c1d8b245cc94e6e81e157242aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:13.746819 14809 ts_tablet_manager.cc:1428] T 1ef14443e7fd464a8ab82eb33819dac3 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
W20250115 03:06:13.746837 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_85ea53c1d8b245cc94e6e81e157242aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.747073 14809 tablet_bootstrap.cc:492] T de2fcf7797684ef78e2c4f40b7298afd P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.747505 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet bdc2e882dc7e469e857d74e41ee2f435 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:13.747751 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_938d94ee23e648b8ba1fbfcbec658c41" } new_table_name: "default.Table_a730bc55668645acadbaa47b450a5338" modify_external_catalogs: true
I20250115 03:06:13.747777 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_c28b01bcd9784e5ead9f2735dec17850"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:13.747838 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b0ca39935d224741840950333b63bcfe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.748037 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c28b01bcd9784e5ead9f2735dec17850 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:13.748106 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b0ca39935d224741840950333b63bcfe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.749084 14535 tablet_service.cc:1467] Processing CreateTablet for tablet f978e85cf659450ca0039368d58b49e5 (DEFAULT_TABLE table=default.Table_4bde77af131940a099eabda16fa2ccb2 [id=4f952e57c86746f6bb1df72e13589832]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.749316 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f978e85cf659450ca0039368d58b49e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.749601 14809 tablet_bootstrap.cc:654] T de2fcf7797684ef78e2c4f40b7298afd P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.749579 14851 consensus_queue.cc:1035] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:13.750499 14812 tablet_bootstrap.cc:492] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.750566 14812 ts_tablet_manager.cc:1397] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20250115 03:06:13.750916 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e78c18fd98f44bcbd08b747f2a0ffd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.750943 14812 raft_consensus.cc:357] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.751034 14812 raft_consensus.cc:383] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.751062 14812 raft_consensus.cc:738] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.751169 14812 consensus_queue.cc:260] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.751258 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 673f9e6cabdd4320b777181c787d15c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.751389 14529 tablet_service.cc:1514] Processing DeleteTablet for tablet 37351252b608492e9acbc818b2754731 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.751489 14812 ts_tablet_manager.cc:1428] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:13.751629 14812 tablet_bootstrap.cc:492] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.751693 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b2210a8bd9946508870844ee2e88fb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 55f01d704b174b2ea108ea6f1df763df) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.752477 14812 tablet_bootstrap.cc:654] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.753610 14163 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_605aff02064949c1b48d374a50c910b2 [id=d0fc70d8af9944099f8e7ee9a416399d] alter complete (version 2)
I20250115 03:06:13.754768 14812 tablet_bootstrap.cc:492] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.754850 14812 ts_tablet_manager.cc:1397] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20250115 03:06:13.755221 14812 raft_consensus.cc:357] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.755313 14812 raft_consensus.cc:383] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.755343 14812 raft_consensus.cc:738] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.755434 14812 consensus_queue.cc:260] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.755652 14812 ts_tablet_manager.cc:1428] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:13.755792 14812 tablet_bootstrap.cc:492] T ae0efe52ac6c40fbae45c725b07b03a3 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.756639 14812 tablet_bootstrap.cc:654] T ae0efe52ac6c40fbae45c725b07b03a3 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.758939 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 55f01d704b174b2ea108ea6f1df763df (DEFAULT_TABLE table=default.Table_64d7a4f0d2694a65a7fb86e55e1255e0 [id=9dcc0154ed914304a15cf96049afaf52]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.759173 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55f01d704b174b2ea108ea6f1df763df. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.759299 15153 raft_consensus.cc:491] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.759359 15153 raft_consensus.cc:513] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.759685 15153 leader_election.cc:290] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:13.760087 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b561abc942014fb79f22fd6a9fec38d0" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:13.760543 14628 leader_election.cc:343] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.762317 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet bdc2e882dc7e469e857d74e41ee2f435
I20250115 03:06:13.762540 14168 catalog_manager.cc:6596] ReplaceTablet: tablet bdc2e882dc7e469e857d74e41ee2f435 deleted and replaced by tablet 53248628d9584f8b886166c1e8ec5c3c
I20250115 03:06:13.762537 14820 tablet_replica.cc:331] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.762656 14820 raft_consensus.cc:2238] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.762748 14820 raft_consensus.cc:2267] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:13.764143 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0c969b652a564ed18fd41bebee47533c consensus state has no leader
I20250115 03:06:13.766368 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet bdc2e882dc7e469e857d74e41ee2f435 with delete_type TABLET_DATA_DELETED (replaced by tablet 53248628d9584f8b886166c1e8ec5c3c) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.766672 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 55f01d704b174b2ea108ea6f1df763df (DEFAULT_TABLE table=default.Table_64d7a4f0d2694a65a7fb86e55e1255e0 [id=9dcc0154ed914304a15cf96049afaf52]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.766908 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55f01d704b174b2ea108ea6f1df763df. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.768537 14812 tablet_bootstrap.cc:492] T ae0efe52ac6c40fbae45c725b07b03a3 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.768611 14812 ts_tablet_manager.cc:1397] T ae0efe52ac6c40fbae45c725b07b03a3 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.002s
I20250115 03:06:13.769016 14812 raft_consensus.cc:357] T ae0efe52ac6c40fbae45c725b07b03a3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.769102 14812 raft_consensus.cc:383] T ae0efe52ac6c40fbae45c725b07b03a3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.769132 14812 raft_consensus.cc:738] T ae0efe52ac6c40fbae45c725b07b03a3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.769234 14812 consensus_queue.cc:260] T ae0efe52ac6c40fbae45c725b07b03a3 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.769246 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b561abc942014fb79f22fd6a9fec38d0" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.769465 14812 ts_tablet_manager.cc:1428] T ae0efe52ac6c40fbae45c725b07b03a3 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20250115 03:06:13.769551 14626 leader_election.cc:343] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.769614 14812 tablet_bootstrap.cc:492] T 7b2210a8bd9946508870844ee2e88fb1 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.769603 14626 leader_election.cc:304] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:13.769942 15153 raft_consensus.cc:2743] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:13.770037 15153 raft_consensus.cc:491] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.770087 15153 raft_consensus.cc:513] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.770269 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet bdc2e882dc7e469e857d74e41ee2f435 with delete_type TABLET_DATA_DELETED (replaced by tablet 53248628d9584f8b886166c1e8ec5c3c) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.770363 15153 leader_election.cc:290] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:13.770428 14812 tablet_bootstrap.cc:654] T 7b2210a8bd9946508870844ee2e88fb1 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.770859 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 55f01d704b174b2ea108ea6f1df763df (DEFAULT_TABLE table=default.Table_64d7a4f0d2694a65a7fb86e55e1255e0 [id=9dcc0154ed914304a15cf96049afaf52]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.771072 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55f01d704b174b2ea108ea6f1df763df. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.771114 14819 tablet_replica.cc:331] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.771215 14819 raft_consensus.cc:2238] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:13.771407 14819 raft_consensus.cc:2267] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.772127 14819 ts_tablet_manager.cc:1905] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.772989 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet bdc2e882dc7e469e857d74e41ee2f435 with delete_type TABLET_DATA_DELETED (replaced by tablet 53248628d9584f8b886166c1e8ec5c3c) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.773320 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef14443e7fd464a8ab82eb33819dac3" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:13.773415 14422 raft_consensus.cc:2463] T 1ef14443e7fd464a8ab82eb33819dac3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:13.773679 14628 leader_election.cc:304] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:13.773874 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef14443e7fd464a8ab82eb33819dac3" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.774070 14819 ts_tablet_manager.cc:1918] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:13.774124 14819 log.cc:1198] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/bdc2e882dc7e469e857d74e41ee2f435
W20250115 03:06:13.774199 14626 leader_election.cc:343] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.774336 14819 ts_tablet_manager.cc:1939] T bdc2e882dc7e469e857d74e41ee2f435 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.774852 15151 raft_consensus.cc:2798] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.774902 15151 raft_consensus.cc:491] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.774935 15151 raft_consensus.cc:3054] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.775712 15151 raft_consensus.cc:513] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.776022 15151 leader_election.cc:290] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:13.776477 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef14443e7fd464a8ab82eb33819dac3" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:13.776613 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet bdc2e882dc7e469e857d74e41ee2f435 (table default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]) successfully deleted
I20250115 03:06:13.776867 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef14443e7fd464a8ab82eb33819dac3" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:13.776993 14422 raft_consensus.cc:3054] T 1ef14443e7fd464a8ab82eb33819dac3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:13.777171 14626 leader_election.cc:343] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.777657 14820 ts_tablet_manager.cc:1905] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.777796 14422 raft_consensus.cc:2463] T 1ef14443e7fd464a8ab82eb33819dac3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:13.779552 14820 ts_tablet_manager.cc:1918] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.779613 14820 log.cc:1198] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/0e78c18fd98f44bcbd08b747f2a0ffd0
I20250115 03:06:13.779796 14820 ts_tablet_manager.cc:1939] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.780428 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 0e78c18fd98f44bcbd08b747f2a0ffd0 (table default.Table_669098605d764723aba5170b1c1f12a1 [id=9c5da4ce43f3446e824ea626b0509209]) successfully deleted
I20250115 03:06:13.780591 14820 tablet_replica.cc:331] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.780694 14820 raft_consensus.cc:2238] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
W20250115 03:06:13.780702 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 673f9e6cabdd4320b777181c787d15c7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 673f9e6cabdd4320b777181c787d15c7 already in progress: creating tablet
I20250115 03:06:13.780792 14820 raft_consensus.cc:2267] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:13.780809 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 37351252b608492e9acbc818b2754731 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 37351252b608492e9acbc818b2754731 already in progress: creating tablet
I20250115 03:06:13.781000 14823 tablet_replica.cc:331] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.781111 14823 raft_consensus.cc:2238] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.781190 14823 raft_consensus.cc:2267] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.781329 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e78c18fd98f44bcbd08b747f2a0ffd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.781370 14628 leader_election.cc:304] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:13.781561 14819 tablet_replica.cc:331] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.781575 15151 raft_consensus.cc:2798] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.781641 15151 raft_consensus.cc:695] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:13.781745 15151 consensus_queue.cc:237] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.781984 14819 raft_consensus.cc:2238] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:13.782153 14819 pending_rounds.cc:62] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75: Trying to abort 1 pending ops.
I20250115 03:06:13.782231 14819 pending_rounds.cc:69] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384889481846784 op_type: NO_OP noop_request { }
I20250115 03:06:13.782337 14819 raft_consensus.cc:2883] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:13.782438 14819 raft_consensus.cc:2267] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.783308 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 914d0f374564460480f93aff40fee17f (DEFAULT_TABLE table=default.Table_b0ca39935d224741840950333b63bcfe [id=08996d7663c94ed1acb94b675012934c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.783591 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 914d0f374564460480f93aff40fee17f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.784070 14812 tablet_bootstrap.cc:492] T 7b2210a8bd9946508870844ee2e88fb1 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.784132 14812 ts_tablet_manager.cc:1397] T 7b2210a8bd9946508870844ee2e88fb1 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.001s
I20250115 03:06:13.784533 14812 raft_consensus.cc:357] T 7b2210a8bd9946508870844ee2e88fb1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.784641 14812 raft_consensus.cc:383] T 7b2210a8bd9946508870844ee2e88fb1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.784670 14812 raft_consensus.cc:738] T 7b2210a8bd9946508870844ee2e88fb1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.784765 14812 consensus_queue.cc:260] T 7b2210a8bd9946508870844ee2e88fb1 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.784991 14165 catalog_manager.cc:5526] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:13.786267 14393 tablet_service.cc:1467] Processing CreateTablet for tablet d4210020188d4f21aed9bb85f3b1f2a1 (DEFAULT_TABLE table=default.Table_8497f888af1440288983d17597a1499f [id=eee0481c12b14e31b6b5c5b9c1688f59]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:13.786403 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 7b2210a8bd9946508870844ee2e88fb1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7b2210a8bd9946508870844ee2e88fb1 already in progress: creating tablet
I20250115 03:06:13.786441 14672 tablet_service.cc:1467] Processing CreateTablet for tablet d4210020188d4f21aed9bb85f3b1f2a1 (DEFAULT_TABLE table=default.Table_8497f888af1440288983d17597a1499f [id=eee0481c12b14e31b6b5c5b9c1688f59]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.786481 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 53248628d9584f8b886166c1e8ec5c3c (DEFAULT_TABLE table=default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.786899 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 914d0f374564460480f93aff40fee17f (DEFAULT_TABLE table=default.Table_b0ca39935d224741840950333b63bcfe [id=08996d7663c94ed1acb94b675012934c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.787151 14536 tablet_service.cc:1467] Processing CreateTablet for tablet b5e7672ebc4442369156d12695261269 (DEFAULT_TABLE table=default.Table_c28b01bcd9784e5ead9f2735dec17850 [id=ab8607ff8b974b59b7a168cf5897ed9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.787212 15151 consensus_queue.cc:1035] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:13.787365 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5e7672ebc4442369156d12695261269. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.787369 14162 master_service.cc:946] ReplaceTablet: received request to replace tablet 4a56ae8a3e3e4f57b13e762cef21f753 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:13.788026 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 53248628d9584f8b886166c1e8ec5c3c (DEFAULT_TABLE table=default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.788276 14663 tablet_service.cc:1467] Processing CreateTablet for tablet b5e7672ebc4442369156d12695261269 (DEFAULT_TABLE table=default.Table_c28b01bcd9784e5ead9f2735dec17850 [id=ab8607ff8b974b59b7a168cf5897ed9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.789296 14529 tablet_service.cc:1467] Processing CreateTablet for tablet 4cc6e937615b4aca8e9625d233452229 (DEFAULT_TABLE table=default.Table_85ea53c1d8b245cc94e6e81e157242aa [id=4d8e724e03dc4e2ab9a6e0281159aca8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.789486 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cc6e937615b4aca8e9625d233452229. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.788043 14812 ts_tablet_manager.cc:1428] T 7b2210a8bd9946508870844ee2e88fb1 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20250115 03:06:13.786484 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4210020188d4f21aed9bb85f3b1f2a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.791388 14527 tablet_service.cc:1467] Processing CreateTablet for tablet d4210020188d4f21aed9bb85f3b1f2a1 (DEFAULT_TABLE table=default.Table_8497f888af1440288983d17597a1499f [id=eee0481c12b14e31b6b5c5b9c1688f59]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.791584 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4210020188d4f21aed9bb85f3b1f2a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.792641 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 914d0f374564460480f93aff40fee17f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.792979 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53248628d9584f8b886166c1e8ec5c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.793694 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 53248628d9584f8b886166c1e8ec5c3c (DEFAULT_TABLE table=default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.793887 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53248628d9584f8b886166c1e8ec5c3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.794919 14811 tablet_bootstrap.cc:492] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.794998 14811 ts_tablet_manager.cc:1397] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.084s user 0.001s sys 0.000s
I20250115 03:06:13.795382 14811 raft_consensus.cc:357] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.795466 14811 raft_consensus.cc:383] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.795493 14811 raft_consensus.cc:738] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.795608 14811 consensus_queue.cc:260] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.795866 14811 ts_tablet_manager.cc:1428] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.795898 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 914d0f374564460480f93aff40fee17f (DEFAULT_TABLE table=default.Table_b0ca39935d224741840950333b63bcfe [id=08996d7663c94ed1acb94b675012934c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.796034 14811 tablet_bootstrap.cc:492] T 7b13f8b40c2f40b8a93ce8030845665c P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.796197 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 914d0f374564460480f93aff40fee17f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.796299 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 4cc6e937615b4aca8e9625d233452229 (DEFAULT_TABLE table=default.Table_85ea53c1d8b245cc94e6e81e157242aa [id=4d8e724e03dc4e2ab9a6e0281159aca8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.796881 14811 tablet_bootstrap.cc:654] T 7b13f8b40c2f40b8a93ce8030845665c P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.798177 14819 ts_tablet_manager.cc:1905] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.799796 14861 raft_consensus.cc:491] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.799877 14861 raft_consensus.cc:513] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.800163 14861 leader_election.cc:290] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.800410 14819 ts_tablet_manager.cc:1918] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:13.800480 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f21761ff87ac43d5b9b47b5905fa5867" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:13.800515 14819 log.cc:1198] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/0e78c18fd98f44bcbd08b747f2a0ffd0
I20250115 03:06:13.800587 14691 raft_consensus.cc:2463] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:13.800850 14819 ts_tablet_manager.cc:1939] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.800832 14358 leader_election.cc:304] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:13.801616 14885 raft_consensus.cc:2798] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.801666 14885 raft_consensus.cc:491] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.801702 14885 raft_consensus.cc:3054] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.802479 14885 raft_consensus.cc:513] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.802786 14885 leader_election.cc:290] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.803098 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f21761ff87ac43d5b9b47b5905fa5867" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:13.803084 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f21761ff87ac43d5b9b47b5905fa5867" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:13.803176 14557 raft_consensus.cc:3054] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.803176 14691 raft_consensus.cc:3054] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.803983 14691 raft_consensus.cc:2463] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:13.803983 14557 raft_consensus.cc:2463] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:13.804456 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet f21761ff87ac43d5b9b47b5905fa5867 (table default.Table_53c9a73ae8ef459c987964efac552c96 [id=37d965d8a5eb408ea3f33313bd22c152]). Expected version 2 got 0
I20250115 03:06:13.804530 15151 consensus_queue.cc:1035] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:13.804711 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_05065fcbba614582b828f0b729c8c07c" } new_table_name: "default.Table_13681f62285b46bc88883e9c804c888e" modify_external_catalogs: true
I20250115 03:06:13.804829 14358 leader_election.cc:304] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:13.805200 14885 raft_consensus.cc:2798] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.805277 14885 raft_consensus.cc:695] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:13.805385 14885 consensus_queue.cc:237] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.806885 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f21761ff87ac43d5b9b47b5905fa5867" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.806980 14556 raft_consensus.cc:2371] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1. Re-sending same reply.
I20250115 03:06:13.807161 14164 catalog_manager.cc:5526] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:13.807335 14164 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet f21761ff87ac43d5b9b47b5905fa5867 (table default.Table_53c9a73ae8ef459c987964efac552c96 [id=37d965d8a5eb408ea3f33313bd22c152]). Expected version 2 got 0
I20250115 03:06:13.808000 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 0e78c18fd98f44bcbd08b747f2a0ffd0 (table default.Table_669098605d764723aba5170b1c1f12a1 [id=9c5da4ce43f3446e824ea626b0509209]) successfully deleted
I20250115 03:06:13.808416 14811 tablet_bootstrap.cc:492] T 7b13f8b40c2f40b8a93ce8030845665c P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.808701 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 4cc6e937615b4aca8e9625d233452229 (DEFAULT_TABLE table=default.Table_85ea53c1d8b245cc94e6e81e157242aa [id=4d8e724e03dc4e2ab9a6e0281159aca8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.808980 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cc6e937615b4aca8e9625d233452229. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.808493 14811 ts_tablet_manager.cc:1397] T 7b13f8b40c2f40b8a93ce8030845665c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20250115 03:06:13.810165 14811 raft_consensus.cc:357] T 7b13f8b40c2f40b8a93ce8030845665c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.810262 14811 raft_consensus.cc:383] T 7b13f8b40c2f40b8a93ce8030845665c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.810294 14811 raft_consensus.cc:738] T 7b13f8b40c2f40b8a93ce8030845665c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.810434 14811 consensus_queue.cc:260] T 7b13f8b40c2f40b8a93ce8030845665c P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.810745 14811 ts_tablet_manager.cc:1428] T 7b13f8b40c2f40b8a93ce8030845665c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.810940 14811 tablet_bootstrap.cc:492] T de2fcf7797684ef78e2c4f40b7298afd P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.812333 14811 tablet_bootstrap.cc:654] T de2fcf7797684ef78e2c4f40b7298afd P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.812353 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b2210a8bd9946508870844ee2e88fb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 55f01d704b174b2ea108ea6f1df763df) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.812701 14399 tablet_service.cc:1467] Processing CreateTablet for tablet b5e7672ebc4442369156d12695261269 (DEFAULT_TABLE table=default.Table_c28b01bcd9784e5ead9f2735dec17850 [id=ab8607ff8b974b59b7a168cf5897ed9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.814287 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5e7672ebc4442369156d12695261269. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.814430 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8497f888af1440288983d17597a1499f" } new_table_name: "default.Table_fb8c07c4a56044248f3fdef9c488f45c" modify_external_catalogs: true
I20250115 03:06:13.814514 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b13f8b40c2f40b8a93ce8030845665c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:12 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.816015 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b947c3886a464283ad54fe14793e5618" } new_table_name: "default.Table_8350c61b90924563b015ad45382885e3" modify_external_catalogs: true
I20250115 03:06:13.816555 14823 ts_tablet_manager.cc:1905] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.817535 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_018ec24d90424975aca7b77736de6136"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.817839 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_018ec24d90424975aca7b77736de6136 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.818617 14823 ts_tablet_manager.cc:1918] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.818678 14823 log.cc:1198] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/0e78c18fd98f44bcbd08b747f2a0ffd0
I20250115 03:06:13.818840 14823 ts_tablet_manager.cc:1939] T 0e78c18fd98f44bcbd08b747f2a0ffd0 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.820158 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_20a4cbca90414ebdac47ec2b786c52eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.820525 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_20a4cbca90414ebdac47ec2b786c52eb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.822116 14820 ts_tablet_manager.cc:1905] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.823284 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53248628d9584f8b886166c1e8ec5c3c. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:13.823626 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5dd115f83c544c4bba0d08fd82220952 consensus state has no leader
I20250115 03:06:13.823832 14812 tablet_bootstrap.cc:492] T 72f9b43975074ac5a7012b132ef74662 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:13.824283 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d4210020188d4f21aed9bb85f3b1f2a1 consensus state has no leader
I20250115 03:06:13.824735 14812 tablet_bootstrap.cc:654] T 72f9b43975074ac5a7012b132ef74662 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.825624 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cc6e937615b4aca8e9625d233452229. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.826704 14422 raft_consensus.cc:1270] T fc056b9dbc374d26bf3feaf524f4d834 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:13.826843 15325 raft_consensus.cc:491] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.826918 15325 raft_consensus.cc:513] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.827219 15325 leader_election.cc:290] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:13.827452 14851 consensus_queue.cc:1035] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:13.827538 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a63742df234d30bdac0173693159e4" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:13.827798 14820 ts_tablet_manager.cc:1918] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20250115 03:06:13.827819 14628 leader_election.cc:343] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.827858 14820 log.cc:1198] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/bdc2e882dc7e469e857d74e41ee2f435
I20250115 03:06:13.828153 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a63742df234d30bdac0173693159e4" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.828238 14820 ts_tablet_manager.cc:1939] T bdc2e882dc7e469e857d74e41ee2f435 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
W20250115 03:06:13.828471 14626 leader_election.cc:343] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.828533 14626 leader_election.cc:304] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:13.828758 15325 raft_consensus.cc:2743] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:13.830933 14851 raft_consensus.cc:491] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.831009 14851 raft_consensus.cc:513] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.831112 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet bdc2e882dc7e469e857d74e41ee2f435 (table default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]) successfully deleted
I20250115 03:06:13.831332 14851 leader_election.cc:290] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.832033 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4210020188d4f21aed9bb85f3b1f2a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.832068 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "186b3bf9f6e14c9cb435a1094564f66a" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:13.831349 14820 tablet_replica.cc:331] T 7b13f8b40c2f40b8a93ce8030845665c P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.832451 14820 raft_consensus.cc:2238] T 7b13f8b40c2f40b8a93ce8030845665c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.832548 14820 raft_consensus.cc:2267] T 7b13f8b40c2f40b8a93ce8030845665c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.832579 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_705abe6f3ef84a19911b5ee291bd84cc" } modify_external_catalogs: true
I20250115 03:06:13.832662 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_705abe6f3ef84a19911b5ee291bd84cc" } modify_external_catalogs: true
I20250115 03:06:13.834007 14162 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet fc056b9dbc374d26bf3feaf524f4d834 (table default.Table_8350c61b90924563b015ad45382885e3 [id=3bcfb6a13f2b4d56b801fcf210764fea]). Expected version 1 got 0
I20250115 03:06:13.834446 14811 tablet_bootstrap.cc:492] T de2fcf7797684ef78e2c4f40b7298afd P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.834530 14811 ts_tablet_manager.cc:1397] T de2fcf7797684ef78e2c4f40b7298afd P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20250115 03:06:13.834813 14823 tablet_replica.cc:331] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.834926 14823 raft_consensus.cc:2238] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.834959 14811 raft_consensus.cc:357] T de2fcf7797684ef78e2c4f40b7298afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.835033 14823 raft_consensus.cc:2267] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.835038 14811 raft_consensus.cc:383] T de2fcf7797684ef78e2c4f40b7298afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.835067 14811 raft_consensus.cc:738] T de2fcf7797684ef78e2c4f40b7298afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.835175 14811 consensus_queue.cc:260] T de2fcf7797684ef78e2c4f40b7298afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.835551 14820 ts_tablet_manager.cc:1905] T 7b13f8b40c2f40b8a93ce8030845665c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.835651 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f4d365d35e3c43e79d3ac20e438e7aa5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.836040 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f4d365d35e3c43e79d3ac20e438e7aa5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.836047 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "186b3bf9f6e14c9cb435a1094564f66a" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:13.836162 14421 raft_consensus.cc:2388] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in current term 1: Already voted for candidate 5c0dae7aff4b44c699275b95ba639770 in this term.
I20250115 03:06:13.836491 14493 leader_election.cc:304] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:13.836757 14853 raft_consensus.cc:3054] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.836872 14811 ts_tablet_manager.cc:1428] T de2fcf7797684ef78e2c4f40b7298afd P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:13.837069 14811 tablet_bootstrap.cc:492] T 1ef14443e7fd464a8ab82eb33819dac3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.837334 14820 ts_tablet_manager.cc:1918] T 7b13f8b40c2f40b8a93ce8030845665c P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.837388 14820 log.cc:1198] T 7b13f8b40c2f40b8a93ce8030845665c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/7b13f8b40c2f40b8a93ce8030845665c
I20250115 03:06:13.837551 14853 raft_consensus.cc:2743] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:13.837581 14820 ts_tablet_manager.cc:1939] T 7b13f8b40c2f40b8a93ce8030845665c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.837814 14811 tablet_bootstrap.cc:654] T 1ef14443e7fd464a8ab82eb33819dac3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.838413 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 0e78c18fd98f44bcbd08b747f2a0ffd0 (table default.Table_669098605d764723aba5170b1c1f12a1 [id=9c5da4ce43f3446e824ea626b0509209]) successfully deleted
I20250115 03:06:13.838678 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 7b13f8b40c2f40b8a93ce8030845665c (table default.Table_92483ac55d3d40249e8b468ee1511225 [id=bd1ddc98267f4afe9bcd4dd0c0951ff7]) successfully deleted
I20250115 03:06:13.838922 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5e7672ebc4442369156d12695261269. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.839047 14811 tablet_bootstrap.cc:492] T 1ef14443e7fd464a8ab82eb33819dac3 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.839113 14811 ts_tablet_manager.cc:1397] T 1ef14443e7fd464a8ab82eb33819dac3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:13.839486 14811 raft_consensus.cc:357] T 1ef14443e7fd464a8ab82eb33819dac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.839591 14811 raft_consensus.cc:383] T 1ef14443e7fd464a8ab82eb33819dac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.839622 14811 raft_consensus.cc:738] T 1ef14443e7fd464a8ab82eb33819dac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.839744 14811 consensus_queue.cc:260] T 1ef14443e7fd464a8ab82eb33819dac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.839985 14811 ts_tablet_manager.cc:1428] T 1ef14443e7fd464a8ab82eb33819dac3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.840135 14811 tablet_bootstrap.cc:492] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.840998 14811 tablet_bootstrap.cc:654] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.841331 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet de2fcf7797684ef78e2c4f40b7298afd with delete_type TABLET_DATA_DELETED (replaced by tablet 2d3157859e684da98bb473d5799b24a6) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.841600 14820 tablet_replica.cc:331] T de2fcf7797684ef78e2c4f40b7298afd P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.841708 14820 raft_consensus.cc:2238] T de2fcf7797684ef78e2c4f40b7298afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.841805 14820 raft_consensus.cc:2267] T de2fcf7797684ef78e2c4f40b7298afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.842731 14823 ts_tablet_manager.cc:1905] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.844239 14823 ts_tablet_manager.cc:1918] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:13.844302 14823 log.cc:1198] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/bdc2e882dc7e469e857d74e41ee2f435
I20250115 03:06:13.844525 14823 ts_tablet_manager.cc:1939] T bdc2e882dc7e469e857d74e41ee2f435 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.845070 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet bdc2e882dc7e469e857d74e41ee2f435 (table default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]) successfully deleted
I20250115 03:06:13.845443 14820 ts_tablet_manager.cc:1905] T de2fcf7797684ef78e2c4f40b7298afd P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.847105 14820 ts_tablet_manager.cc:1918] T de2fcf7797684ef78e2c4f40b7298afd P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.847177 14820 log.cc:1198] T de2fcf7797684ef78e2c4f40b7298afd P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/de2fcf7797684ef78e2c4f40b7298afd
I20250115 03:06:13.847276 14809 tablet_bootstrap.cc:492] T de2fcf7797684ef78e2c4f40b7298afd P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.847358 14820 ts_tablet_manager.cc:1939] T de2fcf7797684ef78e2c4f40b7298afd P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.847363 14809 ts_tablet_manager.cc:1397] T de2fcf7797684ef78e2c4f40b7298afd P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.100s user 0.001s sys 0.001s
I20250115 03:06:13.847782 14809 raft_consensus.cc:357] T de2fcf7797684ef78e2c4f40b7298afd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.847892 14809 raft_consensus.cc:383] T de2fcf7797684ef78e2c4f40b7298afd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.847935 14809 raft_consensus.cc:738] T de2fcf7797684ef78e2c4f40b7298afd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.847950 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet de2fcf7797684ef78e2c4f40b7298afd (table default.Table_64c4e8668b494feca761d65318286040 [id=265c248361624c3f9e5ece27144d8e04]) successfully deleted
I20250115 03:06:13.848055 14809 consensus_queue.cc:260] T de2fcf7797684ef78e2c4f40b7298afd P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.848327 14809 ts_tablet_manager.cc:1428] T de2fcf7797684ef78e2c4f40b7298afd P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.848428 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 28324608e3764b7e8a550bc71a4bad58
I20250115 03:06:13.848511 14809 tablet_bootstrap.cc:492] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.849730 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 28324608e3764b7e8a550bc71a4bad58 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.850342 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 28324608e3764b7e8a550bc71a4bad58 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.850486 14819 tablet_replica.cc:331] T 28324608e3764b7e8a550bc71a4bad58 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.850596 14819 raft_consensus.cc:2238] T 28324608e3764b7e8a550bc71a4bad58 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.850692 14819 raft_consensus.cc:2267] T 28324608e3764b7e8a550bc71a4bad58 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.851085 14819 ts_tablet_manager.cc:1905] T 28324608e3764b7e8a550bc71a4bad58 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.852027 14809 tablet_bootstrap.cc:654] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.853353 14819 ts_tablet_manager.cc:1918] T 28324608e3764b7e8a550bc71a4bad58 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.853422 14819 log.cc:1198] T 28324608e3764b7e8a550bc71a4bad58 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/28324608e3764b7e8a550bc71a4bad58
I20250115 03:06:13.853684 14819 ts_tablet_manager.cc:1939] T 28324608e3764b7e8a550bc71a4bad58 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.854223 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 28324608e3764b7e8a550bc71a4bad58 (table default.Table_705abe6f3ef84a19911b5ee291bd84cc [id=39c58ec19ba2491b93975df1e0690559]) successfully deleted
I20250115 03:06:13.854382 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet de2fcf7797684ef78e2c4f40b7298afd with delete_type TABLET_DATA_DELETED (replaced by tablet 2d3157859e684da98bb473d5799b24a6) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.854584 14819 tablet_replica.cc:331] T de2fcf7797684ef78e2c4f40b7298afd P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.854727 14819 raft_consensus.cc:2238] T de2fcf7797684ef78e2c4f40b7298afd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.854944 14819 raft_consensus.cc:2267] T de2fcf7797684ef78e2c4f40b7298afd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.855134 14820 tablet_replica.cc:331] T 28324608e3764b7e8a550bc71a4bad58 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.855284 14820 raft_consensus.cc:2238] T 28324608e3764b7e8a550bc71a4bad58 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.855312 15299 tablet.cc:1721] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:13.855408 14820 raft_consensus.cc:2267] T 28324608e3764b7e8a550bc71a4bad58 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.855681 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 28324608e3764b7e8a550bc71a4bad58 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.856348 14820 ts_tablet_manager.cc:1905] T 28324608e3764b7e8a550bc71a4bad58 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.854295 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet fc056b9dbc374d26bf3feaf524f4d834 (table default.Table_8350c61b90924563b015ad45382885e3 [id=3bcfb6a13f2b4d56b801fcf210764fea]). Expected version 1 got 0
I20250115 03:06:13.856081 14819 ts_tablet_manager.cc:1905] T de2fcf7797684ef78e2c4f40b7298afd P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.857228 15274 tablet.cc:1721] T fc056b9dbc374d26bf3feaf524f4d834 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:13.858657 14819 ts_tablet_manager.cc:1918] T de2fcf7797684ef78e2c4f40b7298afd P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.858723 14819 log.cc:1198] T de2fcf7797684ef78e2c4f40b7298afd P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/de2fcf7797684ef78e2c4f40b7298afd
I20250115 03:06:13.858906 14819 ts_tablet_manager.cc:1939] T de2fcf7797684ef78e2c4f40b7298afd P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.859203 14820 ts_tablet_manager.cc:1918] T 28324608e3764b7e8a550bc71a4bad58 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.859274 14820 log.cc:1198] T 28324608e3764b7e8a550bc71a4bad58 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/28324608e3764b7e8a550bc71a4bad58
I20250115 03:06:13.859459 14820 ts_tablet_manager.cc:1939] T 28324608e3764b7e8a550bc71a4bad58 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.859439 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet de2fcf7797684ef78e2c4f40b7298afd (table default.Table_64c4e8668b494feca761d65318286040 [id=265c248361624c3f9e5ece27144d8e04]) successfully deleted
I20250115 03:06:13.859972 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 28324608e3764b7e8a550bc71a4bad58 (table default.Table_705abe6f3ef84a19911b5ee291bd84cc [id=39c58ec19ba2491b93975df1e0690559]) successfully deleted
I20250115 03:06:13.860885 14812 tablet_bootstrap.cc:492] T 72f9b43975074ac5a7012b132ef74662 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.860981 14812 ts_tablet_manager.cc:1397] T 72f9b43975074ac5a7012b132ef74662 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.037s user 0.001s sys 0.001s
I20250115 03:06:13.861379 14812 raft_consensus.cc:357] T 72f9b43975074ac5a7012b132ef74662 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.861480 14812 raft_consensus.cc:383] T 72f9b43975074ac5a7012b132ef74662 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.844980 14823 tablet_replica.cc:331] T 7b2210a8bd9946508870844ee2e88fb1 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.861512 14812 raft_consensus.cc:738] T 72f9b43975074ac5a7012b132ef74662 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.861625 14812 consensus_queue.cc:260] T 72f9b43975074ac5a7012b132ef74662 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.861913 14812 ts_tablet_manager.cc:1428] T 72f9b43975074ac5a7012b132ef74662 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:13.862243 14969 tablet.cc:1721] T fc056b9dbc374d26bf3feaf524f4d834 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:13.863592 14812 tablet_bootstrap.cc:492] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.864480 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet fc056b9dbc374d26bf3feaf524f4d834 (table default.Table_8350c61b90924563b015ad45382885e3 [id=3bcfb6a13f2b4d56b801fcf210764fea]). Expected version 1 got 0
I20250115 03:06:13.864626 14812 tablet_bootstrap.cc:654] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.865824 14171 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet fc056b9dbc374d26bf3feaf524f4d834 (table default.Table_8350c61b90924563b015ad45382885e3 [id=3bcfb6a13f2b4d56b801fcf210764fea]). Expected version 1 got 0
I20250115 03:06:13.861609 14823 raft_consensus.cc:2238] T 7b2210a8bd9946508870844ee2e88fb1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.866912 14823 raft_consensus.cc:2267] T 7b2210a8bd9946508870844ee2e88fb1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.867002 14812 tablet_bootstrap.cc:492] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.867116 14812 ts_tablet_manager.cc:1397] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20250115 03:06:13.867987 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2db28e1f2ea5450092ed00f593c4939f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:13.868081 14823 ts_tablet_manager.cc:1905] T 7b2210a8bd9946508870844ee2e88fb1 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:13.868289 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2db28e1f2ea5450092ed00f593c4939f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.869102 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_236770d2959743108285eae378066e45"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.869379 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_236770d2959743108285eae378066e45 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.870213 14166 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_fb8c07c4a56044248f3fdef9c488f45c" } modify_external_catalogs: true
I20250115 03:06:13.870306 14166 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_fb8c07c4a56044248f3fdef9c488f45c" } modify_external_catalogs: true
I20250115 03:06:13.867848 14812 raft_consensus.cc:357] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.870369 14812 raft_consensus.cc:383] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.870406 14812 raft_consensus.cc:738] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.870523 14812 consensus_queue.cc:260] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.870803 14812 ts_tablet_manager.cc:1428] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20250115 03:06:13.870958 14812 tablet_bootstrap.cc:492] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.872823 14165 master_service.cc:946] ReplaceTablet: received request to replace tablet 2dffa87697284dca8f07719f1c861bd4 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:13.874061 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet feefb2be7f1440b98d7893dbbf9db46c from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:13.874852 14812 tablet_bootstrap.cc:654] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.875574 14823 ts_tablet_manager.cc:1918] T 7b2210a8bd9946508870844ee2e88fb1 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.875650 14823 log.cc:1198] T 7b2210a8bd9946508870844ee2e88fb1 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/7b2210a8bd9946508870844ee2e88fb1
I20250115 03:06:13.876405 14809 tablet_bootstrap.cc:492] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.876487 14809 ts_tablet_manager.cc:1397] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.028s user 0.001s sys 0.000s
I20250115 03:06:13.876520 14811 tablet_bootstrap.cc:492] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.876590 14811 ts_tablet_manager.cc:1397] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.036s user 0.000s sys 0.002s
I20250115 03:06:13.876988 14809 raft_consensus.cc:357] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.877000 14811 raft_consensus.cc:357] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.877095 14809 raft_consensus.cc:383] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.877095 14811 raft_consensus.cc:383] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.877126 14811 raft_consensus.cc:738] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.877126 14809 raft_consensus.cc:738] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.877235 14809 consensus_queue.cc:260] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.877236 14811 consensus_queue.cc:260] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.877481 14811 ts_tablet_manager.cc:1428] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:13.877503 14809 ts_tablet_manager.cc:1428] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.877655 14809 tablet_bootstrap.cc:492] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.878363 14162 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_8350c61b90924563b015ad45382885e3 [id=3bcfb6a13f2b4d56b801fcf210764fea] alter complete (version 1)
I20250115 03:06:13.878631 14809 tablet_bootstrap.cc:654] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.879484 14811 tablet_bootstrap.cc:492] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.880185 14809 tablet_bootstrap.cc:492] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.880272 14809 ts_tablet_manager.cc:1397] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:13.880321 14811 tablet_bootstrap.cc:654] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.880703 14809 raft_consensus.cc:357] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.880806 14809 raft_consensus.cc:383] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.880846 14809 raft_consensus.cc:738] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.881079 14809 consensus_queue.cc:260] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.881523 14809 ts_tablet_manager.cc:1428] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:13.881533 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet feefb2be7f1440b98d7893dbbf9db46c
I20250115 03:06:13.881690 14809 tablet_bootstrap.cc:492] T 21cc97960f7c49c7bf57499e66fd7190 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.881814 14171 catalog_manager.cc:6596] ReplaceTablet: tablet feefb2be7f1440b98d7893dbbf9db46c deleted and replaced by tablet 3e46d4a75e9b4ca8b83a188d06280fcd
I20250115 03:06:13.882222 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet feefb2be7f1440b98d7893dbbf9db46c with delete_type TABLET_DATA_DELETED (replaced by tablet 3e46d4a75e9b4ca8b83a188d06280fcd) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.882398 14820 tablet_replica.cc:331] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:13.882390 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet feefb2be7f1440b98d7893dbbf9db46c with delete_type TABLET_DATA_DELETED (replaced by tablet 3e46d4a75e9b4ca8b83a188d06280fcd) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.882517 14820 raft_consensus.cc:2238] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.882622 14820 raft_consensus.cc:2267] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.882927 14809 tablet_bootstrap.cc:654] T 21cc97960f7c49c7bf57499e66fd7190 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.883172 14820 ts_tablet_manager.cc:1905] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.885532 14811 tablet_bootstrap.cc:492] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.885617 14811 ts_tablet_manager.cc:1397] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:13.886004 14811 raft_consensus.cc:357] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.886101 14811 raft_consensus.cc:383] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.886224 14811 raft_consensus.cc:738] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.886372 14811 consensus_queue.cc:260] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.887511 14811 ts_tablet_manager.cc:1428] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:13.887684 14811 tablet_bootstrap.cc:492] T 21cc97960f7c49c7bf57499e66fd7190 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.888549 14811 tablet_bootstrap.cc:654] T 21cc97960f7c49c7bf57499e66fd7190 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.889257 14820 ts_tablet_manager.cc:1918] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:13.889330 14820 log.cc:1198] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/feefb2be7f1440b98d7893dbbf9db46c
I20250115 03:06:13.889567 14820 ts_tablet_manager.cc:1939] T feefb2be7f1440b98d7893dbbf9db46c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:13.890228 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet feefb2be7f1440b98d7893dbbf9db46c (table default.Table_0d7244964d874f5eb1f508e5c0dabf1a [id=15762a37e62f4e89852b2eb66a008dbb]) successfully deleted
I20250115 03:06:13.890583 14819 tablet_replica.cc:331] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.890936 14809 tablet_bootstrap.cc:492] T 21cc97960f7c49c7bf57499e66fd7190 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.891017 14809 ts_tablet_manager.cc:1397] T 21cc97960f7c49c7bf57499e66fd7190 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20250115 03:06:13.890915 14819 raft_consensus.cc:2238] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.891120 14819 raft_consensus.cc:2267] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.891536 14809 raft_consensus.cc:357] T 21cc97960f7c49c7bf57499e66fd7190 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.891628 14809 raft_consensus.cc:383] T 21cc97960f7c49c7bf57499e66fd7190 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.891659 14809 raft_consensus.cc:738] T 21cc97960f7c49c7bf57499e66fd7190 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.891700 14819 ts_tablet_manager.cc:1905] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.891779 14809 consensus_queue.cc:260] T 21cc97960f7c49c7bf57499e66fd7190 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.892625 14809 ts_tablet_manager.cc:1428] T 21cc97960f7c49c7bf57499e66fd7190 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:13.892782 14809 tablet_bootstrap.cc:492] T c4a63742df234d30bdac0173693159e4 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.894137 14811 tablet_bootstrap.cc:492] T 21cc97960f7c49c7bf57499e66fd7190 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.894212 14811 ts_tablet_manager.cc:1397] T 21cc97960f7c49c7bf57499e66fd7190 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:13.894297 14809 tablet_bootstrap.cc:654] T c4a63742df234d30bdac0173693159e4 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.894639 14811 raft_consensus.cc:357] T 21cc97960f7c49c7bf57499e66fd7190 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.894737 14811 raft_consensus.cc:383] T 21cc97960f7c49c7bf57499e66fd7190 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.894784 14811 raft_consensus.cc:738] T 21cc97960f7c49c7bf57499e66fd7190 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.894892 14811 consensus_queue.cc:260] T 21cc97960f7c49c7bf57499e66fd7190 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.895139 14811 ts_tablet_manager.cc:1428] T 21cc97960f7c49c7bf57499e66fd7190 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.895330 14811 tablet_bootstrap.cc:492] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.895925 14809 tablet_bootstrap.cc:492] T c4a63742df234d30bdac0173693159e4 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.896001 14809 ts_tablet_manager.cc:1397] T c4a63742df234d30bdac0173693159e4 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:13.896169 14811 tablet_bootstrap.cc:654] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.896517 14809 raft_consensus.cc:357] T c4a63742df234d30bdac0173693159e4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.896624 14809 raft_consensus.cc:383] T c4a63742df234d30bdac0173693159e4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.896654 14809 raft_consensus.cc:738] T c4a63742df234d30bdac0173693159e4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.896785 14809 consensus_queue.cc:260] T c4a63742df234d30bdac0173693159e4 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.897176 14819 ts_tablet_manager.cc:1918] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:13.897228 14819 log.cc:1198] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/feefb2be7f1440b98d7893dbbf9db46c
I20250115 03:06:13.897434 14819 ts_tablet_manager.cc:1939] T feefb2be7f1440b98d7893dbbf9db46c P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.897684 14809 ts_tablet_manager.cc:1428] T c4a63742df234d30bdac0173693159e4 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:13.897835 14809 tablet_bootstrap.cc:492] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.897984 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet feefb2be7f1440b98d7893dbbf9db46c (table default.Table_0d7244964d874f5eb1f508e5c0dabf1a [id=15762a37e62f4e89852b2eb66a008dbb]) successfully deleted
I20250115 03:06:13.882150 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet feefb2be7f1440b98d7893dbbf9db46c with delete_type TABLET_DATA_DELETED (replaced by tablet 3e46d4a75e9b4ca8b83a188d06280fcd) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.875842 14823 ts_tablet_manager.cc:1939] T 7b2210a8bd9946508870844ee2e88fb1 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.893679 14812 tablet_bootstrap.cc:492] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.898527 14812 ts_tablet_manager.cc:1397] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.002s
I20250115 03:06:13.898802 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 7b2210a8bd9946508870844ee2e88fb1 (table default.Table_64d7a4f0d2694a65a7fb86e55e1255e0 [id=9dcc0154ed914304a15cf96049afaf52]) successfully deleted
I20250115 03:06:13.898986 14823 tablet_replica.cc:331] T 28324608e3764b7e8a550bc71a4bad58 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.898950 14812 raft_consensus.cc:357] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.899039 14812 raft_consensus.cc:383] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.899070 14812 raft_consensus.cc:738] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.899094 14823 raft_consensus.cc:2238] T 28324608e3764b7e8a550bc71a4bad58 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.899174 14812 consensus_queue.cc:260] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.899420 14812 ts_tablet_manager.cc:1428] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:13.899574 14809 tablet_bootstrap.cc:654] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.900358 14809 tablet_bootstrap.cc:492] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.900424 14809 ts_tablet_manager.cc:1397] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:13.900521 14812 tablet_bootstrap.cc:492] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.900779 14809 raft_consensus.cc:357] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.900857 14809 raft_consensus.cc:383] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.900883 14809 raft_consensus.cc:738] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.901018 14809 consensus_queue.cc:260] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.901235 14809 ts_tablet_manager.cc:1428] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.901360 14809 tablet_bootstrap.cc:492] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.901450 14812 tablet_bootstrap.cc:654] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.902149 14809 tablet_bootstrap.cc:654] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.904726 14809 tablet_bootstrap.cc:492] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.904805 14809 ts_tablet_manager.cc:1397] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:13.899178 14823 raft_consensus.cc:2267] T 28324608e3764b7e8a550bc71a4bad58 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.905202 14809 raft_consensus.cc:357] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.905288 14809 raft_consensus.cc:383] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.905315 14809 raft_consensus.cc:738] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.905421 14809 consensus_queue.cc:260] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.905669 14809 ts_tablet_manager.cc:1428] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.905718 14811 tablet_bootstrap.cc:492] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.905799 14811 ts_tablet_manager.cc:1397] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20250115 03:06:13.905820 14809 tablet_bootstrap.cc:492] T 7b2210a8bd9946508870844ee2e88fb1 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.906144 14811 raft_consensus.cc:357] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.906224 14811 raft_consensus.cc:383] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.906250 14811 raft_consensus.cc:738] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.906379 14811 consensus_queue.cc:260] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.906634 14809 tablet_bootstrap.cc:654] T 7b2210a8bd9946508870844ee2e88fb1 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.907711 14811 ts_tablet_manager.cc:1428] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:13.907867 14811 tablet_bootstrap.cc:492] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.908672 14811 tablet_bootstrap.cc:654] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.909672 14812 tablet_bootstrap.cc:492] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.909750 14812 ts_tablet_manager.cc:1397] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20250115 03:06:13.910147 14812 raft_consensus.cc:357] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.910252 14812 raft_consensus.cc:383] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.910292 14812 raft_consensus.cc:738] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.910969 14812 consensus_queue.cc:260] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.911238 14812 ts_tablet_manager.cc:1428] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.911303 14861 raft_consensus.cc:491] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.911404 14812 tablet_bootstrap.cc:492] T 543bbb3ae87c46a6982e6fa1ae325db6 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.911381 14861 raft_consensus.cc:513] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.911715 14861 leader_election.cc:290] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.912070 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0efe52ac6c40fbae45c725b07b03a3" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:13.912168 14689 raft_consensus.cc:2463] T ae0efe52ac6c40fbae45c725b07b03a3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:13.912441 14358 leader_election.cc:304] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:13.912839 14861 raft_consensus.cc:2798] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.912891 14861 raft_consensus.cc:491] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.912878 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0efe52ac6c40fbae45c725b07b03a3" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:13.912945 14861 raft_consensus.cc:3054] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.913198 14812 tablet_bootstrap.cc:654] T 543bbb3ae87c46a6982e6fa1ae325db6 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.913615 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 2dffa87697284dca8f07719f1c861bd4
I20250115 03:06:13.913830 14861 raft_consensus.cc:513] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.913903 14165 catalog_manager.cc:6596] ReplaceTablet: tablet 2dffa87697284dca8f07719f1c861bd4 deleted and replaced by tablet a1cbe29c08c7477e8944b7e108d46e7a
I20250115 03:06:13.914088 14812 tablet_bootstrap.cc:492] T 543bbb3ae87c46a6982e6fa1ae325db6 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.914108 14861 leader_election.cc:290] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.914157 14812 ts_tablet_manager.cc:1397] T 543bbb3ae87c46a6982e6fa1ae325db6 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:13.914347 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dffa87697284dca8f07719f1c861bd4 with delete_type TABLET_DATA_DELETED (replaced by tablet a1cbe29c08c7477e8944b7e108d46e7a) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.914445 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dffa87697284dca8f07719f1c861bd4 with delete_type TABLET_DATA_DELETED (replaced by tablet a1cbe29c08c7477e8944b7e108d46e7a) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.914505 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0efe52ac6c40fbae45c725b07b03a3" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:13.914597 14689 raft_consensus.cc:3054] T ae0efe52ac6c40fbae45c725b07b03a3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.914750 14812 raft_consensus.cc:357] T 543bbb3ae87c46a6982e6fa1ae325db6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.914841 14812 raft_consensus.cc:383] T 543bbb3ae87c46a6982e6fa1ae325db6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.914872 14812 raft_consensus.cc:738] T 543bbb3ae87c46a6982e6fa1ae325db6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.914995 14812 consensus_queue.cc:260] T 543bbb3ae87c46a6982e6fa1ae325db6 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.915306 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dffa87697284dca8f07719f1c861bd4 with delete_type TABLET_DATA_DELETED (replaced by tablet a1cbe29c08c7477e8944b7e108d46e7a) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:13.915321 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 2dffa87697284dca8f07719f1c861bd4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2dffa87697284dca8f07719f1c861bd4 already in progress: creating tablet
I20250115 03:06:13.915500 14689 raft_consensus.cc:2463] T ae0efe52ac6c40fbae45c725b07b03a3 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:13.915596 14812 ts_tablet_manager.cc:1428] T 543bbb3ae87c46a6982e6fa1ae325db6 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.915750 14812 tablet_bootstrap.cc:492] T 5883306d8b23451bad367b1809f38bcd P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:13.915975 14356 leader_election.cc:343] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.916076 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae0efe52ac6c40fbae45c725b07b03a3" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:13.916178 14358 leader_election.cc:304] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:13.916597 14812 tablet_bootstrap.cc:654] T 5883306d8b23451bad367b1809f38bcd P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:13.916735 14356 leader_election.cc:343] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:13.917513 14861 raft_consensus.cc:2798] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.917608 14861 raft_consensus.cc:695] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:13.917712 14861 consensus_queue.cc:237] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.918524 14811 tablet_bootstrap.cc:492] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.918609 14811 ts_tablet_manager.cc:1397] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20250115 03:06:13.919028 14811 raft_consensus.cc:357] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.919124 14811 raft_consensus.cc:383] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.919152 14811 raft_consensus.cc:738] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.919272 14811 consensus_queue.cc:260] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.919322 14167 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet b561abc942014fb79f22fd6a9fec38d0 (table default.Table_fa83f65f02b04ad491cee3b2a617927f [id=46f096547b3a4de9a46522b978ac4b8c]). Expected version 1 got 0
I20250115 03:06:13.919739 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet b561abc942014fb79f22fd6a9fec38d0 (table default.Table_fa83f65f02b04ad491cee3b2a617927f [id=46f096547b3a4de9a46522b978ac4b8c]). Expected version 1 got 0
W20250115 03:06:13.921172 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 2dffa87697284dca8f07719f1c861bd4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2dffa87697284dca8f07719f1c861bd4 already in progress: creating tablet
I20250115 03:06:13.922228 14811 ts_tablet_manager.cc:1428] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20250115 03:06:13.923307 14811 tablet_bootstrap.cc:492] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.924350 14811 tablet_bootstrap.cc:654] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.925422 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet d4210020188d4f21aed9bb85f3b1f2a1
I20250115 03:06:13.925976 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet d4210020188d4f21aed9bb85f3b1f2a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.925979 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet d4210020188d4f21aed9bb85f3b1f2a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.926443 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet d4210020188d4f21aed9bb85f3b1f2a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:13.926762 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet d4210020188d4f21aed9bb85f3b1f2a1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d4210020188d4f21aed9bb85f3b1f2a1 already in progress: creating tablet
W20250115 03:06:13.926781 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet d4210020188d4f21aed9bb85f3b1f2a1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d4210020188d4f21aed9bb85f3b1f2a1 already in progress: creating tablet
I20250115 03:06:13.927585 14823 ts_tablet_manager.cc:1905] T 28324608e3764b7e8a550bc71a4bad58 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.929484 14823 ts_tablet_manager.cc:1918] T 28324608e3764b7e8a550bc71a4bad58 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.929560 14823 log.cc:1198] T 28324608e3764b7e8a550bc71a4bad58 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/28324608e3764b7e8a550bc71a4bad58
I20250115 03:06:13.929741 14823 ts_tablet_manager.cc:1939] T 28324608e3764b7e8a550bc71a4bad58 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.930186 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 47b08f6e7dec4c5a871a49c79421209e (DEFAULT_TABLE table=default.Table_f4d365d35e3c43e79d3ac20e438e7aa5 [id=835a8f488a7c42af850632b1286f9509]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.930258 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 28324608e3764b7e8a550bc71a4bad58 (table default.Table_705abe6f3ef84a19911b5ee291bd84cc [id=39c58ec19ba2491b93975df1e0690559]) successfully deleted
I20250115 03:06:13.930408 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47b08f6e7dec4c5a871a49c79421209e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.931049 14169 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 0c969b652a564ed18fd41bebee47533c (table default.Table_a730bc55668645acadbaa47b450a5338 [id=e31e6fb5a7e941c494ef416ef4762fb7]). Expected version 1 got 0
I20250115 03:06:13.933455 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 47b08f6e7dec4c5a871a49c79421209e (DEFAULT_TABLE table=default.Table_f4d365d35e3c43e79d3ac20e438e7aa5 [id=835a8f488a7c42af850632b1286f9509]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.933547 14823 tablet_replica.cc:331] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.933619 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 47b08f6e7dec4c5a871a49c79421209e (DEFAULT_TABLE table=default.Table_f4d365d35e3c43e79d3ac20e438e7aa5 [id=835a8f488a7c42af850632b1286f9509]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.933657 14823 raft_consensus.cc:2238] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:13.933683 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47b08f6e7dec4c5a871a49c79421209e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.933823 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47b08f6e7dec4c5a871a49c79421209e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.934273 14811 tablet_bootstrap.cc:492] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:13.934336 14811 ts_tablet_manager.cc:1397] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.001s
I20250115 03:06:13.934713 14811 raft_consensus.cc:357] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.934813 14811 raft_consensus.cc:383] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.934844 14811 raft_consensus.cc:738] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.934957 14811 consensus_queue.cc:260] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.935192 14811 ts_tablet_manager.cc:1428] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:13.935338 14811 tablet_bootstrap.cc:492] T 7b2210a8bd9946508870844ee2e88fb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.935506 14162 catalog_manager.cc:5526] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:13.936172 14853 raft_consensus.cc:491] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:13.936251 14853 raft_consensus.cc:513] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.936480 14401 tablet_service.cc:1467] Processing CreateTablet for tablet ccf6b58c55d747da89c1c218a70ce09f (DEFAULT_TABLE table=default.Table_20a4cbca90414ebdac47ec2b786c52eb [id=e441d6f5acea4837accbfb8e4856f82d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.936570 14853 leader_election.cc:290] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.936674 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 4d4b8d4ed9f44112b83c6ff94aee2206 (DEFAULT_TABLE table=default.Table_018ec24d90424975aca7b77736de6136 [id=adfd7e86aa0f48e5b5e36b4113517dba]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.936861 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e43ce064a02347efa81ca274d76c6960" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:13.936980 14689 raft_consensus.cc:2463] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:13.937054 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e43ce064a02347efa81ca274d76c6960" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:13.937146 14422 raft_consensus.cc:2463] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:13.937240 14493 leader_election.cc:304] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:13.936679 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccf6b58c55d747da89c1c218a70ce09f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.937603 14853 raft_consensus.cc:2798] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:13.937657 14853 raft_consensus.cc:491] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:13.937693 14853 raft_consensus.cc:3054] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.938472 14853 raft_consensus.cc:513] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.938771 14853 leader_election.cc:290] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:13.938818 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d4b8d4ed9f44112b83c6ff94aee2206. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.939340 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e43ce064a02347efa81ca274d76c6960" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:13.939448 14689 raft_consensus.cc:3054] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.939663 14530 tablet_service.cc:1467] Processing CreateTablet for tablet ccf6b58c55d747da89c1c218a70ce09f (DEFAULT_TABLE table=default.Table_20a4cbca90414ebdac47ec2b786c52eb [id=e441d6f5acea4837accbfb8e4856f82d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.939862 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccf6b58c55d747da89c1c218a70ce09f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.936198 14811 tablet_bootstrap.cc:654] T 7b2210a8bd9946508870844ee2e88fb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.940234 14689 raft_consensus.cc:2463] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:13.940757 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 4d4b8d4ed9f44112b83c6ff94aee2206 (DEFAULT_TABLE table=default.Table_018ec24d90424975aca7b77736de6136 [id=adfd7e86aa0f48e5b5e36b4113517dba]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.940949 14493 leader_election.cc:304] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:13.940985 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d4b8d4ed9f44112b83c6ff94aee2206. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.941088 14853 raft_consensus.cc:2798] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:13.941149 14853 raft_consensus.cc:695] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:13.941251 14853 consensus_queue.cc:237] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.942806 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e43ce064a02347efa81ca274d76c6960" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:13.942895 14422 raft_consensus.cc:3054] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:13.943585 14812 tablet_bootstrap.cc:492] T 5883306d8b23451bad367b1809f38bcd P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.943656 14812 ts_tablet_manager.cc:1397] T 5883306d8b23451bad367b1809f38bcd P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.028s user 0.001s sys 0.000s
I20250115 03:06:13.943661 14422 raft_consensus.cc:2463] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:13.944077 14812 raft_consensus.cc:357] T 5883306d8b23451bad367b1809f38bcd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.944180 14812 raft_consensus.cc:383] T 5883306d8b23451bad367b1809f38bcd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.944211 14812 raft_consensus.cc:738] T 5883306d8b23451bad367b1809f38bcd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.944330 14812 consensus_queue.cc:260] T 5883306d8b23451bad367b1809f38bcd P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.944388 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 4d4b8d4ed9f44112b83c6ff94aee2206 (DEFAULT_TABLE table=default.Table_018ec24d90424975aca7b77736de6136 [id=adfd7e86aa0f48e5b5e36b4113517dba]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.944593 14812 ts_tablet_manager.cc:1428] T 5883306d8b23451bad367b1809f38bcd P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.944765 14812 tablet_bootstrap.cc:492] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.944597 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d4b8d4ed9f44112b83c6ff94aee2206. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.946750 14812 tablet_bootstrap.cc:654] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.948100 14663 tablet_service.cc:1467] Processing CreateTablet for tablet ccf6b58c55d747da89c1c218a70ce09f (DEFAULT_TABLE table=default.Table_20a4cbca90414ebdac47ec2b786c52eb [id=e441d6f5acea4837accbfb8e4856f82d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.948287 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccf6b58c55d747da89c1c218a70ce09f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.949565 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_9432efe75c694779976428f3dea16679" } modify_external_catalogs: true
I20250115 03:06:13.949668 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_9432efe75c694779976428f3dea16679" } modify_external_catalogs: true
I20250115 03:06:13.949739 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ff21d7bc463d43cfa3330f7288ba7a55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:13.950035 14811 tablet_bootstrap.cc:492] T 7b2210a8bd9946508870844ee2e88fb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
W20250115 03:06:13.950038 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff21d7bc463d43cfa3330f7288ba7a55 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.950106 14811 ts_tablet_manager.cc:1397] T 7b2210a8bd9946508870844ee2e88fb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20250115 03:06:13.950292 14812 tablet_bootstrap.cc:492] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.950364 14812 ts_tablet_manager.cc:1397] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250115 03:06:13.950482 14811 raft_consensus.cc:357] T 7b2210a8bd9946508870844ee2e88fb1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.950572 14811 raft_consensus.cc:383] T 7b2210a8bd9946508870844ee2e88fb1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.950598 14811 raft_consensus.cc:738] T 7b2210a8bd9946508870844ee2e88fb1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.950701 14166 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_28b265c831a944c7bff1a3ee55a86c64" } modify_external_catalogs: true
I20250115 03:06:13.950726 14811 consensus_queue.cc:260] T 7b2210a8bd9946508870844ee2e88fb1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.950742 14812 raft_consensus.cc:357] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.950798 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 5883306d8b23451bad367b1809f38bcd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.950802 14166 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_28b265c831a944c7bff1a3ee55a86c64" } modify_external_catalogs: true
I20250115 03:06:13.950817 14812 raft_consensus.cc:383] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.951766 14812 raft_consensus.cc:738] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.933823 14823 raft_consensus.cc:2267] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.951941 14811 ts_tablet_manager.cc:1428] T 7b2210a8bd9946508870844ee2e88fb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:13.951905 14812 consensus_queue.cc:260] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.952093 14811 tablet_bootstrap.cc:492] T ae0efe52ac6c40fbae45c725b07b03a3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:13.952145 14812 ts_tablet_manager.cc:1428] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:13.952316 14812 tablet_bootstrap.cc:492] T 2fcce3505b744f668c5355aa982c8485 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.953325 14812 tablet_bootstrap.cc:654] T 2fcce3505b744f668c5355aa982c8485 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.953366 14811 tablet_bootstrap.cc:654] T ae0efe52ac6c40fbae45c725b07b03a3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.953891 14823 ts_tablet_manager.cc:1905] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.956838 14809 tablet_bootstrap.cc:492] T 7b2210a8bd9946508870844ee2e88fb1 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.957005 14809 ts_tablet_manager.cc:1397] T 7b2210a8bd9946508870844ee2e88fb1 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.051s user 0.001s sys 0.000s
I20250115 03:06:13.957448 14809 raft_consensus.cc:357] T 7b2210a8bd9946508870844ee2e88fb1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.957537 14809 raft_consensus.cc:383] T 7b2210a8bd9946508870844ee2e88fb1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.957566 14809 raft_consensus.cc:738] T 7b2210a8bd9946508870844ee2e88fb1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.957664 14809 consensus_queue.cc:260] T 7b2210a8bd9946508870844ee2e88fb1 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.957888 14809 ts_tablet_manager.cc:1428] T 7b2210a8bd9946508870844ee2e88fb1 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:13.958022 14809 tablet_bootstrap.cc:492] T 72f9b43975074ac5a7012b132ef74662 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.959103 14809 tablet_bootstrap.cc:654] T 72f9b43975074ac5a7012b132ef74662 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:13.962116 14171 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e8e168dcba994883bb43ad269eac0c91" } new_table_name: "default.Table_ec8ee0d483474271a3b369a2bcde593d" modify_external_catalogs: true
I20250115 03:06:13.962975 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f957fe3487ec4546a46a8392108003f0" } modify_external_catalogs: true
I20250115 03:06:13.963063 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f957fe3487ec4546a46a8392108003f0" } modify_external_catalogs: true
I20250115 03:06:13.963985 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet 2fcce3505b744f668c5355aa982c8485 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:13.965358 14823 ts_tablet_manager.cc:1918] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:13.965420 14823 log.cc:1198] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/feefb2be7f1440b98d7893dbbf9db46c
I20250115 03:06:13.965628 14823 ts_tablet_manager.cc:1939] T feefb2be7f1440b98d7893dbbf9db46c P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:13.966496 14165 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet d4f770f63a7348afb71904b93c68c379 (table default.Table_117c4a2ba1cb4369881c9bc30c21781b [id=50c73c26b4ed47a4961348db433b7dd4]). Expected version 1 got 0
I20250115 03:06:13.968410 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b2210a8bd9946508870844ee2e88fb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 55f01d704b174b2ea108ea6f1df763df) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:13.968570 14171 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5ad5197958ce45de8b9d78174aaae689 consensus state has no leader
I20250115 03:06:13.968585 14819 tablet_replica.cc:331] T 7b2210a8bd9946508870844ee2e88fb1 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:13.968678 14819 raft_consensus.cc:2238] T 7b2210a8bd9946508870844ee2e88fb1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:13.968746 14819 raft_consensus.cc:2267] T 7b2210a8bd9946508870844ee2e88fb1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:13.969106 14819 ts_tablet_manager.cc:1905] T 7b2210a8bd9946508870844ee2e88fb1 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.969203 14169 catalog_manager.cc:5526] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:13.970939 14819 ts_tablet_manager.cc:1918] T 7b2210a8bd9946508870844ee2e88fb1 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.971006 14819 log.cc:1198] T 7b2210a8bd9946508870844ee2e88fb1 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/7b2210a8bd9946508870844ee2e88fb1
I20250115 03:06:13.971182 14819 ts_tablet_manager.cc:1939] T 7b2210a8bd9946508870844ee2e88fb1 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:13.971737 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 7b2210a8bd9946508870844ee2e88fb1 (table default.Table_64d7a4f0d2694a65a7fb86e55e1255e0 [id=9dcc0154ed914304a15cf96049afaf52]) successfully deleted
I20250115 03:06:13.972326 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 543bbb3ae87c46a6982e6fa1ae325db6
I20250115 03:06:13.972488 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f7b803cd6e0546c394296cb6b4b2dc83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.972801 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f7b803cd6e0546c394296cb6b4b2dc83 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.972858 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 2fcce3505b744f668c5355aa982c8485
I20250115 03:06:13.973110 14168 catalog_manager.cc:6596] ReplaceTablet: tablet 2fcce3505b744f668c5355aa982c8485 deleted and replaced by tablet e368f87556944c46a6d8f160ead17ac9
I20250115 03:06:13.973685 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 543bbb3ae87c46a6982e6fa1ae325db6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.973834 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 543bbb3ae87c46a6982e6fa1ae325db6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.973912 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 543bbb3ae87c46a6982e6fa1ae325db6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.973951 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fcce3505b744f668c5355aa982c8485 with delete_type TABLET_DATA_DELETED (replaced by tablet e368f87556944c46a6d8f160ead17ac9) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.974049 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fcce3505b744f668c5355aa982c8485 with delete_type TABLET_DATA_DELETED (replaced by tablet e368f87556944c46a6d8f160ead17ac9) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.974109 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet bcf6745e549d47cdbdf97c2dff6997ad
I20250115 03:06:13.974268 14535 tablet_service.cc:1467] Processing CreateTablet for tablet e3302d62a34e43d09c11abca292d9110 (DEFAULT_TABLE table=default.Table_2db28e1f2ea5450092ed00f593c4939f [id=6fe98239ee3346df8b25712aaec67b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.974467 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3302d62a34e43d09c11abca292d9110. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.976284 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fcce3505b744f668c5355aa982c8485 with delete_type TABLET_DATA_DELETED (replaced by tablet e368f87556944c46a6d8f160ead17ac9) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.976481 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet bcf6745e549d47cdbdf97c2dff6997ad with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.977568 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 89d92f59c7424dfc95427329200455e7 (DEFAULT_TABLE table=default.Table_236770d2959743108285eae378066e45 [id=fec89f498f5046839fe491caa3367a36]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.977793 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89d92f59c7424dfc95427329200455e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.979107 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_6bb3a06c1ae749e593cdc3f2b3992eaf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.979390 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6bb3a06c1ae749e593cdc3f2b3992eaf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.979879 14664 tablet_service.cc:1467] Processing CreateTablet for tablet a1cbe29c08c7477e8944b7e108d46e7a (DEFAULT_TABLE table=default.Table_7e53fa73287c43e3a9a0e65908b42593 [id=1f983a10a8704e22b8a2b96ef4e06fbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.980064 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1cbe29c08c7477e8944b7e108d46e7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.980795 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_64c4e8668b494feca761d65318286040" } new_table_name: "default.Table_02c01532217e4a8d9092ecb9bed7c891" modify_external_catalogs: true
I20250115 03:06:13.981104 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 3e46d4a75e9b4ca8b83a188d06280fcd (DEFAULT_TABLE table=default.Table_0d7244964d874f5eb1f508e5c0dabf1a [id=15762a37e62f4e89852b2eb66a008dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.981163 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ebca4429c164491c974b3ee244c3b1de"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:13.981276 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e46d4a75e9b4ca8b83a188d06280fcd. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:13.981422 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ebca4429c164491c974b3ee244c3b1de in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.982658 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet ef002211ee1c4a149232c8e433277add
I20250115 03:06:13.982937 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet bcf6745e549d47cdbdf97c2dff6997ad with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.983151 14672 tablet_service.cc:1467] Processing CreateTablet for tablet e3302d62a34e43d09c11abca292d9110 (DEFAULT_TABLE table=default.Table_2db28e1f2ea5450092ed00f593c4939f [id=6fe98239ee3346df8b25712aaec67b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.983338 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3302d62a34e43d09c11abca292d9110. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.979118 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d9b79701501644caa5b8be7be1d354df"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:13.984629 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d9b79701501644caa5b8be7be1d354df in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:13.985567 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet feefb2be7f1440b98d7893dbbf9db46c (table default.Table_0d7244964d874f5eb1f508e5c0dabf1a [id=15762a37e62f4e89852b2eb66a008dbb]) successfully deleted
I20250115 03:06:13.985697 14823 tablet_replica.cc:331] T 5883306d8b23451bad367b1809f38bcd P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:13.985666 14396 tablet_service.cc:1467] Processing CreateTablet for tablet a1cbe29c08c7477e8944b7e108d46e7a (DEFAULT_TABLE table=default.Table_7e53fa73287c43e3a9a0e65908b42593 [id=1f983a10a8704e22b8a2b96ef4e06fbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.985811 14823 raft_consensus.cc:2238] T 5883306d8b23451bad367b1809f38bcd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
W20250115 03:06:13.985838 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 2dffa87697284dca8f07719f1c861bd4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2dffa87697284dca8f07719f1c861bd4 already in progress: creating tablet
I20250115 03:06:13.985895 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1cbe29c08c7477e8944b7e108d46e7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.985901 14823 raft_consensus.cc:2267] T 5883306d8b23451bad367b1809f38bcd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:13.985967 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet d4210020188d4f21aed9bb85f3b1f2a1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d4210020188d4f21aed9bb85f3b1f2a1 already in progress: creating tablet
I20250115 03:06:13.985963 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 89d92f59c7424dfc95427329200455e7 (DEFAULT_TABLE table=default.Table_236770d2959743108285eae378066e45 [id=fec89f498f5046839fe491caa3367a36]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.986133 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89d92f59c7424dfc95427329200455e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.986315 14823 ts_tablet_manager.cc:1905] T 5883306d8b23451bad367b1809f38bcd P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:13.987577 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet bcf6745e549d47cdbdf97c2dff6997ad with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.987769 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 3e46d4a75e9b4ca8b83a188d06280fcd (DEFAULT_TABLE table=default.Table_0d7244964d874f5eb1f508e5c0dabf1a [id=15762a37e62f4e89852b2eb66a008dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.987824 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet ef002211ee1c4a149232c8e433277add with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:13.987974 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e46d4a75e9b4ca8b83a188d06280fcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.989725 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet ef002211ee1c4a149232c8e433277add with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:13.987769 14401 tablet_service.cc:1467] Processing CreateTablet for tablet e3302d62a34e43d09c11abca292d9110 (DEFAULT_TABLE table=default.Table_2db28e1f2ea5450092ed00f593c4939f [id=6fe98239ee3346df8b25712aaec67b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.990581 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3302d62a34e43d09c11abca292d9110. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.990597 14809 tablet_bootstrap.cc:492] T 72f9b43975074ac5a7012b132ef74662 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:13.990671 14809 ts_tablet_manager.cc:1397] T 72f9b43975074ac5a7012b132ef74662 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.033s user 0.000s sys 0.002s
I20250115 03:06:13.991150 14809 raft_consensus.cc:357] T 72f9b43975074ac5a7012b132ef74662 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.991240 14809 raft_consensus.cc:383] T 72f9b43975074ac5a7012b132ef74662 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.991272 14809 raft_consensus.cc:738] T 72f9b43975074ac5a7012b132ef74662 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.991396 14809 consensus_queue.cc:260] T 72f9b43975074ac5a7012b132ef74662 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:13.991668 14809 ts_tablet_manager.cc:1428] T 72f9b43975074ac5a7012b132ef74662 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:13.991851 14809 tablet_bootstrap.cc:492] T 0c969b652a564ed18fd41bebee47533c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:13.992465 14823 ts_tablet_manager.cc:1918] T 5883306d8b23451bad367b1809f38bcd P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:13.992523 14823 log.cc:1198] T 5883306d8b23451bad367b1809f38bcd P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/5883306d8b23451bad367b1809f38bcd
I20250115 03:06:13.992717 14809 tablet_bootstrap.cc:654] T 0c969b652a564ed18fd41bebee47533c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:13.993659 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2d3157859e684da98bb473d5799b24a6 consensus state has no leader
W20250115 03:06:13.994433 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 543bbb3ae87c46a6982e6fa1ae325db6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 543bbb3ae87c46a6982e6fa1ae325db6 already in progress: creating tablet
W20250115 03:06:13.994560 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 2fcce3505b744f668c5355aa982c8485 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2fcce3505b744f668c5355aa982c8485 already in progress: creating tablet
W20250115 03:06:13.995561 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet bcf6745e549d47cdbdf97c2dff6997ad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bcf6745e549d47cdbdf97c2dff6997ad already in progress: creating tablet
W20250115 03:06:13.995651 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet ef002211ee1c4a149232c8e433277add because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ef002211ee1c4a149232c8e433277add already in progress: creating tablet
I20250115 03:06:13.996444 14529 tablet_service.cc:1514] Processing DeleteTablet for tablet ef002211ee1c4a149232c8e433277add with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:13.996711 14527 tablet_service.cc:1467] Processing CreateTablet for tablet a1cbe29c08c7477e8944b7e108d46e7a (DEFAULT_TABLE table=default.Table_7e53fa73287c43e3a9a0e65908b42593 [id=1f983a10a8704e22b8a2b96ef4e06fbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.996826 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 89d92f59c7424dfc95427329200455e7 (DEFAULT_TABLE table=default.Table_236770d2959743108285eae378066e45 [id=fec89f498f5046839fe491caa3367a36]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:13.997252 14812 tablet_bootstrap.cc:492] T 2fcce3505b744f668c5355aa982c8485 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:13.997325 14812 ts_tablet_manager.cc:1397] T 2fcce3505b744f668c5355aa982c8485 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.045s user 0.002s sys 0.000s
I20250115 03:06:13.997512 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1cbe29c08c7477e8944b7e108d46e7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:13.997802 14812 raft_consensus.cc:357] T 2fcce3505b744f668c5355aa982c8485 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:13.997893 14812 raft_consensus.cc:383] T 2fcce3505b744f668c5355aa982c8485 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:13.997924 14812 raft_consensus.cc:738] T 2fcce3505b744f668c5355aa982c8485 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:13.998029 14812 consensus_queue.cc:260] T 2fcce3505b744f668c5355aa982c8485 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:13.998281 14812 ts_tablet_manager.cc:1428] T 2fcce3505b744f668c5355aa982c8485 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:13.998454 14812 tablet_bootstrap.cc:492] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:13.999318 14812 tablet_bootstrap.cc:654] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.000355 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89d92f59c7424dfc95427329200455e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.002956 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 3e46d4a75e9b4ca8b83a188d06280fcd (DEFAULT_TABLE table=default.Table_0d7244964d874f5eb1f508e5c0dabf1a [id=15762a37e62f4e89852b2eb66a008dbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.003835 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e46d4a75e9b4ca8b83a188d06280fcd. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:13.997332 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 543bbb3ae87c46a6982e6fa1ae325db6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 543bbb3ae87c46a6982e6fa1ae325db6 already in progress: creating tablet
W20250115 03:06:14.004344 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 2fcce3505b744f668c5355aa982c8485 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2fcce3505b744f668c5355aa982c8485 already in progress: creating tablet
W20250115 03:06:14.004474 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet bcf6745e549d47cdbdf97c2dff6997ad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bcf6745e549d47cdbdf97c2dff6997ad already in progress: creating tablet
W20250115 03:06:14.004580 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet ef002211ee1c4a149232c8e433277add because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ef002211ee1c4a149232c8e433277add already in progress: creating tablet
I20250115 03:06:14.008378 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet 8c21c254cdef4015bfaba3fb5d61f554 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:14.009207 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet c6771c98e569468ea932882858981d23 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:14.010067 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b2210a8bd9946508870844ee2e88fb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 55f01d704b174b2ea108ea6f1df763df) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.010210 14820 tablet_replica.cc:331] T 7b2210a8bd9946508870844ee2e88fb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.010334 14820 raft_consensus.cc:2238] T 7b2210a8bd9946508870844ee2e88fb1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.010457 14820 raft_consensus.cc:2267] T 7b2210a8bd9946508870844ee2e88fb1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.010905 14820 ts_tablet_manager.cc:1905] T 7b2210a8bd9946508870844ee2e88fb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.011646 14823 ts_tablet_manager.cc:1939] T 5883306d8b23451bad367b1809f38bcd P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.012755 14820 ts_tablet_manager.cc:1918] T 7b2210a8bd9946508870844ee2e88fb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.012825 14820 log.cc:1198] T 7b2210a8bd9946508870844ee2e88fb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/7b2210a8bd9946508870844ee2e88fb1
I20250115 03:06:14.013054 14820 ts_tablet_manager.cc:1939] T 7b2210a8bd9946508870844ee2e88fb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.013617 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 7b2210a8bd9946508870844ee2e88fb1 (table default.Table_64d7a4f0d2694a65a7fb86e55e1255e0 [id=9dcc0154ed914304a15cf96049afaf52]) successfully deleted
I20250115 03:06:14.014639 14535 tablet_service.cc:1467] Processing CreateTablet for tablet f01c319ebd5148858de9ea1a608ddff3 (DEFAULT_TABLE table=default.Table_ff21d7bc463d43cfa3330f7288ba7a55 [id=5de066e9f2174fff9303cba23eeec988]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.014839 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f01c319ebd5148858de9ea1a608ddff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.015542 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 8c21c254cdef4015bfaba3fb5d61f554
I20250115 03:06:14.015792 14169 catalog_manager.cc:6596] ReplaceTablet: tablet 8c21c254cdef4015bfaba3fb5d61f554 deleted and replaced by tablet 17b5e362481c49d2a3a13b1550ceef23
I20250115 03:06:14.016034 15153 raft_consensus.cc:491] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.016139 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet c6771c98e569468ea932882858981d23
I20250115 03:06:14.016106 15153 raft_consensus.cc:513] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.016361 14168 catalog_manager.cc:6596] ReplaceTablet: tablet c6771c98e569468ea932882858981d23 deleted and replaced by tablet 22cc0833bfde458ebe3985c05d29b8e6
I20250115 03:06:14.016427 15153 leader_election.cc:290] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:14.016752 14394 tablet_service.cc:1467] Processing CreateTablet for tablet f01c319ebd5148858de9ea1a608ddff3 (DEFAULT_TABLE table=default.Table_ff21d7bc463d43cfa3330f7288ba7a55 [id=5de066e9f2174fff9303cba23eeec988]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.017004 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f01c319ebd5148858de9ea1a608ddff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.017344 14530 tablet_service.cc:1467] Processing CreateTablet for tablet e368f87556944c46a6d8f160ead17ac9 (DEFAULT_TABLE table=default.Table_2af92748648345c0903a0f99da0d47c5 [id=19efb3734d824699a088ad62e69095f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.017547 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e368f87556944c46a6d8f160ead17ac9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.019042 14401 tablet_service.cc:1467] Processing CreateTablet for tablet e368f87556944c46a6d8f160ead17ac9 (DEFAULT_TABLE table=default.Table_2af92748648345c0903a0f99da0d47c5 [id=19efb3734d824699a088ad62e69095f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.019237 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e368f87556944c46a6d8f160ead17ac9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.019294 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21cc97960f7c49c7bf57499e66fd7190" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.019402 14422 raft_consensus.cc:2463] T 21cc97960f7c49c7bf57499e66fd7190 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:14.019655 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 5883306d8b23451bad367b1809f38bcd (table default.Table_0bdf66c6040b4709b0a6be57188b9d86 [id=c8fdb0f9f9064f3eb8d19f4adf3c439f]) successfully deleted
I20250115 03:06:14.019686 14664 tablet_service.cc:1467] Processing CreateTablet for tablet f01c319ebd5148858de9ea1a608ddff3 (DEFAULT_TABLE table=default.Table_ff21d7bc463d43cfa3330f7288ba7a55 [id=5de066e9f2174fff9303cba23eeec988]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.019851 14823 tablet_replica.cc:331] T 543bbb3ae87c46a6982e6fa1ae325db6 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.019897 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f01c319ebd5148858de9ea1a608ddff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.019979 14823 raft_consensus.cc:2238] T 543bbb3ae87c46a6982e6fa1ae325db6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.020068 14823 raft_consensus.cc:2267] T 543bbb3ae87c46a6982e6fa1ae325db6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.020629 14823 ts_tablet_manager.cc:1905] T 543bbb3ae87c46a6982e6fa1ae325db6 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.021063 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21cc97960f7c49c7bf57499e66fd7190" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.021160 14557 raft_consensus.cc:2463] T 21cc97960f7c49c7bf57499e66fd7190 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:14.020903 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c21c254cdef4015bfaba3fb5d61f554 with delete_type TABLET_DATA_DELETED (replaced by tablet 17b5e362481c49d2a3a13b1550ceef23) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.020982 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet c6771c98e569468ea932882858981d23 with delete_type TABLET_DATA_DELETED (replaced by tablet 22cc0833bfde458ebe3985c05d29b8e6) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.021446 14626 leader_election.cc:304] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:14.021602 15153 raft_consensus.cc:2798] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.021643 15153 raft_consensus.cc:491] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.021678 15153 raft_consensus.cc:3054] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.021894 14663 tablet_service.cc:1467] Processing CreateTablet for tablet e368f87556944c46a6d8f160ead17ac9 (DEFAULT_TABLE table=default.Table_2af92748648345c0903a0f99da0d47c5 [id=19efb3734d824699a088ad62e69095f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.022075 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e368f87556944c46a6d8f160ead17ac9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.022483 15153 raft_consensus.cc:513] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.022805 15153 leader_election.cc:290] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:14.023442 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21cc97960f7c49c7bf57499e66fd7190" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:14.023533 14557 raft_consensus.cc:3054] T 21cc97960f7c49c7bf57499e66fd7190 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.023754 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c21c254cdef4015bfaba3fb5d61f554 with delete_type TABLET_DATA_DELETED (replaced by tablet 17b5e362481c49d2a3a13b1550ceef23) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.023855 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet c6771c98e569468ea932882858981d23 with delete_type TABLET_DATA_DELETED (replaced by tablet 22cc0833bfde458ebe3985c05d29b8e6) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.023950 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet c6771c98e569468ea932882858981d23 with delete_type TABLET_DATA_DELETED (replaced by tablet 22cc0833bfde458ebe3985c05d29b8e6) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.024325 14557 raft_consensus.cc:2463] T 21cc97960f7c49c7bf57499e66fd7190 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:14.024626 14626 leader_election.cc:304] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:14.024786 14811 tablet_bootstrap.cc:492] T ae0efe52ac6c40fbae45c725b07b03a3 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.024824 15153 raft_consensus.cc:2798] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:14.024859 14811 ts_tablet_manager.cc:1397] T ae0efe52ac6c40fbae45c725b07b03a3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.073s user 0.000s sys 0.002s
I20250115 03:06:14.024860 14885 raft_consensus.cc:491] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.024899 15153 raft_consensus.cc:695] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:14.024969 14885 raft_consensus.cc:513] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.025031 15153 consensus_queue.cc:237] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.025321 14885 leader_election.cc:290] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.025308 14811 raft_consensus.cc:357] T ae0efe52ac6c40fbae45c725b07b03a3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.025396 14811 raft_consensus.cc:383] T ae0efe52ac6c40fbae45c725b07b03a3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.025435 14819 tablet_replica.cc:331] T 8c21c254cdef4015bfaba3fb5d61f554 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.025444 14811 raft_consensus.cc:738] T ae0efe52ac6c40fbae45c725b07b03a3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.025552 14819 raft_consensus.cc:2238] T 8c21c254cdef4015bfaba3fb5d61f554 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.025564 14811 consensus_queue.cc:260] T ae0efe52ac6c40fbae45c725b07b03a3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.025640 14819 raft_consensus.cc:2267] T 8c21c254cdef4015bfaba3fb5d61f554 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.025820 14811 ts_tablet_manager.cc:1428] T ae0efe52ac6c40fbae45c725b07b03a3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:14.025986 14811 tablet_bootstrap.cc:492] T 0c969b652a564ed18fd41bebee47533c P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.026841 14811 tablet_bootstrap.cc:654] T 0c969b652a564ed18fd41bebee47533c P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.027387 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8c3406d44a74b20a4ef4b42e8d04ad0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:14.027490 14689 raft_consensus.cc:2463] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:14.028324 14819 ts_tablet_manager.cc:1905] T 8c21c254cdef4015bfaba3fb5d61f554 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.029047 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8c3406d44a74b20a4ef4b42e8d04ad0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.029150 14557 raft_consensus.cc:2463] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:14.029816 14811 tablet_bootstrap.cc:492] T 0c969b652a564ed18fd41bebee47533c P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.029865 14358 leader_election.cc:304] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:14.029906 14811 ts_tablet_manager.cc:1397] T 0c969b652a564ed18fd41bebee47533c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:14.030103 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_c35ae9375d8c41f988ccbd9403698a8a" } new_table_name: "default.Table_cab1959a159a46a6b86daad3647b89b3" modify_external_catalogs: true
I20250115 03:06:14.030215 14819 ts_tablet_manager.cc:1918] T 8c21c254cdef4015bfaba3fb5d61f554 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:14.030275 14819 log.cc:1198] T 8c21c254cdef4015bfaba3fb5d61f554 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/8c21c254cdef4015bfaba3fb5d61f554
I20250115 03:06:14.030296 14811 raft_consensus.cc:357] T 0c969b652a564ed18fd41bebee47533c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.030393 14811 raft_consensus.cc:383] T 0c969b652a564ed18fd41bebee47533c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.030431 14811 raft_consensus.cc:738] T 0c969b652a564ed18fd41bebee47533c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.030521 14819 ts_tablet_manager.cc:1939] T 8c21c254cdef4015bfaba3fb5d61f554 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.030547 14811 consensus_queue.cc:260] T 0c969b652a564ed18fd41bebee47533c P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.030830 14811 ts_tablet_manager.cc:1428] T 0c969b652a564ed18fd41bebee47533c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:14.030987 14819 tablet_replica.cc:331] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.030993 14811 tablet_bootstrap.cc:492] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.031095 14819 raft_consensus.cc:2238] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:14.031260 14819 raft_consensus.cc:2267] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.031894 14811 tablet_bootstrap.cc:654] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.032205 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21cc97960f7c49c7bf57499e66fd7190" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:14.032310 14422 raft_consensus.cc:3054] T 21cc97960f7c49c7bf57499e66fd7190 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.032624 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0dd17013f6334efe97618f0b520512c5" } new_table_name: "default.Table_a6170a1e9f9546abb2af9ba24456080d" modify_external_catalogs: true
I20250115 03:06:14.032682 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 8c21c254cdef4015bfaba3fb5d61f554 (table default.Table_2e115074bf7049bfbb1239b4d9c1ca2e [id=6cf622de00cb490bb794837223aa7bcb]) successfully deleted
I20250115 03:06:14.033010 14811 tablet_bootstrap.cc:492] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.033088 14811 ts_tablet_manager.cc:1397] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:14.033135 14422 raft_consensus.cc:2463] T 21cc97960f7c49c7bf57499e66fd7190 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:14.033367 14885 raft_consensus.cc:2798] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.033423 14885 raft_consensus.cc:491] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.033473 14885 raft_consensus.cc:3054] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.033571 14820 tablet_replica.cc:331] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.033771 14820 raft_consensus.cc:2238] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.033861 14820 raft_consensus.cc:2267] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.034170 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c21c254cdef4015bfaba3fb5d61f554 with delete_type TABLET_DATA_DELETED (replaced by tablet 17b5e362481c49d2a3a13b1550ceef23) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.034284 14885 raft_consensus.cc:513] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.034341 14820 ts_tablet_manager.cc:1905] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.034528 14819 ts_tablet_manager.cc:1905] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.034602 14885 leader_election.cc:290] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.034874 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8c3406d44a74b20a4ef4b42e8d04ad0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:14.034971 14557 raft_consensus.cc:3054] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.035768 14557 raft_consensus.cc:2463] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:14.035982 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8c3406d44a74b20a4ef4b42e8d04ad0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:14.036083 14689 raft_consensus.cc:3054] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.036192 14820 ts_tablet_manager.cc:1918] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:14.036224 14819 ts_tablet_manager.cc:1918] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:14.036252 14820 log.cc:1198] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/c6771c98e569468ea932882858981d23
I20250115 03:06:14.036276 14819 log.cc:1198] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/c6771c98e569468ea932882858981d23
I20250115 03:06:14.036368 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_438b02457c07420fa9969504491d9c31" } new_table_name: "default.Table_5e6d1abc27d042dba8506da7b49a6ee3" modify_external_catalogs: true
I20250115 03:06:14.036473 14820 ts_tablet_manager.cc:1939] T c6771c98e569468ea932882858981d23 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.036473 14819 ts_tablet_manager.cc:1939] T c6771c98e569468ea932882858981d23 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.037015 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet c6771c98e569468ea932882858981d23 (table default.Table_3db38338e5b04298954aded6d5482169 [id=f56a0de6e56441bfa620902f175feb4a]) successfully deleted
I20250115 03:06:14.037101 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_638e39bcf2aa4131ac57afbe63ac9b73"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:14.037314 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet c6771c98e569468ea932882858981d23 (table default.Table_3db38338e5b04298954aded6d5482169 [id=f56a0de6e56441bfa620902f175feb4a]) successfully deleted
W20250115 03:06:14.037401 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_638e39bcf2aa4131ac57afbe63ac9b73 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.038077 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_dbe8351777e748e8aceef2208bd425ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.038348 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dbe8351777e748e8aceef2208bd425ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:14.038769 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 481705dc9ae841659520a6c909879f9b consensus state has no leader
I20250115 03:06:14.039203 14162 catalog_manager.cc:5526] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:14.040208 14809 tablet_bootstrap.cc:492] T 0c969b652a564ed18fd41bebee47533c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.040282 14809 ts_tablet_manager.cc:1397] T 0c969b652a564ed18fd41bebee47533c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.048s user 0.001s sys 0.000s
I20250115 03:06:14.040395 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 2379bae3690c44949e658dde84c1712e (DEFAULT_TABLE table=default.Table_ebca4429c164491c974b3ee244c3b1de [id=01a92c882c3640dcbc950b3e76f97c4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.040603 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2379bae3690c44949e658dde84c1712e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.040674 14809 raft_consensus.cc:357] T 0c969b652a564ed18fd41bebee47533c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.040773 14809 raft_consensus.cc:383] T 0c969b652a564ed18fd41bebee47533c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.040802 14809 raft_consensus.cc:738] T 0c969b652a564ed18fd41bebee47533c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.040910 14809 consensus_queue.cc:260] T 0c969b652a564ed18fd41bebee47533c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.041198 14809 ts_tablet_manager.cc:1428] T 0c969b652a564ed18fd41bebee47533c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:14.041582 14812 tablet_bootstrap.cc:492] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.041661 14812 ts_tablet_manager.cc:1397] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.043s user 0.000s sys 0.001s
I20250115 03:06:14.042016 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 2379bae3690c44949e658dde84c1712e (DEFAULT_TABLE table=default.Table_ebca4429c164491c974b3ee244c3b1de [id=01a92c882c3640dcbc950b3e76f97c4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.042086 14812 raft_consensus.cc:357] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.042176 14812 raft_consensus.cc:383] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.042208 14812 raft_consensus.cc:738] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.042310 14812 consensus_queue.cc:260] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.042388 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 99d9180ed9b24363a4c44d01ea1d6e9b (DEFAULT_TABLE table=default.Table_6bb3a06c1ae749e593cdc3f2b3992eaf [id=89578bd1e7e64c0fa550f20f592ccc6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.042563 14812 ts_tablet_manager.cc:1428] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.042734 14812 tablet_bootstrap.cc:492] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.042860 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 99d9180ed9b24363a4c44d01ea1d6e9b (DEFAULT_TABLE table=default.Table_6bb3a06c1ae749e593cdc3f2b3992eaf [id=89578bd1e7e64c0fa550f20f592ccc6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.036916 14820 tablet_replica.cc:331] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.043517 14820 raft_consensus.cc:2238] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:14.043593 14812 tablet_bootstrap.cc:654] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.043674 14820 raft_consensus.cc:2267] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.044111 14820 ts_tablet_manager.cc:1905] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.045032 14356 leader_election.cc:304] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:14.033465 14811 raft_consensus.cc:357] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.045970 14811 raft_consensus.cc:383] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.046002 14811 raft_consensus.cc:738] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.046100 14811 consensus_queue.cc:260] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.046837 14811 ts_tablet_manager.cc:1428] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.014s user 0.001s sys 0.000s
I20250115 03:06:14.046975 14811 tablet_bootstrap.cc:492] T 5883306d8b23451bad367b1809f38bcd P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.048270 14811 tablet_bootstrap.cc:654] T 5883306d8b23451bad367b1809f38bcd P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.048755 14820 ts_tablet_manager.cc:1918] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:14.048756 14689 raft_consensus.cc:2463] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:14.048815 14820 log.cc:1198] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/8c21c254cdef4015bfaba3fb5d61f554
I20250115 03:06:14.049041 14820 ts_tablet_manager.cc:1939] T 8c21c254cdef4015bfaba3fb5d61f554 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.049378 14809 tablet_bootstrap.cc:492] T 5883306d8b23451bad367b1809f38bcd P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.050266 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99d9180ed9b24363a4c44d01ea1d6e9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.050290 14809 tablet_bootstrap.cc:654] T 5883306d8b23451bad367b1809f38bcd P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.052436 14167 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 0c969b652a564ed18fd41bebee47533c (table default.Table_a730bc55668645acadbaa47b450a5338 [id=e31e6fb5a7e941c494ef416ef4762fb7]). Expected version 1 got 0
I20250115 03:06:14.053520 14885 raft_consensus.cc:2798] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:14.053606 14885 raft_consensus.cc:695] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:14.053548 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_8750bca4a0104d30babab0f0c0a8c8dc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:14.053709 14885 consensus_queue.cc:237] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:14.053869 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8750bca4a0104d30babab0f0c0a8c8dc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.054473 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 0c969b652a564ed18fd41bebee47533c (table default.Table_a730bc55668645acadbaa47b450a5338 [id=e31e6fb5a7e941c494ef416ef4762fb7]). Expected version 1 got 0
I20250115 03:06:14.055717 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 2379bae3690c44949e658dde84c1712e (DEFAULT_TABLE table=default.Table_ebca4429c164491c974b3ee244c3b1de [id=01a92c882c3640dcbc950b3e76f97c4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.055938 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2379bae3690c44949e658dde84c1712e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.056475 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 8c21c254cdef4015bfaba3fb5d61f554 (table default.Table_2e115074bf7049bfbb1239b4d9c1ca2e [id=6cf622de00cb490bb794837223aa7bcb]) successfully deleted
I20250115 03:06:14.057334 14817 raft_consensus.cc:491] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.057297 14169 catalog_manager.cc:5526] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:14.057415 14817 raft_consensus.cc:513] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.057724 14817 leader_election.cc:290] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.057854 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 99d9180ed9b24363a4c44d01ea1d6e9b (DEFAULT_TABLE table=default.Table_6bb3a06c1ae749e593cdc3f2b3992eaf [id=89578bd1e7e64c0fa550f20f592ccc6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.058053 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99d9180ed9b24363a4c44d01ea1d6e9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.058410 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2379bae3690c44949e658dde84c1712e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.058583 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72f9b43975074ac5a7012b132ef74662" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:14.058415 14823 ts_tablet_manager.cc:1918] T 543bbb3ae87c46a6982e6fa1ae325db6 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.058698 14689 raft_consensus.cc:2463] T 72f9b43975074ac5a7012b132ef74662 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:14.058959 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72f9b43975074ac5a7012b132ef74662" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.059055 14422 raft_consensus.cc:2463] T 72f9b43975074ac5a7012b132ef74662 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:14.058765 14823 log.cc:1198] T 543bbb3ae87c46a6982e6fa1ae325db6 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/543bbb3ae87c46a6982e6fa1ae325db6
I20250115 03:06:14.059028 14400 tablet_service.cc:1467] Processing CreateTablet for tablet f3ac7f1e1bfb4383895dc2c2123a743b (DEFAULT_TABLE table=default.Table_f7b803cd6e0546c394296cb6b4b2dc83 [id=d4a8db3fb58b46cda80c8e652dbd33b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.059175 14493 leader_election.cc:304] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:14.059293 14823 ts_tablet_manager.cc:1939] T 543bbb3ae87c46a6982e6fa1ae325db6 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.059302 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3ac7f1e1bfb4383895dc2c2123a743b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.059319 14817 raft_consensus.cc:2798] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.059357 14817 raft_consensus.cc:491] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.059391 14817 raft_consensus.cc:3054] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.060178 14817 raft_consensus.cc:513] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.060469 14817 leader_election.cc:290] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.060901 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99d9180ed9b24363a4c44d01ea1d6e9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.061034 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 22cc0833bfde458ebe3985c05d29b8e6 (DEFAULT_TABLE table=default.Table_3db38338e5b04298954aded6d5482169 [id=f56a0de6e56441bfa620902f175feb4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.061215 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22cc0833bfde458ebe3985c05d29b8e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.061648 14165 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet d4f770f63a7348afb71904b93c68c379 (table default.Table_117c4a2ba1cb4369881c9bc30c21781b [id=50c73c26b4ed47a4961348db433b7dd4]). Expected version 1 got 0
I20250115 03:06:14.061748 14165 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 0c969b652a564ed18fd41bebee47533c (table default.Table_a730bc55668645acadbaa47b450a5338 [id=e31e6fb5a7e941c494ef416ef4762fb7]). Expected version 1 got 0
I20250115 03:06:14.062350 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72f9b43975074ac5a7012b132ef74662" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:14.062439 14689 raft_consensus.cc:3054] T 72f9b43975074ac5a7012b132ef74662 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.062978 14400 tablet_service.cc:1467] Processing CreateTablet for tablet a2fc04950cfe4c608a4712afd257ca76 (DEFAULT_TABLE table=default.Table_d9b79701501644caa5b8be7be1d354df [id=8f811bda53fc4d76a88a3dcf69aa3dc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.063089 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 22cc0833bfde458ebe3985c05d29b8e6 (DEFAULT_TABLE table=default.Table_3db38338e5b04298954aded6d5482169 [id=f56a0de6e56441bfa620902f175feb4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.063165 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2fc04950cfe4c608a4712afd257ca76. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.063235 14689 raft_consensus.cc:2463] T 72f9b43975074ac5a7012b132ef74662 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:14.063275 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22cc0833bfde458ebe3985c05d29b8e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.063506 14493 leader_election.cc:304] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:14.063861 14817 raft_consensus.cc:2798] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:14.063938 14817 raft_consensus.cc:695] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:14.064045 14817 consensus_queue.cc:237] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:14.064113 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 382116b1fb6a4f888ab1d55c29917ec8 consensus state has no leader
I20250115 03:06:14.065192 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72f9b43975074ac5a7012b132ef74662" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:14.065285 14422 raft_consensus.cc:3054] T 72f9b43975074ac5a7012b132ef74662 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.065613 14663 tablet_service.cc:1467] Processing CreateTablet for tablet f3ac7f1e1bfb4383895dc2c2123a743b (DEFAULT_TABLE table=default.Table_f7b803cd6e0546c394296cb6b4b2dc83 [id=d4a8db3fb58b46cda80c8e652dbd33b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.065806 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3ac7f1e1bfb4383895dc2c2123a743b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.066061 14422 raft_consensus.cc:2463] T 72f9b43975074ac5a7012b132ef74662 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:14.066972 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a730bc55668645acadbaa47b450a5338" } modify_external_catalogs: true
I20250115 03:06:14.067062 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a730bc55668645acadbaa47b450a5338" } modify_external_catalogs: true
I20250115 03:06:14.067776 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 22cc0833bfde458ebe3985c05d29b8e6 (DEFAULT_TABLE table=default.Table_3db38338e5b04298954aded6d5482169 [id=f56a0de6e56441bfa620902f175feb4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.067953 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22cc0833bfde458ebe3985c05d29b8e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.067927 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet dca04d171deb488690ea58bd75995021 (table default.Table_3bd81cb8bc2d4b74ae3dbcbdb74aae8b [id=05f9fd2ebec34d2299ea59c47a74c828]). Expected version 4 got 0
W20250115 03:06:14.068198 14162 catalog_manager.cc:5718] Failed to send AlterTable RPC for tablet dca04d171deb488690ea58bd75995021 (table default.Table_3bd81cb8bc2d4b74ae3dbcbdb74aae8b [id=05f9fd2ebec34d2299ea59c47a74c828], current schema version=4): Not found: failed to reset TS proxy: No leader found: Tablet dca04d171deb488690ea58bd75995021 consensus state has no leader
I20250115 03:06:14.068878 14663 tablet_service.cc:1467] Processing CreateTablet for tablet a2fc04950cfe4c608a4712afd257ca76 (DEFAULT_TABLE table=default.Table_d9b79701501644caa5b8be7be1d354df [id=8f811bda53fc4d76a88a3dcf69aa3dc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.069090 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2fc04950cfe4c608a4712afd257ca76. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.072312 15151 raft_consensus.cc:491] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.072394 15151 raft_consensus.cc:513] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.072654 14527 tablet_service.cc:1467] Processing CreateTablet for tablet f3ac7f1e1bfb4383895dc2c2123a743b (DEFAULT_TABLE table=default.Table_f7b803cd6e0546c394296cb6b4b2dc83 [id=d4a8db3fb58b46cda80c8e652dbd33b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.072734 15151 leader_election.cc:290] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.072856 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3ac7f1e1bfb4383895dc2c2123a743b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.073140 14535 tablet_service.cc:1467] Processing CreateTablet for tablet a2fc04950cfe4c608a4712afd257ca76 (DEFAULT_TABLE table=default.Table_d9b79701501644caa5b8be7be1d354df [id=8f811bda53fc4d76a88a3dcf69aa3dc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.073338 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c969b652a564ed18fd41bebee47533c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.073304 14164 catalog_manager.cc:5526] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:14.073446 14557 raft_consensus.cc:2463] T 0c969b652a564ed18fd41bebee47533c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:14.074231 15299 tablet.cc:1721] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:14.074596 14626 leader_election.cc:304] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:14.074765 15151 raft_consensus.cc:2798] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.074826 15151 raft_consensus.cc:491] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.074862 15151 raft_consensus.cc:3054] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.075662 15151 raft_consensus.cc:513] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.075986 15151 leader_election.cc:290] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.076222 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c969b652a564ed18fd41bebee47533c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:14.076314 14421 raft_consensus.cc:3054] T 0c969b652a564ed18fd41bebee47533c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.076851 15274 tablet.cc:1721] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:14.076972 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c969b652a564ed18fd41bebee47533c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:14.077066 14557 raft_consensus.cc:3054] T 0c969b652a564ed18fd41bebee47533c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.077129 14421 raft_consensus.cc:2463] T 0c969b652a564ed18fd41bebee47533c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:14.077359 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_24cbe6b962b248a9b340506b63c41b20"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.077636 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_24cbe6b962b248a9b340506b63c41b20 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.077850 14557 raft_consensus.cc:2463] T 0c969b652a564ed18fd41bebee47533c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:14.078079 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 543bbb3ae87c46a6982e6fa1ae325db6 (table default.Table_9432efe75c694779976428f3dea16679 [id=98a21757d0884730ae2a1bbc97359df6]) successfully deleted
I20250115 03:06:14.078107 14626 leader_election.cc:304] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:14.078241 15151 raft_consensus.cc:2798] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:14.078305 15151 raft_consensus.cc:695] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:14.078409 15151 consensus_queue.cc:237] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.079159 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_bd6a79400bc244d69676d8df1069cbb2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.083491 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd6a79400bc244d69676d8df1069cbb2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.079939 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c969b652a564ed18fd41bebee47533c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.083642 14422 raft_consensus.cc:2371] T 0c969b652a564ed18fd41bebee47533c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1. Re-sending same reply.
I20250115 03:06:14.081588 14969 tablet.cc:1721] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:14.073326 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2fc04950cfe4c608a4712afd257ca76. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.082222 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 0c969b652a564ed18fd41bebee47533c
I20250115 03:06:14.087478 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c969b652a564ed18fd41bebee47533c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.087666 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c969b652a564ed18fd41bebee47533c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.087946 14823 tablet_replica.cc:331] T 2fcce3505b744f668c5355aa982c8485 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.088065 14823 raft_consensus.cc:2238] T 2fcce3505b744f668c5355aa982c8485 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.088146 14823 raft_consensus.cc:2267] T 2fcce3505b744f668c5355aa982c8485 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.089329 14819 tablet_replica.cc:331] T 0c969b652a564ed18fd41bebee47533c P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.089300 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 17b5e362481c49d2a3a13b1550ceef23 (DEFAULT_TABLE table=default.Table_2e115074bf7049bfbb1239b4d9c1ca2e [id=6cf622de00cb490bb794837223aa7bcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.089437 14819 raft_consensus.cc:2238] T 0c969b652a564ed18fd41bebee47533c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.089495 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17b5e362481c49d2a3a13b1550ceef23. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.089530 14819 raft_consensus.cc:2267] T 0c969b652a564ed18fd41bebee47533c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.089588 14823 ts_tablet_manager.cc:1905] T 2fcce3505b744f668c5355aa982c8485 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.090595 14167 master_service.cc:946] ReplaceTablet: received request to replace tablet e3302d62a34e43d09c11abca292d9110 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:14.090834 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c969b652a564ed18fd41bebee47533c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.091331 14820 tablet_replica.cc:331] T 0c969b652a564ed18fd41bebee47533c P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.091449 14820 raft_consensus.cc:2238] T 0c969b652a564ed18fd41bebee47533c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.091529 14820 raft_consensus.cc:2267] T 0c969b652a564ed18fd41bebee47533c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.091897 14820 ts_tablet_manager.cc:1905] T 0c969b652a564ed18fd41bebee47533c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.092751 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_a6170a1e9f9546abb2af9ba24456080d [id=488a9a6268f64669ae17e1a67caa73a5] alter complete (version 3)
I20250115 03:06:14.093603 14820 ts_tablet_manager.cc:1918] T 0c969b652a564ed18fd41bebee47533c P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.093665 14820 log.cc:1198] T 0c969b652a564ed18fd41bebee47533c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/0c969b652a564ed18fd41bebee47533c
I20250115 03:06:14.093852 14820 ts_tablet_manager.cc:1939] T 0c969b652a564ed18fd41bebee47533c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.095113 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 0c969b652a564ed18fd41bebee47533c (table default.Table_a730bc55668645acadbaa47b450a5338 [id=e31e6fb5a7e941c494ef416ef4762fb7]) successfully deleted
I20250115 03:06:14.098757 14819 ts_tablet_manager.cc:1905] T 0c969b652a564ed18fd41bebee47533c P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.100647 14819 ts_tablet_manager.cc:1918] T 0c969b652a564ed18fd41bebee47533c P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.100713 14819 log.cc:1198] T 0c969b652a564ed18fd41bebee47533c P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/0c969b652a564ed18fd41bebee47533c
I20250115 03:06:14.100896 14819 ts_tablet_manager.cc:1939] T 0c969b652a564ed18fd41bebee47533c P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.102968 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet e3302d62a34e43d09c11abca292d9110
I20250115 03:06:14.103246 14167 catalog_manager.cc:6596] ReplaceTablet: tablet e3302d62a34e43d09c11abca292d9110 deleted and replaced by tablet a4484651d964411e90ed93c6caab0a13
I20250115 03:06:14.103358 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet e3302d62a34e43d09c11abca292d9110 with delete_type TABLET_DATA_DELETED (replaced by tablet a4484651d964411e90ed93c6caab0a13) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.103582 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 0c969b652a564ed18fd41bebee47533c (table default.Table_a730bc55668645acadbaa47b450a5338 [id=e31e6fb5a7e941c494ef416ef4762fb7]) successfully deleted
W20250115 03:06:14.103705 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet e3302d62a34e43d09c11abca292d9110 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e3302d62a34e43d09c11abca292d9110 already in progress: creating tablet
I20250115 03:06:14.103996 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet e3302d62a34e43d09c11abca292d9110 with delete_type TABLET_DATA_DELETED (replaced by tablet a4484651d964411e90ed93c6caab0a13) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.104113 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet e3302d62a34e43d09c11abca292d9110 with delete_type TABLET_DATA_DELETED (replaced by tablet a4484651d964411e90ed93c6caab0a13) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:14.104384 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet e3302d62a34e43d09c11abca292d9110 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e3302d62a34e43d09c11abca292d9110 already in progress: creating tablet
I20250115 03:06:14.105531 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 17b5e362481c49d2a3a13b1550ceef23 (DEFAULT_TABLE table=default.Table_2e115074bf7049bfbb1239b4d9c1ca2e [id=6cf622de00cb490bb794837223aa7bcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.105746 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17b5e362481c49d2a3a13b1550ceef23. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.107522 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 17b5e362481c49d2a3a13b1550ceef23 (DEFAULT_TABLE table=default.Table_2e115074bf7049bfbb1239b4d9c1ca2e [id=6cf622de00cb490bb794837223aa7bcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.107720 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17b5e362481c49d2a3a13b1550ceef23. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.109405 14823 ts_tablet_manager.cc:1918] T 2fcce3505b744f668c5355aa982c8485 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.109477 14823 log.cc:1198] T 2fcce3505b744f668c5355aa982c8485 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/2fcce3505b744f668c5355aa982c8485
I20250115 03:06:14.112016 14823 ts_tablet_manager.cc:1939] T 2fcce3505b744f668c5355aa982c8485 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.112610 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 2fcce3505b744f668c5355aa982c8485 (table default.Table_2af92748648345c0903a0f99da0d47c5 [id=19efb3734d824699a088ad62e69095f3]) successfully deleted
W20250115 03:06:14.113024 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet bcf6745e549d47cdbdf97c2dff6997ad because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bcf6745e549d47cdbdf97c2dff6997ad already in progress: creating tablet
W20250115 03:06:14.113413 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet ef002211ee1c4a149232c8e433277add because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ef002211ee1c4a149232c8e433277add already in progress: creating tablet
I20250115 03:06:14.113617 14809 tablet_bootstrap.cc:492] T 5883306d8b23451bad367b1809f38bcd P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.113611 14823 tablet_replica.cc:331] T 8c21c254cdef4015bfaba3fb5d61f554 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.113704 14809 ts_tablet_manager.cc:1397] T 5883306d8b23451bad367b1809f38bcd P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.064s user 0.000s sys 0.001s
I20250115 03:06:14.113735 14823 raft_consensus.cc:2238] T 8c21c254cdef4015bfaba3fb5d61f554 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.113831 14823 raft_consensus.cc:2267] T 8c21c254cdef4015bfaba3fb5d61f554 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.114151 14809 raft_consensus.cc:357] T 5883306d8b23451bad367b1809f38bcd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.114259 14809 raft_consensus.cc:383] T 5883306d8b23451bad367b1809f38bcd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.114298 14809 raft_consensus.cc:738] T 5883306d8b23451bad367b1809f38bcd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.114413 14809 consensus_queue.cc:260] T 5883306d8b23451bad367b1809f38bcd P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.114697 14809 ts_tablet_manager.cc:1428] T 5883306d8b23451bad367b1809f38bcd P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20250115 03:06:14.114979 14809 tablet_bootstrap.cc:492] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.115897 14809 tablet_bootstrap.cc:654] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.118191 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 5883306d8b23451bad367b1809f38bcd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.119176 14819 tablet_replica.cc:331] T 5883306d8b23451bad367b1809f38bcd P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.119304 14819 raft_consensus.cc:2238] T 5883306d8b23451bad367b1809f38bcd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.119400 14819 raft_consensus.cc:2267] T 5883306d8b23451bad367b1809f38bcd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.119899 14823 ts_tablet_manager.cc:1905] T 8c21c254cdef4015bfaba3fb5d61f554 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.121475 14167 master_service.cc:946] ReplaceTablet: received request to replace tablet 5883306d8b23451bad367b1809f38bcd from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:14.124254 14812 tablet_bootstrap.cc:492] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.124343 14812 ts_tablet_manager.cc:1397] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.082s user 0.001s sys 0.000s
I20250115 03:06:14.124819 14812 raft_consensus.cc:357] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.124960 14812 raft_consensus.cc:383] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.125007 14812 raft_consensus.cc:738] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.125118 14812 consensus_queue.cc:260] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.125387 14812 ts_tablet_manager.cc:1428] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.125574 14812 tablet_bootstrap.cc:492] T 39245099377549dd81238eabd690b6da P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.126448 14812 tablet_bootstrap.cc:654] T 39245099377549dd81238eabd690b6da P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.129004 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_236770d2959743108285eae378066e45" } modify_external_catalogs: true
I20250115 03:06:14.129103 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_236770d2959743108285eae378066e45" } modify_external_catalogs: true
I20250115 03:06:14.132452 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 6629f2415b554c1c95927c721532fcfc (table default.Table_35d1dd748a6f4e79adced77fe7b2d08d [id=f94088b0776f4be990ddcec04afbca43]). Expected version 1 got 0
W20250115 03:06:14.132741 14167 catalog_manager.cc:5718] Failed to send AlterTable RPC for tablet 6629f2415b554c1c95927c721532fcfc (table default.Table_35d1dd748a6f4e79adced77fe7b2d08d [id=f94088b0776f4be990ddcec04afbca43], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 6629f2415b554c1c95927c721532fcfc consensus state has no leader
I20250115 03:06:14.132812 14819 ts_tablet_manager.cc:1905] T 5883306d8b23451bad367b1809f38bcd P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.134832 14819 ts_tablet_manager.cc:1918] T 5883306d8b23451bad367b1809f38bcd P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.134904 14819 log.cc:1198] T 5883306d8b23451bad367b1809f38bcd P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/5883306d8b23451bad367b1809f38bcd
I20250115 03:06:14.135092 14819 ts_tablet_manager.cc:1939] T 5883306d8b23451bad367b1809f38bcd P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.136189 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 5883306d8b23451bad367b1809f38bcd (table default.Table_0bdf66c6040b4709b0a6be57188b9d86 [id=c8fdb0f9f9064f3eb8d19f4adf3c439f]) successfully deleted
I20250115 03:06:14.136698 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 82c2f41769d84bd48bfc1bc7f504be4b (DEFAULT_TABLE table=default.Table_24cbe6b962b248a9b340506b63c41b20 [id=9866275e65ab4dfcbfee6bf209c3654e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.137073 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82c2f41769d84bd48bfc1bc7f504be4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.137073 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 2d78a7793f84424bad42a1aba89eab92 (DEFAULT_TABLE table=default.Table_638e39bcf2aa4131ac57afbe63ac9b73 [id=50928432524e42d79855e9143c537b61]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.137269 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d78a7793f84424bad42a1aba89eab92. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.138859 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 89d92f59c7424dfc95427329200455e7
I20250115 03:06:14.139035 14401 tablet_service.cc:1467] Processing CreateTablet for tablet b1644ba8616349dc9d10a790b6089571 (DEFAULT_TABLE table=default.Table_dbe8351777e748e8aceef2208bd425ae [id=dd1bb6f7163f46748711bc33cc8a3f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.139050 14664 tablet_service.cc:1467] Processing CreateTablet for tablet b1644ba8616349dc9d10a790b6089571 (DEFAULT_TABLE table=default.Table_dbe8351777e748e8aceef2208bd425ae [id=dd1bb6f7163f46748711bc33cc8a3f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.139230 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1644ba8616349dc9d10a790b6089571. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.139230 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1644ba8616349dc9d10a790b6089571. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.141116 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 82c2f41769d84bd48bfc1bc7f504be4b (DEFAULT_TABLE table=default.Table_24cbe6b962b248a9b340506b63c41b20 [id=9866275e65ab4dfcbfee6bf209c3654e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.141335 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82c2f41769d84bd48bfc1bc7f504be4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.141991 14811 tablet_bootstrap.cc:492] T 5883306d8b23451bad367b1809f38bcd P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.142064 14811 ts_tablet_manager.cc:1397] T 5883306d8b23451bad367b1809f38bcd P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.095s user 0.001s sys 0.000s
I20250115 03:06:14.142234 14812 tablet_bootstrap.cc:492] T 39245099377549dd81238eabd690b6da P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.142310 14812 ts_tablet_manager.cc:1397] T 39245099377549dd81238eabd690b6da P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20250115 03:06:14.142354 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 6629f2415b554c1c95927c721532fcfc (table default.Table_35d1dd748a6f4e79adced77fe7b2d08d [id=f94088b0776f4be990ddcec04afbca43]). Expected version 1 got 0
I20250115 03:06:14.142524 14811 raft_consensus.cc:357] T 5883306d8b23451bad367b1809f38bcd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.142613 14811 raft_consensus.cc:383] T 5883306d8b23451bad367b1809f38bcd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.142638 14811 raft_consensus.cc:738] T 5883306d8b23451bad367b1809f38bcd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.142755 14811 consensus_queue.cc:260] T 5883306d8b23451bad367b1809f38bcd P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.142758 14812 raft_consensus.cc:357] T 39245099377549dd81238eabd690b6da P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.142849 14812 raft_consensus.cc:383] T 39245099377549dd81238eabd690b6da P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.142880 14812 raft_consensus.cc:738] T 39245099377549dd81238eabd690b6da P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.142985 14812 consensus_queue.cc:260] T 39245099377549dd81238eabd690b6da P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.143028 14811 ts_tablet_manager.cc:1428] T 5883306d8b23451bad367b1809f38bcd P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.143253 14812 ts_tablet_manager.cc:1428] T 39245099377549dd81238eabd690b6da P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.143338 14394 tablet_service.cc:1467] Processing CreateTablet for tablet a4484651d964411e90ed93c6caab0a13 (DEFAULT_TABLE table=default.Table_2db28e1f2ea5450092ed00f593c4939f [id=6fe98239ee3346df8b25712aaec67b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.143348 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 2d78a7793f84424bad42a1aba89eab92 (DEFAULT_TABLE table=default.Table_638e39bcf2aa4131ac57afbe63ac9b73 [id=50928432524e42d79855e9143c537b61]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.143420 14812 tablet_bootstrap.cc:492] T ef002211ee1c4a149232c8e433277add P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.143533 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4484651d964411e90ed93c6caab0a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.143545 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d78a7793f84424bad42a1aba89eab92. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.144426 14812 tablet_bootstrap.cc:654] T ef002211ee1c4a149232c8e433277add P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.145069 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 82c2f41769d84bd48bfc1bc7f504be4b (DEFAULT_TABLE table=default.Table_24cbe6b962b248a9b340506b63c41b20 [id=9866275e65ab4dfcbfee6bf209c3654e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.145188 15151 raft_consensus.cc:491] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.145277 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82c2f41769d84bd48bfc1bc7f504be4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.145288 14812 tablet_bootstrap.cc:492] T ef002211ee1c4a149232c8e433277add P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.145254 15151 raft_consensus.cc:513] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.145350 14812 ts_tablet_manager.cc:1397] T ef002211ee1c4a149232c8e433277add P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:14.145576 15151 leader_election.cc:290] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.145577 14530 tablet_service.cc:1467] Processing CreateTablet for tablet b1644ba8616349dc9d10a790b6089571 (DEFAULT_TABLE table=default.Table_dbe8351777e748e8aceef2208bd425ae [id=dd1bb6f7163f46748711bc33cc8a3f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.145777 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1644ba8616349dc9d10a790b6089571. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.145779 14812 raft_consensus.cc:357] T ef002211ee1c4a149232c8e433277add P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.145862 14812 raft_consensus.cc:383] T ef002211ee1c4a149232c8e433277add P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.145895 14812 raft_consensus.cc:738] T ef002211ee1c4a149232c8e433277add P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.145985 14812 consensus_queue.cc:260] T ef002211ee1c4a149232c8e433277add P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.146222 14812 ts_tablet_manager.cc:1428] T ef002211ee1c4a149232c8e433277add P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:14.146351 14812 tablet_bootstrap.cc:492] T bcf6745e549d47cdbdf97c2dff6997ad P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.147076 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6629f2415b554c1c95927c721532fcfc" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.147305 14812 tablet_bootstrap.cc:654] T bcf6745e549d47cdbdf97c2dff6997ad P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.147667 14811 tablet_bootstrap.cc:492] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
W20250115 03:06:14.147799 14628 leader_election.cc:343] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.147821 14536 tablet_service.cc:1467] Processing CreateTablet for tablet e594ea5cff81475886cdceb8400645a6 (DEFAULT_TABLE table=default.Table_bd6a79400bc244d69676d8df1069cbb2 [id=e8ef3a6c99c14ba585efea22568617a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.148015 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e594ea5cff81475886cdceb8400645a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.148164 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6629f2415b554c1c95927c721532fcfc" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.148510 14811 tablet_bootstrap.cc:654] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:14.148581 14626 leader_election.cc:343] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.148646 14626 leader_election.cc:304] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.148834 15325 raft_consensus.cc:2743] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:14.149492 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 89d92f59c7424dfc95427329200455e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.149618 14669 tablet_service.cc:1514] Processing DeleteTablet for tablet ef002211ee1c4a149232c8e433277add with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.149646 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 39245099377549dd81238eabd690b6da with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.149891 14401 tablet_service.cc:1467] Processing CreateTablet for tablet e594ea5cff81475886cdceb8400645a6 (DEFAULT_TABLE table=default.Table_bd6a79400bc244d69676d8df1069cbb2 [id=e8ef3a6c99c14ba585efea22568617a9]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:14.150061 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 89d92f59c7424dfc95427329200455e7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 89d92f59c7424dfc95427329200455e7 already in progress: creating tablet
I20250115 03:06:14.150079 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e594ea5cff81475886cdceb8400645a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.152156 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 4ff51d514fb84809aa53fa9fc1010681 (DEFAULT_TABLE table=default.Table_8750bca4a0104d30babab0f0c0a8c8dc [id=98f48acd89a6461bb4511141cab4d88b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.152336 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ff51d514fb84809aa53fa9fc1010681. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.154217 14672 tablet_service.cc:1467] Processing CreateTablet for tablet a4484651d964411e90ed93c6caab0a13 (DEFAULT_TABLE table=default.Table_2db28e1f2ea5450092ed00f593c4939f [id=6fe98239ee3346df8b25712aaec67b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.154407 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4484651d964411e90ed93c6caab0a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.154672 14809 tablet_bootstrap.cc:492] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.154731 14809 ts_tablet_manager.cc:1397] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.040s user 0.001s sys 0.000s
I20250115 03:06:14.155041 14809 raft_consensus.cc:357] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.155109 14809 raft_consensus.cc:383] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.155128 14809 raft_consensus.cc:738] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.155225 14809 consensus_queue.cc:260] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.155457 14809 ts_tablet_manager.cc:1428] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:14.155673 14809 tablet_bootstrap.cc:492] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.156255 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 4ff51d514fb84809aa53fa9fc1010681 (DEFAULT_TABLE table=default.Table_8750bca4a0104d30babab0f0c0a8c8dc [id=98f48acd89a6461bb4511141cab4d88b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.156432 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ff51d514fb84809aa53fa9fc1010681. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.156525 14809 tablet_bootstrap.cc:654] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.157408 14811 tablet_bootstrap.cc:492] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.157482 14811 ts_tablet_manager.cc:1397] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20250115 03:06:14.157660 14162 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5ad5197958ce45de8b9d78174aaae689 (table default.Table_ec8ee0d483474271a3b369a2bcde593d [id=d8084de64a264d28a3370971617a694c]). Expected version 1 got 0
I20250115 03:06:14.158020 14811 raft_consensus.cc:357] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.158108 14811 raft_consensus.cc:383] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.158138 14811 raft_consensus.cc:738] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.158246 14811 consensus_queue.cc:260] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.158289 15151 raft_consensus.cc:491] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.158346 15151 raft_consensus.cc:513] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.158388 14858 raft_consensus.cc:491] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.158499 14811 ts_tablet_manager.cc:1428] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:14.158474 14858 raft_consensus.cc:513] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.158643 14811 tablet_bootstrap.cc:492] T 543bbb3ae87c46a6982e6fa1ae325db6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.158672 15151 leader_election.cc:290] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.158802 14858 leader_election.cc:290] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.159454 14811 tablet_bootstrap.cc:654] T 543bbb3ae87c46a6982e6fa1ae325db6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.159492 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f4fc7a15692446fba7fcdb17b6b33d56"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.159830 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f4fc7a15692446fba7fcdb17b6b33d56 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.160241 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a63742df234d30bdac0173693159e4" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:14.160336 14689 raft_consensus.cc:2463] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:14.160569 14493 leader_election.cc:304] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:14.160902 14858 raft_consensus.cc:2798] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.160982 14858 raft_consensus.cc:491] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.161016 14858 raft_consensus.cc:3054] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.161387 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a63742df234d30bdac0173693159e4" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.161470 14422 raft_consensus.cc:2463] T c4a63742df234d30bdac0173693159e4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:14.161782 14858 raft_consensus.cc:513] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.162076 14858 leader_election.cc:290] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.162636 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a63742df234d30bdac0173693159e4" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:14.162731 14689 raft_consensus.cc:3054] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.163465 14689 raft_consensus.cc:2463] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:14.163841 14493 leader_election.cc:304] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:14.164215 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a63742df234d30bdac0173693159e4" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:14.164304 14422 raft_consensus.cc:3054] T c4a63742df234d30bdac0173693159e4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.165076 14858 raft_consensus.cc:2798] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:14.165156 14858 raft_consensus.cc:695] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:14.165174 14422 raft_consensus.cc:2463] T c4a63742df234d30bdac0173693159e4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:14.165273 14858 consensus_queue.cc:237] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.165884 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4f770f63a7348afb71904b93c68c379" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.147642 14529 tablet_service.cc:1514] Processing DeleteTablet for tablet 89d92f59c7424dfc95427329200455e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.136703 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 2d78a7793f84424bad42a1aba89eab92 (DEFAULT_TABLE table=default.Table_638e39bcf2aa4131ac57afbe63ac9b73 [id=50928432524e42d79855e9143c537b61]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:14.166188 14628 leader_election.cc:343] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.166360 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d78a7793f84424bad42a1aba89eab92. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:14.166532 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 89d92f59c7424dfc95427329200455e7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 89d92f59c7424dfc95427329200455e7 already in progress: creating tablet
I20250115 03:06:14.168780 14811 tablet_bootstrap.cc:492] T 543bbb3ae87c46a6982e6fa1ae325db6 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.168856 14811 ts_tablet_manager.cc:1397] T 543bbb3ae87c46a6982e6fa1ae325db6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20250115 03:06:14.169270 14811 raft_consensus.cc:357] T 543bbb3ae87c46a6982e6fa1ae325db6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.169363 14811 raft_consensus.cc:383] T 543bbb3ae87c46a6982e6fa1ae325db6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.169390 14811 raft_consensus.cc:738] T 543bbb3ae87c46a6982e6fa1ae325db6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.169482 14811 consensus_queue.cc:260] T 543bbb3ae87c46a6982e6fa1ae325db6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.169556 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8750bca4a0104d30babab0f0c0a8c8dc" } new_table_name: "default.Table_60f25fa4d7af45d3865a654b1e1d4e5f" modify_external_catalogs: true
I20250115 03:06:14.169723 14811 ts_tablet_manager.cc:1428] T 543bbb3ae87c46a6982e6fa1ae325db6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:14.170845 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4f770f63a7348afb71904b93c68c379" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.170950 14557 raft_consensus.cc:2463] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:14.171155 14811 tablet_bootstrap.cc:492] T 2fcce3505b744f668c5355aa982c8485 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.171530 14626 leader_election.cc:304] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.171734 15151 raft_consensus.cc:2798] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.171777 15151 raft_consensus.cc:491] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.171809 15151 raft_consensus.cc:3054] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.172021 14811 tablet_bootstrap.cc:654] T 2fcce3505b744f668c5355aa982c8485 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.172247 14812 tablet_bootstrap.cc:492] T bcf6745e549d47cdbdf97c2dff6997ad P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.172324 14812 ts_tablet_manager.cc:1397] T bcf6745e549d47cdbdf97c2dff6997ad P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.001s
I20250115 03:06:14.172622 15151 raft_consensus.cc:513] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.173202 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4f770f63a7348afb71904b93c68c379" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:14.173421 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4f770f63a7348afb71904b93c68c379" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:14.173509 14557 raft_consensus.cc:3054] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.172912 15151 leader_election.cc:290] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.173799 14664 tablet_service.cc:1467] Processing CreateTablet for tablet e594ea5cff81475886cdceb8400645a6 (DEFAULT_TABLE table=default.Table_bd6a79400bc244d69676d8df1069cbb2 [id=e8ef3a6c99c14ba585efea22568617a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.173988 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e594ea5cff81475886cdceb8400645a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.174294 14557 raft_consensus.cc:2463] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:14.174810 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3bd81cb8bc2d4b74ae3dbcbdb74aae8b" } modify_external_catalogs: true
I20250115 03:06:14.174907 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3bd81cb8bc2d4b74ae3dbcbdb74aae8b" } modify_external_catalogs: true
I20250115 03:06:14.176039 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 5883306d8b23451bad367b1809f38bcd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.176179 14820 tablet_replica.cc:331] T 5883306d8b23451bad367b1809f38bcd P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.176285 14820 raft_consensus.cc:2238] T 5883306d8b23451bad367b1809f38bcd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.176285 14527 tablet_service.cc:1467] Processing CreateTablet for tablet a4484651d964411e90ed93c6caab0a13 (DEFAULT_TABLE table=default.Table_2db28e1f2ea5450092ed00f593c4939f [id=6fe98239ee3346df8b25712aaec67b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.176405 14820 raft_consensus.cc:2267] T 5883306d8b23451bad367b1809f38bcd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.176514 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4484651d964411e90ed93c6caab0a13. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.176612 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f4d365d35e3c43e79d3ac20e438e7aa5" } modify_external_catalogs: true
I20250115 03:06:14.176702 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f4d365d35e3c43e79d3ac20e438e7aa5" } modify_external_catalogs: true
I20250115 03:06:14.177171 14820 ts_tablet_manager.cc:1905] T 5883306d8b23451bad367b1809f38bcd P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.178429 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 89d92f59c7424dfc95427329200455e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.178692 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 4ff51d514fb84809aa53fa9fc1010681 (DEFAULT_TABLE table=default.Table_8750bca4a0104d30babab0f0c0a8c8dc [id=98f48acd89a6461bb4511141cab4d88b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.178876 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ff51d514fb84809aa53fa9fc1010681. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.179040 14820 ts_tablet_manager.cc:1918] T 5883306d8b23451bad367b1809f38bcd P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.179102 14820 log.cc:1198] T 5883306d8b23451bad367b1809f38bcd P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/5883306d8b23451bad367b1809f38bcd
I20250115 03:06:14.179283 14820 ts_tablet_manager.cc:1939] T 5883306d8b23451bad367b1809f38bcd P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.181035 14853 raft_consensus.cc:491] T 1ef14443e7fd464a8ab82eb33819dac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.181123 14853 raft_consensus.cc:513] T 1ef14443e7fd464a8ab82eb33819dac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.181435 14853 leader_election.cc:290] T 1ef14443e7fd464a8ab82eb33819dac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.181895 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef14443e7fd464a8ab82eb33819dac3" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:14.181982 14811 tablet_bootstrap.cc:492] T 2fcce3505b744f668c5355aa982c8485 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.182044 14811 ts_tablet_manager.cc:1397] T 2fcce3505b744f668c5355aa982c8485 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20250115 03:06:14.182322 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 5ad5197958ce45de8b9d78174aaae689 (table default.Table_ec8ee0d483474271a3b369a2bcde593d [id=d8084de64a264d28a3370971617a694c]). Expected version 1 got 0
I20250115 03:06:14.182385 14811 raft_consensus.cc:357] T 2fcce3505b744f668c5355aa982c8485 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.182461 14811 raft_consensus.cc:383] T 2fcce3505b744f668c5355aa982c8485 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.182485 14811 raft_consensus.cc:738] T 2fcce3505b744f668c5355aa982c8485 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.182444 14167 catalog_manager.cc:5526] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:14.182575 14811 consensus_queue.cc:260] T 2fcce3505b744f668c5355aa982c8485 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.182790 14811 ts_tablet_manager.cc:1428] T 2fcce3505b744f668c5355aa982c8485 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:14.182933 14811 tablet_bootstrap.cc:492] T dca04d171deb488690ea58bd75995021 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.183768 14811 tablet_bootstrap.cc:654] T dca04d171deb488690ea58bd75995021 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:14.173561 14628 leader_election.cc:343] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.184347 14626 leader_election.cc:304] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.184368 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ef14443e7fd464a8ab82eb33819dac3" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.184479 14421 raft_consensus.cc:2388] T 1ef14443e7fd464a8ab82eb33819dac3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in current term 1: Already voted for candidate 5c0dae7aff4b44c699275b95ba639770 in this term.
I20250115 03:06:14.184559 15151 raft_consensus.cc:2798] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:14.184625 15151 raft_consensus.cc:695] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:14.184729 15151 consensus_queue.cc:237] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.185307 14493 leader_election.cc:304] T 1ef14443e7fd464a8ab82eb33819dac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.173372 14812 raft_consensus.cc:357] T bcf6745e549d47cdbdf97c2dff6997ad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.185417 14812 raft_consensus.cc:383] T bcf6745e549d47cdbdf97c2dff6997ad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.185442 14812 raft_consensus.cc:738] T bcf6745e549d47cdbdf97c2dff6997ad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.185549 14812 consensus_queue.cc:260] T bcf6745e549d47cdbdf97c2dff6997ad P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.185765 14166 master_service.cc:946] ReplaceTablet: received request to replace tablet 186b3bf9f6e14c9cb435a1094564f66a from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:14.185825 14812 ts_tablet_manager.cc:1428] T bcf6745e549d47cdbdf97c2dff6997ad P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.013s user 0.001s sys 0.000s
I20250115 03:06:14.185958 14812 tablet_bootstrap.cc:492] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.186120 14853 raft_consensus.cc:3054] T 1ef14443e7fd464a8ab82eb33819dac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.186666 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ebca4429c164491c974b3ee244c3b1de" } modify_external_catalogs: true
I20250115 03:06:14.186753 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ebca4429c164491c974b3ee244c3b1de" } modify_external_catalogs: true
I20250115 03:06:14.186892 14812 tablet_bootstrap.cc:654] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.186923 14853 raft_consensus.cc:2743] T 1ef14443e7fd464a8ab82eb33819dac3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:14.187927 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_af3419632bb0478bab77f1fa32e9fb52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.188234 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_af3419632bb0478bab77f1fa32e9fb52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.137085 14823 ts_tablet_manager.cc:1918] T 8c21c254cdef4015bfaba3fb5d61f554 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:14.188967 14823 log.cc:1198] T 8c21c254cdef4015bfaba3fb5d61f554 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/8c21c254cdef4015bfaba3fb5d61f554
I20250115 03:06:14.188895 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e050a17d73d342fda8108a99a665f222"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:14.189183 14823 ts_tablet_manager.cc:1939] T 8c21c254cdef4015bfaba3fb5d61f554 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
W20250115 03:06:14.189232 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e050a17d73d342fda8108a99a665f222 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.189639 14823 tablet_replica.cc:331] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.189729 14823 raft_consensus.cc:2238] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.189806 14823 raft_consensus.cc:2267] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.189790 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 8c21c254cdef4015bfaba3fb5d61f554 (table default.Table_2e115074bf7049bfbb1239b4d9c1ca2e [id=6cf622de00cb490bb794837223aa7bcb]) successfully deleted
I20250115 03:06:14.192826 14861 consensus_queue.cc:1035] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.194399 14812 tablet_bootstrap.cc:492] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.194470 14812 ts_tablet_manager.cc:1397] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:14.194828 14823 ts_tablet_manager.cc:1905] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.194866 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ea55848527c6480fbe8b8fa219480faa" } new_table_name: "default.Table_271f46df55f14ffd9b7bb48352620543" modify_external_catalogs: true
I20250115 03:06:14.194888 14812 raft_consensus.cc:357] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.194962 14812 raft_consensus.cc:383] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.194991 14812 raft_consensus.cc:738] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.195113 14812 consensus_queue.cc:260] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.195441 14812 ts_tablet_manager.cc:1428] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.198104 14823 ts_tablet_manager.cc:1918] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:14.198164 14823 log.cc:1198] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/c6771c98e569468ea932882858981d23
W20250115 03:06:14.199044 14162 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4ff51d514fb84809aa53fa9fc1010681 consensus state has no leader
I20250115 03:06:14.199503 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b20b6ea81915403f986c91997f637c86"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.199901 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b20b6ea81915403f986c91997f637c86 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.201594 14823 ts_tablet_manager.cc:1939] T c6771c98e569468ea932882858981d23 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.201817 14812 tablet_bootstrap.cc:492] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.202653 14812 tablet_bootstrap.cc:654] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.203223 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 5883306d8b23451bad367b1809f38bcd (table default.Table_0bdf66c6040b4709b0a6be57188b9d86 [id=c8fdb0f9f9064f3eb8d19f4adf3c439f]) successfully deleted
I20250115 03:06:14.203799 14885 raft_consensus.cc:491] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.203877 14885 raft_consensus.cc:513] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.204186 14885 leader_election.cc:290] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.204422 14812 tablet_bootstrap.cc:492] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.204500 14812 ts_tablet_manager.cc:1397] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:14.204777 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ad5197958ce45de8b9d78174aaae689" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:14.204828 14885 consensus_queue.cc:1035] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.204851 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ad5197958ce45de8b9d78174aaae689" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.204883 14689 raft_consensus.cc:2463] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:14.204967 14557 raft_consensus.cc:2463] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:14.205245 14358 leader_election.cc:304] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:14.205437 14861 raft_consensus.cc:2798] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.205498 14861 raft_consensus.cc:491] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.205535 14861 raft_consensus.cc:3054] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.205587 14812 raft_consensus.cc:357] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.205852 14812 raft_consensus.cc:383] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.205886 14812 raft_consensus.cc:738] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.205999 14812 consensus_queue.cc:260] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.206228 14812 ts_tablet_manager.cc:1428] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:14.206393 14861 raft_consensus.cc:513] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.206714 14861 leader_election.cc:290] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.206840 14812 tablet_bootstrap.cc:492] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.207585 14812 tablet_bootstrap.cc:654] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.208809 14823 tablet_replica.cc:331] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.208920 14823 raft_consensus.cc:2238] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:14.209048 14823 pending_rounds.cc:62] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 1 pending ops.
I20250115 03:06:14.209069 14823 pending_rounds.cc:69] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384892226510848 op_type: NO_OP noop_request { }
I20250115 03:06:14.209108 14823 raft_consensus.cc:2883] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:14.209147 14823 raft_consensus.cc:2267] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.209702 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 47b08f6e7dec4c5a871a49c79421209e
I20250115 03:06:14.209969 14811 tablet_bootstrap.cc:492] T dca04d171deb488690ea58bd75995021 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.210036 14811 ts_tablet_manager.cc:1397] T dca04d171deb488690ea58bd75995021 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.000s
I20250115 03:06:14.210356 14168 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet f21761ff87ac43d5b9b47b5905fa5867 (table default.Table_53c9a73ae8ef459c987964efac552c96 [id=37d965d8a5eb408ea3f33313bd22c152]). Expected version 2 got 0
I20250115 03:06:14.210374 14811 raft_consensus.cc:357] T dca04d171deb488690ea58bd75995021 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.210462 14811 raft_consensus.cc:383] T dca04d171deb488690ea58bd75995021 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.210489 14811 raft_consensus.cc:738] T dca04d171deb488690ea58bd75995021 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.210592 14811 consensus_queue.cc:260] T dca04d171deb488690ea58bd75995021 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.210819 14811 ts_tablet_manager.cc:1428] T dca04d171deb488690ea58bd75995021 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:14.210973 14811 tablet_bootstrap.cc:492] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.211000 14809 tablet_bootstrap.cc:492] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.211073 14809 ts_tablet_manager.cc:1397] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.055s user 0.000s sys 0.001s
I20250115 03:06:14.211515 14809 raft_consensus.cc:357] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.211603 14809 raft_consensus.cc:383] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.211632 14809 raft_consensus.cc:738] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.211722 14811 tablet_bootstrap.cc:654] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.211735 14809 consensus_queue.cc:260] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.211992 14809 ts_tablet_manager.cc:1428] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:14.212144 14809 tablet_bootstrap.cc:492] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.213089 14163 catalog_manager.cc:5526] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:14.213261 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet d4f770f63a7348afb71904b93c68c379 (table default.Table_117c4a2ba1cb4369881c9bc30c21781b [id=50c73c26b4ed47a4961348db433b7dd4]). Expected version 1 got 0
I20250115 03:06:14.214452 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 186b3bf9f6e14c9cb435a1094564f66a
I20250115 03:06:14.214684 14166 catalog_manager.cc:6596] ReplaceTablet: tablet 186b3bf9f6e14c9cb435a1094564f66a deleted and replaced by tablet 0f3ba1b2ff964d648e5f36f410107b5b
I20250115 03:06:14.214691 14811 tablet_bootstrap.cc:492] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.214767 14811 ts_tablet_manager.cc:1397] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:14.214972 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 2379bae3690c44949e658dde84c1712e
I20250115 03:06:14.215158 14811 raft_consensus.cc:357] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.215245 14811 raft_consensus.cc:383] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.215274 14811 raft_consensus.cc:738] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.215539 14811 consensus_queue.cc:260] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.216071 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet c6771c98e569468ea932882858981d23 (table default.Table_3db38338e5b04298954aded6d5482169 [id=f56a0de6e56441bfa620902f175feb4a]) successfully deleted
I20250115 03:06:14.216472 14665 tablet_service.cc:1467] Processing CreateTablet for tablet f5af8b81e7ff4acab5aef13003656988 (DEFAULT_TABLE table=default.Table_f4fc7a15692446fba7fcdb17b6b33d56 [id=364a7905f3f044f684e9c75927f7efda]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.216678 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5af8b81e7ff4acab5aef13003656988. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.216744 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 47b08f6e7dec4c5a871a49c79421209e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.216876 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 186b3bf9f6e14c9cb435a1094564f66a with delete_type TABLET_DATA_DELETED (replaced by tablet 0f3ba1b2ff964d648e5f36f410107b5b) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.217007 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 2379bae3690c44949e658dde84c1712e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.217226 14819 tablet_replica.cc:331] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.217326 14819 raft_consensus.cc:2238] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.217401 14819 raft_consensus.cc:2267] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.217720 14819 ts_tablet_manager.cc:1905] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.218665 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 47b08f6e7dec4c5a871a49c79421209e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.218779 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 186b3bf9f6e14c9cb435a1094564f66a with delete_type TABLET_DATA_DELETED (replaced by tablet 0f3ba1b2ff964d648e5f36f410107b5b) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.218868 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 2379bae3690c44949e658dde84c1712e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.219094 14399 tablet_service.cc:1467] Processing CreateTablet for tablet f5af8b81e7ff4acab5aef13003656988 (DEFAULT_TABLE table=default.Table_f4fc7a15692446fba7fcdb17b6b33d56 [id=364a7905f3f044f684e9c75927f7efda]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.219300 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5af8b81e7ff4acab5aef13003656988. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.219664 14819 ts_tablet_manager.cc:1918] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.219727 14819 log.cc:1198] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/186b3bf9f6e14c9cb435a1094564f66a
I20250115 03:06:14.219892 14819 ts_tablet_manager.cc:1939] T 186b3bf9f6e14c9cb435a1094564f66a P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
W20250115 03:06:14.221463 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6f83dd9a2d13420daefdacdb0f565a7d consensus state has no leader
I20250115 03:06:14.221819 14535 tablet_service.cc:1467] Processing CreateTablet for tablet f5af8b81e7ff4acab5aef13003656988 (DEFAULT_TABLE table=default.Table_f4fc7a15692446fba7fcdb17b6b33d56 [id=364a7905f3f044f684e9c75927f7efda]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.222026 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5af8b81e7ff4acab5aef13003656988. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.222005 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_48dcc3ce48c94335a128ff0ce47649ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.222240 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 47b08f6e7dec4c5a871a49c79421209e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47b08f6e7dec4c5a871a49c79421209e already in progress: creating tablet
W20250115 03:06:14.222294 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_48dcc3ce48c94335a128ff0ce47649ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.222342 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 186b3bf9f6e14c9cb435a1094564f66a (table default.Table_e83118015644482ca56543e76b9691bd [id=089b3b2c658a4702869cf70ddd9baf73]) successfully deleted
W20250115 03:06:14.222451 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 2379bae3690c44949e658dde84c1712e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2379bae3690c44949e658dde84c1712e already in progress: creating tablet
I20250115 03:06:14.223814 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 47b08f6e7dec4c5a871a49c79421209e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.223922 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 186b3bf9f6e14c9cb435a1094564f66a with delete_type TABLET_DATA_DELETED (replaced by tablet 0f3ba1b2ff964d648e5f36f410107b5b) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.223990 14529 tablet_service.cc:1514] Processing DeleteTablet for tablet 2379bae3690c44949e658dde84c1712e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.224103 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ad5197958ce45de8b9d78174aaae689" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:14.224186 14689 raft_consensus.cc:3054] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.224314 14820 tablet_replica.cc:331] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.224431 14820 raft_consensus.cc:2238] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.224972 14689 raft_consensus.cc:2463] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:14.226369 14358 leader_election.cc:304] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:14.228530 15390 raft_consensus.cc:2798] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:14.228646 15390 raft_consensus.cc:695] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:14.228813 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet dca04d171deb488690ea58bd75995021
I20250115 03:06:14.228791 15390 consensus_queue.cc:237] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.230403 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ad5197958ce45de8b9d78174aaae689" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:14.230507 14556 raft_consensus.cc:3054] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.231385 14556 raft_consensus.cc:2463] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
W20250115 03:06:14.231638 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 47b08f6e7dec4c5a871a49c79421209e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47b08f6e7dec4c5a871a49c79421209e already in progress: creating tablet
I20250115 03:06:14.234549 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet bcf6745e549d47cdbdf97c2dff6997ad with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.235268 15151 raft_consensus.cc:491] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.235340 15151 raft_consensus.cc:513] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.235636 15151 leader_election.cc:290] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.235754 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet dca04d171deb488690ea58bd75995021 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.236532 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0580a8f7b2984162b9275d74334b1905" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.236639 14529 tablet_service.cc:1514] Processing DeleteTablet for tablet dca04d171deb488690ea58bd75995021 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.236747 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 543bbb3ae87c46a6982e6fa1ae325db6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.236817 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fcce3505b744f668c5355aa982c8485 with delete_type TABLET_DATA_DELETED (replaced by tablet e368f87556944c46a6d8f160ead17ac9) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:14.236820 14628 leader_election.cc:343] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.237166 14858 raft_consensus.cc:491] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.237234 14858 raft_consensus.cc:513] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.237517 14858 leader_election.cc:290] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
W20250115 03:06:14.238343 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet dca04d171deb488690ea58bd75995021 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dca04d171deb488690ea58bd75995021 already in progress: creating tablet
I20250115 03:06:14.238487 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39245099377549dd81238eabd690b6da" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:14.238606 14691 raft_consensus.cc:2463] T 39245099377549dd81238eabd690b6da P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:14.239081 14820 raft_consensus.cc:2267] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.239427 14820 ts_tablet_manager.cc:1905] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.239413 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet dca04d171deb488690ea58bd75995021 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.240063 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39245099377549dd81238eabd690b6da" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.241567 14820 ts_tablet_manager.cc:1918] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.241631 14820 log.cc:1198] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/186b3bf9f6e14c9cb435a1094564f66a
I20250115 03:06:14.241847 14820 ts_tablet_manager.cc:1939] T 186b3bf9f6e14c9cb435a1094564f66a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.242211 14820 tablet_replica.cc:331] T dca04d171deb488690ea58bd75995021 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.242336 14820 raft_consensus.cc:2238] T dca04d171deb488690ea58bd75995021 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.242419 14820 raft_consensus.cc:2267] T dca04d171deb488690ea58bd75995021 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:14.242919 14493 leader_election.cc:343] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.243394 14493 leader_election.cc:304] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.243641 14853 raft_consensus.cc:2798] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.243692 14853 raft_consensus.cc:491] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.243741 14853 raft_consensus.cc:3054] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.244067 14809 tablet_bootstrap.cc:654] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.244510 14853 raft_consensus.cc:513] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.244822 14853 leader_election.cc:290] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:14.245579 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet 673f9e6cabdd4320b777181c787d15c7 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:14.245991 14823 ts_tablet_manager.cc:1905] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.246857 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39245099377549dd81238eabd690b6da" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:14.246958 14691 raft_consensus.cc:3054] T 39245099377549dd81238eabd690b6da P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.247790 14691 raft_consensus.cc:2463] T 39245099377549dd81238eabd690b6da P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:14.247900 14823 ts_tablet_manager.cc:1918] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:14.247961 14823 log.cc:1198] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/0c969b652a564ed18fd41bebee47533c
I20250115 03:06:14.248133 14823 ts_tablet_manager.cc:1939] T 0c969b652a564ed18fd41bebee47533c P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.248368 14493 leader_election.cc:304] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:14.248636 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 0c969b652a564ed18fd41bebee47533c (table default.Table_a730bc55668645acadbaa47b450a5338 [id=e31e6fb5a7e941c494ef416ef4762fb7]) successfully deleted
I20250115 03:06:14.248679 14823 tablet_replica.cc:331] T ef002211ee1c4a149232c8e433277add P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.248785 14823 raft_consensus.cc:2238] T ef002211ee1c4a149232c8e433277add P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
W20250115 03:06:14.248800 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet e3302d62a34e43d09c11abca292d9110 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e3302d62a34e43d09c11abca292d9110 already in progress: creating tablet
I20250115 03:06:14.248855 14823 raft_consensus.cc:2267] T ef002211ee1c4a149232c8e433277add P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.249274 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0580a8f7b2984162b9275d74334b1905" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.249440 14823 ts_tablet_manager.cc:1905] T ef002211ee1c4a149232c8e433277add P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.249964 14969 tablet.cc:1721] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:14.250298 14812 tablet_bootstrap.cc:492] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.250272 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_117c4a2ba1cb4369881c9bc30c21781b" } new_table_name: "default.Table_c832613845e141f2b165d1ef2a601af3" modify_external_catalogs: true
I20250115 03:06:14.250371 14812 ts_tablet_manager.cc:1397] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.044s user 0.001s sys 0.000s
I20250115 03:06:14.250747 14812 raft_consensus.cc:357] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.250833 14812 raft_consensus.cc:383] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.250861 14812 raft_consensus.cc:738] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.250967 14812 consensus_queue.cc:260] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.251212 14812 ts_tablet_manager.cc:1428] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.251235 14823 ts_tablet_manager.cc:1918] T ef002211ee1c4a149232c8e433277add P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.251283 14823 log.cc:1198] T ef002211ee1c4a149232c8e433277add P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/ef002211ee1c4a149232c8e433277add
I20250115 03:06:14.251396 14812 tablet_bootstrap.cc:492] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.251475 14823 ts_tablet_manager.cc:1939] T ef002211ee1c4a149232c8e433277add P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.252153 14812 tablet_bootstrap.cc:654] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.252147 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39245099377549dd81238eabd690b6da" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:14.252617 15274 tablet.cc:1721] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:14.252990 14823 tablet_replica.cc:331] T 39245099377549dd81238eabd690b6da P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.253091 14823 raft_consensus.cc:2238] T 39245099377549dd81238eabd690b6da P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.253171 14823 raft_consensus.cc:2267] T 39245099377549dd81238eabd690b6da P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.253149 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet ef002211ee1c4a149232c8e433277add (table default.Table_f957fe3487ec4546a46a8392108003f0 [id=fb3a764b51e74e08a281d128e21466e3]) successfully deleted
I20250115 03:06:14.253556 14823 ts_tablet_manager.cc:1905] T 39245099377549dd81238eabd690b6da P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.253715 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 186b3bf9f6e14c9cb435a1094564f66a (table default.Table_e83118015644482ca56543e76b9691bd [id=089b3b2c658a4702869cf70ddd9baf73]) successfully deleted
I20250115 03:06:14.253800 15299 tablet.cc:1721] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20250115 03:06:14.253880 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 2379bae3690c44949e658dde84c1712e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2379bae3690c44949e658dde84c1712e already in progress: creating tablet
I20250115 03:06:14.254652 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 39245099377549dd81238eabd690b6da with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.254737 14853 raft_consensus.cc:2798] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:14.254817 14853 raft_consensus.cc:695] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
W20250115 03:06:14.254844 14626 leader_election.cc:343] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.254906 14626 leader_election.cc:304] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.254921 14853 consensus_queue.cc:237] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.255077 15151 raft_consensus.cc:2743] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:14.256202 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4d6634c15deb4073a13c251e972e912c" } new_table_name: "default.Table_4b95b36b8d744822b248adc43a061bb1" modify_external_catalogs: true
I20250115 03:06:14.256884 14823 ts_tablet_manager.cc:1918] T 39245099377549dd81238eabd690b6da P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.256963 14823 log.cc:1198] T 39245099377549dd81238eabd690b6da P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/39245099377549dd81238eabd690b6da
I20250115 03:06:14.257129 14823 ts_tablet_manager.cc:1939] T 39245099377549dd81238eabd690b6da P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.257853 14823 tablet_replica.cc:331] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.257937 14170 master_service.cc:946] ReplaceTablet: received request to replace tablet 53248628d9584f8b886166c1e8ec5c3c from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:14.257967 14823 raft_consensus.cc:2238] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:14.258014 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 39245099377549dd81238eabd690b6da (table default.Table_aeb59441e2604aa189d7b95ea5e98ed2 [id=c121e0ff11a141d6b1c10f66e38b1006]) successfully deleted
I20250115 03:06:14.258088 14823 pending_rounds.cc:62] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 1 pending ops.
I20250115 03:06:14.258113 14823 pending_rounds.cc:69] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384890713686016 op_type: NO_OP noop_request { }
W20250115 03:06:14.258142 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 89d92f59c7424dfc95427329200455e7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 89d92f59c7424dfc95427329200455e7 already in progress: creating tablet
I20250115 03:06:14.258157 14823 raft_consensus.cc:2883] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:14.258203 14823 raft_consensus.cc:2267] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:14.258258 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 47b08f6e7dec4c5a871a49c79421209e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47b08f6e7dec4c5a871a49c79421209e already in progress: creating tablet
I20250115 03:06:14.259130 14823 ts_tablet_manager.cc:1905] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:14.259871 14493 leader_election.cc:343] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.260798 14823 ts_tablet_manager.cc:1918] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:14.260852 14823 log.cc:1198] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/186b3bf9f6e14c9cb435a1094564f66a
I20250115 03:06:14.261085 14823 ts_tablet_manager.cc:1939] T 186b3bf9f6e14c9cb435a1094564f66a P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.262207 14809 tablet_bootstrap.cc:492] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.262286 14809 ts_tablet_manager.cc:1397] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.050s user 0.000s sys 0.002s
I20250115 03:06:14.262727 14809 raft_consensus.cc:357] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.262827 14809 raft_consensus.cc:383] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.262857 14809 raft_consensus.cc:738] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.262957 14809 consensus_queue.cc:260] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.263114 14422 raft_consensus.cc:3054] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.264003 14422 raft_consensus.cc:1270] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 5c0dae7aff4b44c699275b95ba639770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:14.264501 15153 consensus_queue.cc:1035] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.265130 14556 raft_consensus.cc:1270] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer 5c0dae7aff4b44c699275b95ba639770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:14.265362 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 186b3bf9f6e14c9cb435a1094564f66a (table default.Table_e83118015644482ca56543e76b9691bd [id=089b3b2c658a4702869cf70ddd9baf73]) successfully deleted
I20250115 03:06:14.265528 15153 raft_consensus.cc:491] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.265597 15153 raft_consensus.cc:513] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:14.265753 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 2379bae3690c44949e658dde84c1712e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2379bae3690c44949e658dde84c1712e already in progress: creating tablet
I20250115 03:06:14.265928 15153 leader_election.cc:290] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.265939 14823 tablet_replica.cc:331] T bcf6745e549d47cdbdf97c2dff6997ad P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.266065 14823 raft_consensus.cc:2238] T bcf6745e549d47cdbdf97c2dff6997ad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.266142 14823 raft_consensus.cc:2267] T bcf6745e549d47cdbdf97c2dff6997ad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.266376 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca04d171deb488690ea58bd75995021" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:14.266852 14628 leader_election.cc:343] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.266856 14823 ts_tablet_manager.cc:1905] T bcf6745e549d47cdbdf97c2dff6997ad P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.267414 14689 raft_consensus.cc:1270] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:14.267665 15151 consensus_queue.cc:1035] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.267728 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca04d171deb488690ea58bd75995021" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.268000 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d1e81884e3a94c6abc91f7a6e9646351"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.268316 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d1e81884e3a94c6abc91f7a6e9646351 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:14.268401 14626 leader_election.cc:343] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.268472 14626 leader_election.cc:304] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.268648 15153 raft_consensus.cc:2743] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:14.268997 14823 ts_tablet_manager.cc:1918] T bcf6745e549d47cdbdf97c2dff6997ad P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.269057 14823 log.cc:1198] T bcf6745e549d47cdbdf97c2dff6997ad P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/bcf6745e549d47cdbdf97c2dff6997ad
I20250115 03:06:14.269239 14823 ts_tablet_manager.cc:1939] T bcf6745e549d47cdbdf97c2dff6997ad P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.269251 14858 consensus_queue.cc:1035] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.269745 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet bcf6745e549d47cdbdf97c2dff6997ad (table default.Table_28b265c831a944c7bff1a3ee55a86c64 [id=65f79614672344d88c2ec541c43da990]) successfully deleted
I20250115 03:06:14.269930 14823 tablet_replica.cc:331] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.270030 14823 raft_consensus.cc:2238] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.270107 14823 raft_consensus.cc:2267] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.270526 14823 ts_tablet_manager.cc:1905] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.270874 14812 tablet_bootstrap.cc:492] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.270957 14812 ts_tablet_manager.cc:1397] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.000s
I20250115 03:06:14.271121 14162 catalog_manager.cc:5526] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:14.271327 14162 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5ad5197958ce45de8b9d78174aaae689 (table default.Table_ec8ee0d483474271a3b369a2bcde593d [id=d8084de64a264d28a3370971617a694c]). Expected version 1 got 0
I20250115 03:06:14.271375 14812 raft_consensus.cc:357] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.271471 14812 raft_consensus.cc:383] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.271502 14812 raft_consensus.cc:738] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.271605 14812 consensus_queue.cc:260] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.271839 14812 ts_tablet_manager.cc:1428] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.271991 14812 tablet_bootstrap.cc:492] T 673f9e6cabdd4320b777181c787d15c7 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.272167 14823 ts_tablet_manager.cc:1918] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.272225 14823 log.cc:1198] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/dca04d171deb488690ea58bd75995021
I20250115 03:06:14.272353 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_afc5c7e955fb4b3ba43151cb8823a76b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.272692 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_afc5c7e955fb4b3ba43151cb8823a76b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.272418 14823 ts_tablet_manager.cc:1939] T dca04d171deb488690ea58bd75995021 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.272835 14812 tablet_bootstrap.cc:654] T 673f9e6cabdd4320b777181c787d15c7 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.274645 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_dbe8351777e748e8aceef2208bd425ae" } new_table_name: "default.Table_129ccce41ec44747b8b0a8c0df193bf8" modify_external_catalogs: true
I20250115 03:06:14.272985 15390 raft_consensus.cc:491] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.275385 15390 raft_consensus.cc:513] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.275494 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 68974a1815cb482d8b28d54f42634608 (DEFAULT_TABLE table=default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.274678 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet f21761ff87ac43d5b9b47b5905fa5867 (table default.Table_53c9a73ae8ef459c987964efac552c96 [id=37d965d8a5eb408ea3f33313bd22c152]). Expected version 2 got 0
I20250115 03:06:14.275724 15390 leader_election.cc:290] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.275812 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 0580a8f7b2984162b9275d74334b1905 (table default.Table_3e704f1e03e34a2ca2377db61214c5a0 [id=0790b08615854d3688f940aba20b6520]). Expected version 1 got 0
I20250115 03:06:14.276175 15153 raft_consensus.cc:491] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.276248 15153 raft_consensus.cc:513] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.275733 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68974a1815cb482d8b28d54f42634608. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.276547 15153 leader_election.cc:290] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.276623 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "543bbb3ae87c46a6982e6fa1ae325db6" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.276739 14557 raft_consensus.cc:2463] T 543bbb3ae87c46a6982e6fa1ae325db6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:14.276918 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ef42586d164315b5fd54ba91227bad" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:14.277868 14626 leader_election.cc:343] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.278007 14356 leader_election.cc:304] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:14.278170 15390 raft_consensus.cc:2798] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.278223 15390 raft_consensus.cc:491] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.278257 15390 raft_consensus.cc:3054] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.278488 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet dca04d171deb488690ea58bd75995021 (table default.Table_3bd81cb8bc2d4b74ae3dbcbdb74aae8b [id=05f9fd2ebec34d2299ea59c47a74c828]) successfully deleted
I20250115 03:06:14.275491 14421 raft_consensus.cc:1270] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:14.279110 15390 raft_consensus.cc:513] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.279410 15390 leader_election.cc:290] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.279559 14165 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_53c9a73ae8ef459c987964efac552c96 [id=37d965d8a5eb408ea3f33313bd22c152] alter complete (version 2)
I20250115 03:06:14.280279 14812 tablet_bootstrap.cc:492] T 673f9e6cabdd4320b777181c787d15c7 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.280354 14812 ts_tablet_manager.cc:1397] T 673f9e6cabdd4320b777181c787d15c7 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20250115 03:06:14.280745 14812 raft_consensus.cc:357] T 673f9e6cabdd4320b777181c787d15c7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.280838 14812 raft_consensus.cc:383] T 673f9e6cabdd4320b777181c787d15c7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.280884 14812 raft_consensus.cc:738] T 673f9e6cabdd4320b777181c787d15c7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.281082 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ef42586d164315b5fd54ba91227bad" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.281195 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "543bbb3ae87c46a6982e6fa1ae325db6" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:14.281283 14554 raft_consensus.cc:3054] T 543bbb3ae87c46a6982e6fa1ae325db6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.281975 14165 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet e43ce064a02347efa81ca274d76c6960 (table default.Table_4b95b36b8d744822b248adc43a061bb1 [id=0d8deaf903704c00a4a0576354ecfbe6]). Expected version 1 got 0
I20250115 03:06:14.282213 14554 raft_consensus.cc:2463] T 543bbb3ae87c46a6982e6fa1ae325db6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:14.282622 14853 consensus_queue.cc:1035] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:14.282655 14628 leader_election.cc:343] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.282709 14628 leader_election.cc:304] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.280802 15274 tablet.cc:1721] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:14.282887 15325 raft_consensus.cc:2743] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:14.283061 14358 leader_election.cc:343] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 543bbb3ae87c46a6982e6fa1ae325db6
I20250115 03:06:14.283202 14356 leader_election.cc:304] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770
I20250115 03:06:14.283203 14812 consensus_queue.cc:260] T 673f9e6cabdd4320b777181c787d15c7 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.283458 14861 raft_consensus.cc:2798] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:14.283535 14861 raft_consensus.cc:695] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:14.283541 14812 ts_tablet_manager.cc:1428] T 673f9e6cabdd4320b777181c787d15c7 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:14.283612 15146 mvcc.cc:204] Tried to move back new op lower bound from 7114384892976967680 to 7114384892661981184. Current Snapshot: MvccSnapshot[applied={T|T < 7114384892976967680}]
I20250115 03:06:14.283640 14861 consensus_queue.cc:237] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.283689 14812 tablet_bootstrap.cc:492] T 6f83dd9a2d13420daefdacdb0f565a7d P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.284860 14812 tablet_bootstrap.cc:654] T 6f83dd9a2d13420daefdacdb0f565a7d P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.284953 14969 tablet.cc:1721] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20250115 03:06:14.286051 14358 leader_election.cc:343] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 543bbb3ae87c46a6982e6fa1ae325db6
I20250115 03:06:14.286834 14812 tablet_bootstrap.cc:492] T 6f83dd9a2d13420daefdacdb0f565a7d P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.286902 14812 ts_tablet_manager.cc:1397] T 6f83dd9a2d13420daefdacdb0f565a7d P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:14.287281 14812 raft_consensus.cc:357] T 6f83dd9a2d13420daefdacdb0f565a7d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.287370 14812 raft_consensus.cc:383] T 6f83dd9a2d13420daefdacdb0f565a7d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.287397 14812 raft_consensus.cc:738] T 6f83dd9a2d13420daefdacdb0f565a7d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.287503 14812 consensus_queue.cc:260] T 6f83dd9a2d13420daefdacdb0f565a7d P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.287746 14812 ts_tablet_manager.cc:1428] T 6f83dd9a2d13420daefdacdb0f565a7d P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.287907 14812 tablet_bootstrap.cc:492] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.288740 14812 tablet_bootstrap.cc:654] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.288869 14809 ts_tablet_manager.cc:1428] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.027s user 0.000s sys 0.001s
I20250115 03:06:14.289063 14809 tablet_bootstrap.cc:492] T 2fcce3505b744f668c5355aa982c8485 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.289965 14809 tablet_bootstrap.cc:654] T 2fcce3505b744f668c5355aa982c8485 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.290863 14812 tablet_bootstrap.cc:492] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.290936 14812 ts_tablet_manager.cc:1397] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20250115 03:06:14.291257 14812 raft_consensus.cc:357] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.291337 14812 raft_consensus.cc:383] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.291357 14812 raft_consensus.cc:738] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.291440 14812 consensus_queue.cc:260] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.291707 14812 ts_tablet_manager.cc:1428] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:14.291851 14812 tablet_bootstrap.cc:492] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.292531 14811 ts_tablet_manager.cc:1428] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.078s user 0.001s sys 0.000s
I20250115 03:06:14.292642 14812 tablet_bootstrap.cc:654] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.292724 14811 tablet_bootstrap.cc:492] T bcf6745e549d47cdbdf97c2dff6997ad P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.293296 15325 raft_consensus.cc:491] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.293377 15325 raft_consensus.cc:513] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.293713 15325 leader_election.cc:290] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.293946 14811 tablet_bootstrap.cc:654] T bcf6745e549d47cdbdf97c2dff6997ad P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.294107 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b1775fbfc0eb4cf4a59a0f9a853a081e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:14.294163 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bda0e0238da246feb36450f40dd061e5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:14.294428 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b1775fbfc0eb4cf4a59a0f9a853a081e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:14.294486 14626 leader_election.cc:343] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.294745 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bda0e0238da246feb36450f40dd061e5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:14.295091 14628 leader_election.cc:343] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.295151 14628 leader_election.cc:304] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.295413 15325 raft_consensus.cc:2743] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:14.296468 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_64d7a4f0d2694a65a7fb86e55e1255e0" } modify_external_catalogs: true
I20250115 03:06:14.296566 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_64d7a4f0d2694a65a7fb86e55e1255e0" } modify_external_catalogs: true
I20250115 03:06:14.297114 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 68974a1815cb482d8b28d54f42634608 (DEFAULT_TABLE table=default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.297351 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68974a1815cb482d8b28d54f42634608. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.299419 14820 ts_tablet_manager.cc:1905] T dca04d171deb488690ea58bd75995021 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.300093 15325 consensus_queue.cc:1035] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.300849 14812 tablet_bootstrap.cc:492] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.300938 14812 ts_tablet_manager.cc:1397] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:14.301299 14812 raft_consensus.cc:357] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.301364 14812 raft_consensus.cc:383] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.301383 14812 raft_consensus.cc:738] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.301478 14812 consensus_queue.cc:260] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.301713 14812 ts_tablet_manager.cc:1428] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:14.301700 14820 ts_tablet_manager.cc:1918] T dca04d171deb488690ea58bd75995021 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.301718 14399 tablet_service.cc:1467] Processing CreateTablet for tablet fa08518530ae4d5487af386a650c4672 (DEFAULT_TABLE table=default.Table_48dcc3ce48c94335a128ff0ce47649ae [id=12b47ada2a2c48f2b6807fa62fcca1a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.301769 14820 log.cc:1198] T dca04d171deb488690ea58bd75995021 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/dca04d171deb488690ea58bd75995021
I20250115 03:06:14.301867 14812 tablet_bootstrap.cc:492] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.301920 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa08518530ae4d5487af386a650c4672. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.301945 14820 ts_tablet_manager.cc:1939] T dca04d171deb488690ea58bd75995021 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.302736 14820 tablet_replica.cc:331] T 543bbb3ae87c46a6982e6fa1ae325db6 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.302784 14812 tablet_bootstrap.cc:654] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.302845 14820 raft_consensus.cc:2238] T 543bbb3ae87c46a6982e6fa1ae325db6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.302934 14820 raft_consensus.cc:2267] T 543bbb3ae87c46a6982e6fa1ae325db6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.305478 14812 tablet_bootstrap.cc:492] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.305557 14812 ts_tablet_manager.cc:1397] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20250115 03:06:14.305961 14812 raft_consensus.cc:357] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.306046 14812 raft_consensus.cc:383] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.306075 14812 raft_consensus.cc:738] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.306180 14812 consensus_queue.cc:260] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.306416 14812 ts_tablet_manager.cc:1428] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.306576 14812 tablet_bootstrap.cc:492] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.306793 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet d4f770f63a7348afb71904b93c68c379 (table default.Table_c832613845e141f2b165d1ef2a601af3 [id=50c73c26b4ed47a4961348db433b7dd4]). Expected version 2 got 0
I20250115 03:06:14.307754 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 68974a1815cb482d8b28d54f42634608 (DEFAULT_TABLE table=default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.307801 15390 raft_consensus.cc:491] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.307863 15390 raft_consensus.cc:513] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.307983 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68974a1815cb482d8b28d54f42634608. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.308015 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_a1547d1d923f4f4da05aaeb519f4cdf0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.308312 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a1547d1d923f4f4da05aaeb519f4cdf0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.310093 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet dca04d171deb488690ea58bd75995021 (table default.Table_3bd81cb8bc2d4b74ae3dbcbdb74aae8b [id=05f9fd2ebec34d2299ea59c47a74c828]) successfully deleted
I20250115 03:06:14.310077 14660 tablet_service.cc:1467] Processing CreateTablet for tablet fa08518530ae4d5487af386a650c4672 (DEFAULT_TABLE table=default.Table_48dcc3ce48c94335a128ff0ce47649ae [id=12b47ada2a2c48f2b6807fa62fcca1a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.310287 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa08518530ae4d5487af386a650c4672. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.308163 15390 leader_election.cc:290] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.312901 14165 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_c832613845e141f2b165d1ef2a601af3 [id=50c73c26b4ed47a4961348db433b7dd4] alter complete (version 2)
W20250115 03:06:14.313172 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b1644ba8616349dc9d10a790b6089571 consensus state has no leader
I20250115 03:06:14.313349 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 53248628d9584f8b886166c1e8ec5c3c
I20250115 03:06:14.313369 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_60943a26c80945ae84effaed75c0cc42"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:14.313557 14170 catalog_manager.cc:6596] ReplaceTablet: tablet 53248628d9584f8b886166c1e8ec5c3c deleted and replaced by tablet 7f67cbb929cd4ae589a14606aa168b46
W20250115 03:06:14.313655 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_60943a26c80945ae84effaed75c0cc42 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.315246 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 53248628d9584f8b886166c1e8ec5c3c with delete_type TABLET_DATA_DELETED (replaced by tablet 7f67cbb929cd4ae589a14606aa168b46) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.315384 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 543bbb3ae87c46a6982e6fa1ae325db6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.315425 14536 tablet_service.cc:1467] Processing CreateTablet for tablet fa08518530ae4d5487af386a650c4672 (DEFAULT_TABLE table=default.Table_48dcc3ce48c94335a128ff0ce47649ae [id=12b47ada2a2c48f2b6807fa62fcca1a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.315631 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa08518530ae4d5487af386a650c4672. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:14.315742 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 53248628d9584f8b886166c1e8ec5c3c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 53248628d9584f8b886166c1e8ec5c3c already in progress: creating tablet
I20250115 03:06:14.315760 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b561abc942014fb79f22fd6a9fec38d0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.315855 14554 raft_consensus.cc:2463] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:14.316008 14399 tablet_service.cc:1467] Processing CreateTablet for tablet db1aa206e9994de6b9a355a621cb2ab3 (DEFAULT_TABLE table=default.Table_e050a17d73d342fda8108a99a665f222 [id=eea2f4b389314abbac5f983a79d5cf7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.316293 14356 leader_election.cc:304] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:14.316497 14819 tablet_replica.cc:331] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.316619 14819 raft_consensus.cc:2238] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:14.316753 14819 pending_rounds.cc:62] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75: Trying to abort 1 pending ops.
I20250115 03:06:14.316779 14819 pending_rounds.cc:69] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384893067046912 op_type: NO_OP noop_request { }
I20250115 03:06:14.316828 14819 raft_consensus.cc:2883] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:14.316870 14819 raft_consensus.cc:2267] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.317015 15390 raft_consensus.cc:2798] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.317063 15390 raft_consensus.cc:491] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.317097 15390 raft_consensus.cc:3054] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.317665 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 673f9e6cabdd4320b777181c787d15c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.317799 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 53248628d9584f8b886166c1e8ec5c3c with delete_type TABLET_DATA_DELETED (replaced by tablet 7f67cbb929cd4ae589a14606aa168b46) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.317994 15390 raft_consensus.cc:513] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.318136 15325 consensus_queue.cc:1035] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.318281 15390 leader_election.cc:290] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.318965 14164 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet e43ce064a02347efa81ca274d76c6960 (table default.Table_4b95b36b8d744822b248adc43a061bb1 [id=0d8deaf903704c00a4a0576354ecfbe6]). Expected version 1 got 0
I20250115 03:06:14.319532 14812 tablet_bootstrap.cc:654] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.317020 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db1aa206e9994de6b9a355a621cb2ab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.320596 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 6f83dd9a2d13420daefdacdb0f565a7d (table default.Table_271f46df55f14ffd9b7bb48352620543 [id=44488be7335b488a9ff7cf913ce4273a]). Expected version 1 got 0
I20250115 03:06:14.321058 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3ba1b2ff964d648e5f36f410107b5b (DEFAULT_TABLE table=default.Table_e83118015644482ca56543e76b9691bd [id=089b3b2c658a4702869cf70ddd9baf73]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.321249 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f3ba1b2ff964d648e5f36f410107b5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.323349 14529 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3ba1b2ff964d648e5f36f410107b5b (DEFAULT_TABLE table=default.Table_e83118015644482ca56543e76b9691bd [id=089b3b2c658a4702869cf70ddd9baf73]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.323575 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f3ba1b2ff964d648e5f36f410107b5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.324136 14167 master_service.cc:946] ReplaceTablet: received request to replace tablet 6629f2415b554c1c95927c721532fcfc from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:14.321040 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3ba1b2ff964d648e5f36f410107b5b (DEFAULT_TABLE table=default.Table_e83118015644482ca56543e76b9691bd [id=089b3b2c658a4702869cf70ddd9baf73]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.323298 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 9808c7f96a3147848da5f78948895699 (DEFAULT_TABLE table=default.Table_af3419632bb0478bab77f1fa32e9fb52 [id=369c42ec23ec4a27a71a54be23a410a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.325112 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9808c7f96a3147848da5f78948895699. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.325547 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 9808c7f96a3147848da5f78948895699 (DEFAULT_TABLE table=default.Table_af3419632bb0478bab77f1fa32e9fb52 [id=369c42ec23ec4a27a71a54be23a410a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.325750 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9808c7f96a3147848da5f78948895699. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.326936 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f3ba1b2ff964d648e5f36f410107b5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.304894 14820 ts_tablet_manager.cc:1905] T 543bbb3ae87c46a6982e6fa1ae325db6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.327880 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b561abc942014fb79f22fd6a9fec38d0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:14.327980 14557 raft_consensus.cc:3054] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.328756 14557 raft_consensus.cc:2463] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:14.329581 14536 tablet_service.cc:1467] Processing CreateTablet for tablet db1aa206e9994de6b9a355a621cb2ab3 (DEFAULT_TABLE table=default.Table_e050a17d73d342fda8108a99a665f222 [id=eea2f4b389314abbac5f983a79d5cf7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.329782 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db1aa206e9994de6b9a355a621cb2ab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.329864 14356 leader_election.cc:304] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:14.330629 15390 raft_consensus.cc:2798] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:14.331562 15390 raft_consensus.cc:695] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:14.331694 15390 consensus_queue.cc:237] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.331959 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 55f01d704b174b2ea108ea6f1df763df
I20250115 03:06:14.332404 14660 tablet_service.cc:1467] Processing CreateTablet for tablet db1aa206e9994de6b9a355a621cb2ab3 (DEFAULT_TABLE table=default.Table_e050a17d73d342fda8108a99a665f222 [id=eea2f4b389314abbac5f983a79d5cf7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.332620 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db1aa206e9994de6b9a355a621cb2ab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.333752 14165 catalog_manager.cc:5526] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:14.333956 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet b561abc942014fb79f22fd6a9fec38d0 (table default.Table_fa83f65f02b04ad491cee3b2a617927f [id=46f096547b3a4de9a46522b978ac4b8c]). Expected version 1 got 0
I20250115 03:06:14.334352 14823 tablet_replica.cc:331] T 673f9e6cabdd4320b777181c787d15c7 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.334383 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f01d704b174b2ea108ea6f1df763df with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.334473 14823 raft_consensus.cc:2238] T 673f9e6cabdd4320b777181c787d15c7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.334568 14823 raft_consensus.cc:2267] T 673f9e6cabdd4320b777181c787d15c7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.334796 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 9808c7f96a3147848da5f78948895699 (DEFAULT_TABLE table=default.Table_af3419632bb0478bab77f1fa32e9fb52 [id=369c42ec23ec4a27a71a54be23a410a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.334983 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9808c7f96a3147848da5f78948895699. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.335263 14823 ts_tablet_manager.cc:1905] T 673f9e6cabdd4320b777181c787d15c7 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.335767 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f01d704b174b2ea108ea6f1df763df with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.336092 14820 ts_tablet_manager.cc:1918] T 543bbb3ae87c46a6982e6fa1ae325db6 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.336155 14820 log.cc:1198] T 543bbb3ae87c46a6982e6fa1ae325db6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/543bbb3ae87c46a6982e6fa1ae325db6
I20250115 03:06:14.336346 14820 ts_tablet_manager.cc:1939] T 543bbb3ae87c46a6982e6fa1ae325db6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.336807 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 6629f2415b554c1c95927c721532fcfc
I20250115 03:06:14.336899 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 53248628d9584f8b886166c1e8ec5c3c with delete_type TABLET_DATA_DELETED (replaced by tablet 7f67cbb929cd4ae589a14606aa168b46) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.337109 14167 catalog_manager.cc:6596] ReplaceTablet: tablet 6629f2415b554c1c95927c721532fcfc deleted and replaced by tablet 5b7022db9edc4de8909e16769a3ea136
I20250115 03:06:14.337402 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b561abc942014fb79f22fd6a9fec38d0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:14.337460 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 6629f2415b554c1c95927c721532fcfc with delete_type TABLET_DATA_DELETED (replaced by tablet 5b7022db9edc4de8909e16769a3ea136) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.337530 14691 raft_consensus.cc:2463] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:14.337402 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b561abc942014fb79f22fd6a9fec38d0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:14.337687 14823 ts_tablet_manager.cc:1918] T 673f9e6cabdd4320b777181c787d15c7 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.337729 14687 raft_consensus.cc:3054] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.337747 14823 log.cc:1198] T 673f9e6cabdd4320b777181c787d15c7 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/673f9e6cabdd4320b777181c787d15c7
I20250115 03:06:14.337891 14820 tablet_replica.cc:331] T 2fcce3505b744f668c5355aa982c8485 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.337927 14823 ts_tablet_manager.cc:1939] T 673f9e6cabdd4320b777181c787d15c7 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.338007 14820 raft_consensus.cc:2238] T 2fcce3505b744f668c5355aa982c8485 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.338095 14820 raft_consensus.cc:2267] T 2fcce3505b744f668c5355aa982c8485 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.338523 14687 raft_consensus.cc:2463] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:14.338635 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 673f9e6cabdd4320b777181c787d15c7 (table default.Table_42a1977fa70d4f16b43bf861b66524a7 [id=c9b28587ba934521bd4158dfaf416f6e]) successfully deleted
I20250115 03:06:14.338869 14820 ts_tablet_manager.cc:1905] T 2fcce3505b744f668c5355aa982c8485 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:14.339057 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 53248628d9584f8b886166c1e8ec5c3c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 53248628d9584f8b886166c1e8ec5c3c already in progress: creating tablet
I20250115 03:06:14.338866 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 6629f2415b554c1c95927c721532fcfc with delete_type TABLET_DATA_DELETED (replaced by tablet 5b7022db9edc4de8909e16769a3ea136) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:14.339880 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 55f01d704b174b2ea108ea6f1df763df because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55f01d704b174b2ea108ea6f1df763df already in progress: creating tablet
I20250115 03:06:14.340852 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 543bbb3ae87c46a6982e6fa1ae325db6 (table default.Table_9432efe75c694779976428f3dea16679 [id=98a21757d0884730ae2a1bbc97359df6]) successfully deleted
I20250115 03:06:14.341043 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f01d704b174b2ea108ea6f1df763df with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.341177 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 6629f2415b554c1c95927c721532fcfc with delete_type TABLET_DATA_DELETED (replaced by tablet 5b7022db9edc4de8909e16769a3ea136) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.341370 14820 ts_tablet_manager.cc:1918] T 2fcce3505b744f668c5355aa982c8485 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.341429 14820 log.cc:1198] T 2fcce3505b744f668c5355aa982c8485 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/2fcce3505b744f668c5355aa982c8485
I20250115 03:06:14.341610 14820 ts_tablet_manager.cc:1939] T 2fcce3505b744f668c5355aa982c8485 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.341620 14823 tablet_replica.cc:331] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.341754 14823 raft_consensus.cc:2238] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.341845 14823 raft_consensus.cc:2267] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.342376 14811 tablet_bootstrap.cc:492] T bcf6745e549d47cdbdf97c2dff6997ad P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.342470 14811 ts_tablet_manager.cc:1397] T bcf6745e549d47cdbdf97c2dff6997ad P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.050s user 0.002s sys 0.000s
I20250115 03:06:14.342801 14812 tablet_bootstrap.cc:492] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.342872 14812 ts_tablet_manager.cc:1397] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.036s user 0.001s sys 0.000s
I20250115 03:06:14.343091 14811 raft_consensus.cc:357] T bcf6745e549d47cdbdf97c2dff6997ad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.343191 14811 raft_consensus.cc:383] T bcf6745e549d47cdbdf97c2dff6997ad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.343225 14811 raft_consensus.cc:738] T bcf6745e549d47cdbdf97c2dff6997ad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.343365 14811 consensus_queue.cc:260] T bcf6745e549d47cdbdf97c2dff6997ad P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.344195 14812 raft_consensus.cc:357] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.344291 14812 raft_consensus.cc:383] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.344321 14812 raft_consensus.cc:738] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.344408 14812 consensus_queue.cc:260] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.345722 14812 ts_tablet_manager.cc:1428] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:14.346938 14811 ts_tablet_manager.cc:1428] T bcf6745e549d47cdbdf97c2dff6997ad P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:14.347110 14811 tablet_bootstrap.cc:492] T 6629f2415b554c1c95927c721532fcfc P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.347142 14812 tablet_bootstrap.cc:492] T f54942abdfb748f2a939993a3c2d5db0 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.348136 14812 tablet_bootstrap.cc:654] T f54942abdfb748f2a939993a3c2d5db0 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.349783 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5088472ecce645e2935b8b4cb18aa458" } modify_external_catalogs: true
I20250115 03:06:14.349891 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5088472ecce645e2935b8b4cb18aa458" } modify_external_catalogs: true
I20250115 03:06:14.352629 14811 tablet_bootstrap.cc:654] T 6629f2415b554c1c95927c721532fcfc P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.353303 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet bcf6745e549d47cdbdf97c2dff6997ad with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.353488 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet e379cb7a61a8415994a7419266346d8e
I20250115 03:06:14.354077 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet e379cb7a61a8415994a7419266346d8e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.354329 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet e379cb7a61a8415994a7419266346d8e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.355243 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet e379cb7a61a8415994a7419266346d8e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.356037 14165 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 2d3157859e684da98bb473d5799b24a6 (table default.Table_02c01532217e4a8d9092ecb9bed7c891 [id=265c248361624c3f9e5ece27144d8e04]). Expected version 1 got 0
I20250115 03:06:14.356969 15153 raft_consensus.cc:491] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.357048 15153 raft_consensus.cc:513] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.357302 14557 raft_consensus.cc:3054] T ae0efe52ac6c40fbae45c725b07b03a3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.357395 15153 leader_election.cc:290] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.357956 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55dcc7ac82ac44ab9d85055720760d3d" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:14.358296 14626 leader_election.cc:343] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.358363 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55dcc7ac82ac44ab9d85055720760d3d" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.358526 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 2d3157859e684da98bb473d5799b24a6 (table default.Table_02c01532217e4a8d9092ecb9bed7c891 [id=265c248361624c3f9e5ece27144d8e04]). Expected version 1 got 0
I20250115 03:06:14.359472 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 872ad705e3e94c0eab4397df9f28f9ca (DEFAULT_TABLE table=default.Table_afc5c7e955fb4b3ba43151cb8823a76b [id=bac33f20b4524a05bde7452db44d1104]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:14.359643 14628 leader_election.cc:343] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.359724 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 872ad705e3e94c0eab4397df9f28f9ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.359706 14628 leader_election.cc:304] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.359897 15153 raft_consensus.cc:2743] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:14.360222 14820 tablet_replica.cc:331] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.360195 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 13ce129d8a9245baab7c772f0f5fbaa7 (DEFAULT_TABLE table=default.Table_d1e81884e3a94c6abc91f7a6e9646351 [id=6cba4fb2f7104c6e953340b04e5dc494]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.360291 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 2fcce3505b744f668c5355aa982c8485 (table default.Table_2af92748648345c0903a0f99da0d47c5 [id=19efb3734d824699a088ad62e69095f3]) successfully deleted
I20250115 03:06:14.360359 14820 raft_consensus.cc:2238] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:14.360502 14820 pending_rounds.cc:62] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1: Trying to abort 1 pending ops.
I20250115 03:06:14.360530 14820 pending_rounds.cc:69] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384892949164032 op_type: NO_OP noop_request { }
I20250115 03:06:14.360523 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 872ad705e3e94c0eab4397df9f28f9ca (DEFAULT_TABLE table=default.Table_afc5c7e955fb4b3ba43151cb8823a76b [id=bac33f20b4524a05bde7452db44d1104]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.360579 14820 raft_consensus.cc:2883] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:14.360625 14820 raft_consensus.cc:2267] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.360742 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 872ad705e3e94c0eab4397df9f28f9ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.361399 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 13ce129d8a9245baab7c772f0f5fbaa7 (DEFAULT_TABLE table=default.Table_d1e81884e3a94c6abc91f7a6e9646351 [id=6cba4fb2f7104c6e953340b04e5dc494]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.361626 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13ce129d8a9245baab7c772f0f5fbaa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.361754 14823 ts_tablet_manager.cc:1905] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.362733 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 79579515fb7e45a8b16fd36dbc6a965a (DEFAULT_TABLE table=default.Table_b1775fbfc0eb4cf4a59a0f9a853a081e [id=ed57b2cba1db497fa0af4bba35580a35]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.362833 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 79579515fb7e45a8b16fd36dbc6a965a (DEFAULT_TABLE table=default.Table_b1775fbfc0eb4cf4a59a0f9a853a081e [id=ed57b2cba1db497fa0af4bba35580a35]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.362934 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79579515fb7e45a8b16fd36dbc6a965a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.363009 15390 consensus_queue.cc:1035] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.363601 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 7f67cbb929cd4ae589a14606aa168b46 (DEFAULT_TABLE table=default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.363791 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 13ce129d8a9245baab7c772f0f5fbaa7 (DEFAULT_TABLE table=default.Table_d1e81884e3a94c6abc91f7a6e9646351 [id=6cba4fb2f7104c6e953340b04e5dc494]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.363987 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13ce129d8a9245baab7c772f0f5fbaa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.364264 14823 ts_tablet_manager.cc:1918] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.364321 14823 log.cc:1198] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/6629f2415b554c1c95927c721532fcfc
I20250115 03:06:14.364500 14823 ts_tablet_manager.cc:1939] T 6629f2415b554c1c95927c721532fcfc P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.365864 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 872ad705e3e94c0eab4397df9f28f9ca (DEFAULT_TABLE table=default.Table_afc5c7e955fb4b3ba43151cb8823a76b [id=bac33f20b4524a05bde7452db44d1104]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.365952 14886 mvcc.cc:204] Tried to move back new op lower bound from 7114384892988657664 to 7114384891664441344. Current Snapshot: MvccSnapshot[applied={T|T < 7114384892988657664}]
I20250115 03:06:14.366199 15299 tablet.cc:1721] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.367738 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 7f67cbb929cd4ae589a14606aa168b46 (DEFAULT_TABLE table=default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.367807 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 872ad705e3e94c0eab4397df9f28f9ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.367949 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f67cbb929cd4ae589a14606aa168b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.369851 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13ce129d8a9245baab7c772f0f5fbaa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.369946 14969 tablet.cc:1721] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.370162 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f67cbb929cd4ae589a14606aa168b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.370407 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79579515fb7e45a8b16fd36dbc6a965a. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:14.371800 15395 consensus_queue.cc:1159] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20250115 03:06:14.372725 14820 ts_tablet_manager.cc:1905] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:14.372892 15402 consensus_queue.cc:1159] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e8df7247ee9c43e0aa18216da2661f75" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20250115 03:06:14.373317 14819 ts_tablet_manager.cc:1905] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.375216 14819 ts_tablet_manager.cc:1918] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:14.375274 14819 log.cc:1198] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/543bbb3ae87c46a6982e6fa1ae325db6
I20250115 03:06:14.375489 14819 ts_tablet_manager.cc:1939] T 543bbb3ae87c46a6982e6fa1ae325db6 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.376087 14820 ts_tablet_manager.cc:1918] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:14.376153 14820 log.cc:1198] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/39245099377549dd81238eabd690b6da
I20250115 03:06:14.376180 14819 tablet_replica.cc:331] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.376286 14819 raft_consensus.cc:2238] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.376363 14820 ts_tablet_manager.cc:1939] T 39245099377549dd81238eabd690b6da P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.376376 14819 raft_consensus.cc:2267] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.376381 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 79579515fb7e45a8b16fd36dbc6a965a (DEFAULT_TABLE table=default.Table_b1775fbfc0eb4cf4a59a0f9a853a081e [id=ed57b2cba1db497fa0af4bba35580a35]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.376624 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79579515fb7e45a8b16fd36dbc6a965a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.376855 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 543bbb3ae87c46a6982e6fa1ae325db6 (table default.Table_9432efe75c694779976428f3dea16679 [id=98a21757d0884730ae2a1bbc97359df6]) successfully deleted
W20250115 03:06:14.377013 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 55f01d704b174b2ea108ea6f1df763df because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55f01d704b174b2ea108ea6f1df763df already in progress: creating tablet
W20250115 03:06:14.377136 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 6629f2415b554c1c95927c721532fcfc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6629f2415b554c1c95927c721532fcfc already in progress: creating tablet
I20250115 03:06:14.378224 14820 tablet_replica.cc:331] T bcf6745e549d47cdbdf97c2dff6997ad P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.378340 14820 raft_consensus.cc:2238] T bcf6745e549d47cdbdf97c2dff6997ad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.378430 14820 raft_consensus.cc:2267] T bcf6745e549d47cdbdf97c2dff6997ad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.378466 14819 ts_tablet_manager.cc:1905] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.378652 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 8baf86b37e9942ba813b099a0da87ca5 (DEFAULT_TABLE table=default.Table_a1547d1d923f4f4da05aaeb519f4cdf0 [id=e929ffaba567498c815393836b264a12]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.378890 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8baf86b37e9942ba813b099a0da87ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.379086 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 7f67cbb929cd4ae589a14606aa168b46 (DEFAULT_TABLE table=default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.379274 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f67cbb929cd4ae589a14606aa168b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.379946 14819 ts_tablet_manager.cc:1918] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:14.379992 14819 log.cc:1198] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/e379cb7a61a8415994a7419266346d8e
I20250115 03:06:14.380141 14819 ts_tablet_manager.cc:1939] T e379cb7a61a8415994a7419266346d8e P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.381122 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 5b7022db9edc4de8909e16769a3ea136 (DEFAULT_TABLE table=default.Table_35d1dd748a6f4e79adced77fe7b2d08d [id=f94088b0776f4be990ddcec04afbca43]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.381166 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 39245099377549dd81238eabd690b6da (table default.Table_aeb59441e2604aa189d7b95ea5e98ed2 [id=c121e0ff11a141d6b1c10f66e38b1006]) successfully deleted
W20250115 03:06:14.381289 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 53248628d9584f8b886166c1e8ec5c3c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 53248628d9584f8b886166c1e8ec5c3c already in progress: creating tablet
I20250115 03:06:14.381361 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b7022db9edc4de8909e16769a3ea136. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.381352 14401 tablet_service.cc:1467] Processing CreateTablet for tablet ec9f14c1ab014f29843a568c460a28d6 (DEFAULT_TABLE table=default.Table_60943a26c80945ae84effaed75c0cc42 [id=d94e2b655f424f549af052bc8dee0430]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:14.381431 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 6629f2415b554c1c95927c721532fcfc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6629f2415b554c1c95927c721532fcfc already in progress: creating tablet
I20250115 03:06:14.381459 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet e379cb7a61a8415994a7419266346d8e (table default.Table_5088472ecce645e2935b8b4cb18aa458 [id=295264d1a39945ee9ff50a0997a5becd]) successfully deleted
W20250115 03:06:14.381547 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 55f01d704b174b2ea108ea6f1df763df because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55f01d704b174b2ea108ea6f1df763df already in progress: creating tablet
I20250115 03:06:14.381563 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec9f14c1ab014f29843a568c460a28d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.381943 15299 tablet.cc:1721] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:14.382545 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 5b7022db9edc4de8909e16769a3ea136 (DEFAULT_TABLE table=default.Table_35d1dd748a6f4e79adced77fe7b2d08d [id=f94088b0776f4be990ddcec04afbca43]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.382759 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b7022db9edc4de8909e16769a3ea136. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.383239 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d85908ffc4694aaabd57c6ae118cdb06"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.383605 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d85908ffc4694aaabd57c6ae118cdb06 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.385351 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 8baf86b37e9942ba813b099a0da87ca5 (DEFAULT_TABLE table=default.Table_a1547d1d923f4f4da05aaeb519f4cdf0 [id=e929ffaba567498c815393836b264a12]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.385558 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8baf86b37e9942ba813b099a0da87ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.387612 14527 tablet_service.cc:1467] Processing CreateTablet for tablet ec9f14c1ab014f29843a568c460a28d6 (DEFAULT_TABLE table=default.Table_60943a26c80945ae84effaed75c0cc42 [id=d94e2b655f424f549af052bc8dee0430]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.387809 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec9f14c1ab014f29843a568c460a28d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.388162 14809 tablet_bootstrap.cc:492] T 2fcce3505b744f668c5355aa982c8485 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.388229 14809 ts_tablet_manager.cc:1397] T 2fcce3505b744f668c5355aa982c8485 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.099s user 0.000s sys 0.001s
I20250115 03:06:14.388607 14809 raft_consensus.cc:357] T 2fcce3505b744f668c5355aa982c8485 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.388687 14809 raft_consensus.cc:383] T 2fcce3505b744f668c5355aa982c8485 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.388720 14809 raft_consensus.cc:738] T 2fcce3505b744f668c5355aa982c8485 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.388818 14809 consensus_queue.cc:260] T 2fcce3505b744f668c5355aa982c8485 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.388885 14691 raft_consensus.cc:1270] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:14.389077 14809 ts_tablet_manager.cc:1428] T 2fcce3505b744f668c5355aa982c8485 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.389225 14809 tablet_bootstrap.cc:492] T 39245099377549dd81238eabd690b6da P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.389920 14861 consensus_queue.cc:1035] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.390147 14809 tablet_bootstrap.cc:654] T 39245099377549dd81238eabd690b6da P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.390144 14861 consensus_queue.cc:1035] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250115 03:06:14.392534 14664 tablet_service.cc:1467] Processing CreateTablet for tablet ec9f14c1ab014f29843a568c460a28d6 (DEFAULT_TABLE table=default.Table_60943a26c80945ae84effaed75c0cc42 [id=d94e2b655f424f549af052bc8dee0430]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.392738 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec9f14c1ab014f29843a568c460a28d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.392997 14165 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_4b95b36b8d744822b248adc43a061bb1 [id=0d8deaf903704c00a4a0576354ecfbe6] alter complete (version 1)
I20250115 03:06:14.393872 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 8baf86b37e9942ba813b099a0da87ca5 (DEFAULT_TABLE table=default.Table_a1547d1d923f4f4da05aaeb519f4cdf0 [id=e929ffaba567498c815393836b264a12]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.394101 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8baf86b37e9942ba813b099a0da87ca5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.394990 14556 raft_consensus.cc:1270] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:14.396409 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 5b7022db9edc4de8909e16769a3ea136 (DEFAULT_TABLE table=default.Table_35d1dd748a6f4e79adced77fe7b2d08d [id=f94088b0776f4be990ddcec04afbca43]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.398907 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fcce3505b744f668c5355aa982c8485 with delete_type TABLET_DATA_DELETED (replaced by tablet e368f87556944c46a6d8f160ead17ac9) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:14.399144 14157 catalog_manager.cc:4670] Async tablet task AlterTable RPC for tablet 481705dc9ae841659520a6c909879f9b (table default.Table_cab1959a159a46a6b86daad3647b89b3 [id=fe32f8f8243b4803876ed2b371e9e602], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 481705dc9ae841659520a6c909879f9b consensus state has no leader
I20250115 03:06:14.400141 14861 consensus_queue.cc:1035] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.401250 14819 tablet_replica.cc:331] T 2fcce3505b744f668c5355aa982c8485 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.401378 14819 raft_consensus.cc:2238] T 2fcce3505b744f668c5355aa982c8485 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.401471 14819 raft_consensus.cc:2267] T 2fcce3505b744f668c5355aa982c8485 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.402292 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet b561abc942014fb79f22fd6a9fec38d0 (table default.Table_fa83f65f02b04ad491cee3b2a617927f [id=46f096547b3a4de9a46522b978ac4b8c]). Expected version 1 got 0
I20250115 03:06:14.403179 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b7022db9edc4de8909e16769a3ea136. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.406260 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ee5faa8cd90f40199372322d90c5a204" } new_table_name: "default.Table_5ac706949e34442b9151efe1a577e83f" modify_external_catalogs: true
I20250115 03:06:14.406824 14819 ts_tablet_manager.cc:1905] T 2fcce3505b744f668c5355aa982c8485 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.409766 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet b561abc942014fb79f22fd6a9fec38d0 (table default.Table_fa83f65f02b04ad491cee3b2a617927f [id=46f096547b3a4de9a46522b978ac4b8c]). Expected version 1 got 0
I20250115 03:06:14.410444 14819 ts_tablet_manager.cc:1918] T 2fcce3505b744f668c5355aa982c8485 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.410542 14819 log.cc:1198] T 2fcce3505b744f668c5355aa982c8485 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/2fcce3505b744f668c5355aa982c8485
I20250115 03:06:14.410766 14819 ts_tablet_manager.cc:1939] T 2fcce3505b744f668c5355aa982c8485 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.411023 15274 tablet.cc:1721] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.412973 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 2fcce3505b744f668c5355aa982c8485 (table default.Table_2af92748648345c0903a0f99da0d47c5 [id=19efb3734d824699a088ad62e69095f3]) successfully deleted
I20250115 03:06:14.413515 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet b561abc942014fb79f22fd6a9fec38d0 (table default.Table_fa83f65f02b04ad491cee3b2a617927f [id=46f096547b3a4de9a46522b978ac4b8c]). Expected version 1 got 0
I20250115 03:06:14.415380 14820 ts_tablet_manager.cc:1905] T bcf6745e549d47cdbdf97c2dff6997ad P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.415519 15274 tablet.cc:1721] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.422335 15421 tablet.cc:1721] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.422536 14820 ts_tablet_manager.cc:1918] T bcf6745e549d47cdbdf97c2dff6997ad P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.422652 14820 log.cc:1198] T bcf6745e549d47cdbdf97c2dff6997ad P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/bcf6745e549d47cdbdf97c2dff6997ad
I20250115 03:06:14.422899 14820 ts_tablet_manager.cc:1939] T bcf6745e549d47cdbdf97c2dff6997ad P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.423298 14820 tablet_replica.cc:331] T e379cb7a61a8415994a7419266346d8e P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.423482 14820 raft_consensus.cc:2238] T e379cb7a61a8415994a7419266346d8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.423631 14820 raft_consensus.cc:2267] T e379cb7a61a8415994a7419266346d8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.424005 14969 tablet.cc:1721] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:14.424254 14820 ts_tablet_manager.cc:1905] T e379cb7a61a8415994a7419266346d8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.424516 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet bcf6745e549d47cdbdf97c2dff6997ad (table default.Table_28b265c831a944c7bff1a3ee55a86c64 [id=65f79614672344d88c2ec541c43da990]) successfully deleted
I20250115 03:06:14.424880 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_60943a26c80945ae84effaed75c0cc42" } modify_external_catalogs: true
I20250115 03:06:14.425000 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_60943a26c80945ae84effaed75c0cc42" } modify_external_catalogs: true
I20250115 03:06:14.425834 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_160f1efee2ce4861aad307adf98767c3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:14.426703 14162 master_service.cc:946] ReplaceTablet: received request to replace tablet 6f83dd9a2d13420daefdacdb0f565a7d from {username='slave'} at 127.0.0.1:60432
W20250115 03:06:14.426254 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_160f1efee2ce4861aad307adf98767c3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.427330 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d75a227f24d644ceb2bda90d1f6ae998" } modify_external_catalogs: true
I20250115 03:06:14.427410 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d75a227f24d644ceb2bda90d1f6ae998" } modify_external_catalogs: true
I20250115 03:06:14.427853 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_1ad8c0817ede4bbd847c01814eebfa83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:14.428109 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_60f25fa4d7af45d3865a654b1e1d4e5f" } new_table_name: "default.Table_9cc1022af4f14e7a8a8e11cc1803ca87" modify_external_catalogs: true
W20250115 03:06:14.428251 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1ad8c0817ede4bbd847c01814eebfa83 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.428689 14167 master_service.cc:946] ReplaceTablet: received request to replace tablet e4994733ae904c99986cabe4c6efc503 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:14.430454 15299 tablet.cc:1721] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:14.432996 15274 tablet.cc:1721] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:14.435497 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet ec9f14c1ab014f29843a568c460a28d6
I20250115 03:06:14.435858 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 6f83dd9a2d13420daefdacdb0f565a7d
I20250115 03:06:14.435966 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet ec9f14c1ab014f29843a568c460a28d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.436139 14162 catalog_manager.cc:6596] ReplaceTablet: tablet 6f83dd9a2d13420daefdacdb0f565a7d deleted and replaced by tablet 3cbc91ad00b54ede804c3629bcfe7fe6
I20250115 03:06:14.437040 14166 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5e6d1abc27d042dba8506da7b49a6ee3" } modify_external_catalogs: true
I20250115 03:06:14.437125 14166 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5e6d1abc27d042dba8506da7b49a6ee3" } modify_external_catalogs: true
I20250115 03:06:14.437554 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f83dd9a2d13420daefdacdb0f565a7d with delete_type TABLET_DATA_DELETED (replaced by tablet 3cbc91ad00b54ede804c3629bcfe7fe6) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.438365 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f83dd9a2d13420daefdacdb0f565a7d with delete_type TABLET_DATA_DELETED (replaced by tablet 3cbc91ad00b54ede804c3629bcfe7fe6) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.439126 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet ec9f14c1ab014f29843a568c460a28d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.439162 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_cd69692684cd401a9c6f9d24e374582c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.439429 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cd69692684cd401a9c6f9d24e374582c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.439636 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet ec9f14c1ab014f29843a568c460a28d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:14.439869 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4ff51d514fb84809aa53fa9fc1010681 consensus state has no leader
I20250115 03:06:14.440774 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_fa2e1dcccf5e4b34971533a5d66118d7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:14.440955 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet e4994733ae904c99986cabe4c6efc503
W20250115 03:06:14.441062 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fa2e1dcccf5e4b34971533a5d66118d7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.441213 14167 catalog_manager.cc:6596] ReplaceTablet: tablet e4994733ae904c99986cabe4c6efc503 deleted and replaced by tablet f70a69acbdc049beb9d98eada09d9eb8
I20250115 03:06:14.441875 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f83dd9a2d13420daefdacdb0f565a7d with delete_type TABLET_DATA_DELETED (replaced by tablet 3cbc91ad00b54ede804c3629bcfe7fe6) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.442054 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet e4994733ae904c99986cabe4c6efc503 with delete_type TABLET_DATA_DELETED (replaced by tablet f70a69acbdc049beb9d98eada09d9eb8) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:14.442298 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet ec9f14c1ab014f29843a568c460a28d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ec9f14c1ab014f29843a568c460a28d6 already in progress: creating tablet
W20250115 03:06:14.442663 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 6f83dd9a2d13420daefdacdb0f565a7d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f83dd9a2d13420daefdacdb0f565a7d already in progress: creating tablet
I20250115 03:06:14.442833 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet e4994733ae904c99986cabe4c6efc503 with delete_type TABLET_DATA_DELETED (replaced by tablet f70a69acbdc049beb9d98eada09d9eb8) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.443077 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet e4994733ae904c99986cabe4c6efc503 with delete_type TABLET_DATA_DELETED (replaced by tablet f70a69acbdc049beb9d98eada09d9eb8) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.443773 14823 tablet_replica.cc:331] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.443925 14819 tablet_replica.cc:331] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.444075 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 6629f2415b554c1c95927c721532fcfc (table default.Table_35d1dd748a6f4e79adced77fe7b2d08d [id=f94088b0776f4be990ddcec04afbca43]) successfully deleted
I20250115 03:06:14.444092 14819 raft_consensus.cc:2238] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.444265 14819 raft_consensus.cc:2267] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.444499 14823 raft_consensus.cc:2238] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:14.444734 14823 raft_consensus.cc:2267] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.445375 14823 ts_tablet_manager.cc:1905] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.445735 14819 ts_tablet_manager.cc:1905] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.447407 14823 ts_tablet_manager.cc:1918] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:14.447820 14171 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_5ac706949e34442b9151efe1a577e83f [id=ff4681cab8ce44ceb45ed76c70c1132e] alter complete (version 2)
I20250115 03:06:14.447899 14809 tablet_bootstrap.cc:492] T 39245099377549dd81238eabd690b6da P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.447973 14809 ts_tablet_manager.cc:1397] T 39245099377549dd81238eabd690b6da P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.059s user 0.002s sys 0.000s
I20250115 03:06:14.448405 14809 raft_consensus.cc:357] T 39245099377549dd81238eabd690b6da P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.448495 14809 raft_consensus.cc:383] T 39245099377549dd81238eabd690b6da P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.448526 14809 raft_consensus.cc:738] T 39245099377549dd81238eabd690b6da P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.448629 14809 consensus_queue.cc:260] T 39245099377549dd81238eabd690b6da P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.448890 14809 ts_tablet_manager.cc:1428] T 39245099377549dd81238eabd690b6da P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.449103 14809 tablet_bootstrap.cc:492] T bcf6745e549d47cdbdf97c2dff6997ad P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.450057 14809 tablet_bootstrap.cc:654] T bcf6745e549d47cdbdf97c2dff6997ad P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.450764 14823 log.cc:1198] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/e379cb7a61a8415994a7419266346d8e
I20250115 03:06:14.451078 14823 ts_tablet_manager.cc:1939] T e379cb7a61a8415994a7419266346d8e P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.451745 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet e379cb7a61a8415994a7419266346d8e (table default.Table_5088472ecce645e2935b8b4cb18aa458 [id=295264d1a39945ee9ff50a0997a5becd]) successfully deleted
I20250115 03:06:14.452265 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 39245099377549dd81238eabd690b6da with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.452327 14823 tablet_replica.cc:331] T 6f83dd9a2d13420daefdacdb0f565a7d P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.452430 14823 raft_consensus.cc:2238] T 6f83dd9a2d13420daefdacdb0f565a7d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.452522 14823 raft_consensus.cc:2267] T 6f83dd9a2d13420daefdacdb0f565a7d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.453305 14823 ts_tablet_manager.cc:1905] T 6f83dd9a2d13420daefdacdb0f565a7d P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.456108 14811 tablet_bootstrap.cc:492] T 6629f2415b554c1c95927c721532fcfc P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.456198 14811 ts_tablet_manager.cc:1397] T 6629f2415b554c1c95927c721532fcfc P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.109s user 0.002s sys 0.000s
I20250115 03:06:14.456197 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet f01c319ebd5148858de9ea1a608ddff3 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:14.456635 14811 raft_consensus.cc:357] T 6629f2415b554c1c95927c721532fcfc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.456749 14811 raft_consensus.cc:383] T 6629f2415b554c1c95927c721532fcfc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.456784 14811 raft_consensus.cc:738] T 6629f2415b554c1c95927c721532fcfc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.456893 14811 consensus_queue.cc:260] T 6629f2415b554c1c95927c721532fcfc P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.457228 14811 ts_tablet_manager.cc:1428] T 6629f2415b554c1c95927c721532fcfc P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.457435 14811 tablet_bootstrap.cc:492] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.458297 14811 tablet_bootstrap.cc:654] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.459427 14809 tablet_bootstrap.cc:492] T bcf6745e549d47cdbdf97c2dff6997ad P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.459506 14809 ts_tablet_manager.cc:1397] T bcf6745e549d47cdbdf97c2dff6997ad P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20250115 03:06:14.459918 14809 raft_consensus.cc:357] T bcf6745e549d47cdbdf97c2dff6997ad P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.460011 14809 raft_consensus.cc:383] T bcf6745e549d47cdbdf97c2dff6997ad P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.460042 14809 raft_consensus.cc:738] T bcf6745e549d47cdbdf97c2dff6997ad P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.460144 14809 consensus_queue.cc:260] T bcf6745e549d47cdbdf97c2dff6997ad P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.460402 14809 ts_tablet_manager.cc:1428] T bcf6745e549d47cdbdf97c2dff6997ad P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.460562 14809 tablet_bootstrap.cc:492] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.461444 14809 tablet_bootstrap.cc:654] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.461970 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet f01c319ebd5148858de9ea1a608ddff3
I20250115 03:06:14.462213 14171 catalog_manager.cc:6596] ReplaceTablet: tablet f01c319ebd5148858de9ea1a608ddff3 deleted and replaced by tablet 84ce55f729224f78b420e57977419add
I20250115 03:06:14.463014 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet f01c319ebd5148858de9ea1a608ddff3 with delete_type TABLET_DATA_DELETED (replaced by tablet 84ce55f729224f78b420e57977419add) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.463413 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet f01c319ebd5148858de9ea1a608ddff3 with delete_type TABLET_DATA_DELETED (replaced by tablet 84ce55f729224f78b420e57977419add) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.464402 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_160f1efee2ce4861aad307adf98767c3" } new_table_name: "default.Table_3863dfaf57694ee7b4a9ba1e400d7c9d" modify_external_catalogs: true
I20250115 03:06:14.464495 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet f01c319ebd5148858de9ea1a608ddff3 with delete_type TABLET_DATA_DELETED (replaced by tablet 84ce55f729224f78b420e57977419add) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.464785 15426 tablet.cc:1721] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.465636 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5ac706949e34442b9151efe1a577e83f" } new_table_name: "default.Table_0b4f00236d814b8d98a6530e3825328f" modify_external_catalogs: true
I20250115 03:06:14.466383 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet bcf6745e549d47cdbdf97c2dff6997ad with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.468080 14823 ts_tablet_manager.cc:1918] T 6f83dd9a2d13420daefdacdb0f565a7d P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.468186 14823 log.cc:1198] T 6f83dd9a2d13420daefdacdb0f565a7d P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/6f83dd9a2d13420daefdacdb0f565a7d
I20250115 03:06:14.468688 14823 ts_tablet_manager.cc:1939] T 6f83dd9a2d13420daefdacdb0f565a7d P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
W20250115 03:06:14.469292 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 82be772e05d443749914004aea8af300 has no consensus state
I20250115 03:06:14.469483 14809 tablet_bootstrap.cc:492] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.469558 14809 ts_tablet_manager.cc:1397] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20250115 03:06:14.469955 14809 raft_consensus.cc:357] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.470057 14809 raft_consensus.cc:383] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.470091 14809 raft_consensus.cc:738] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.470199 14809 consensus_queue.cc:260] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.470481 14809 ts_tablet_manager.cc:1428] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:14.470664 14809 tablet_bootstrap.cc:492] T dca04d171deb488690ea58bd75995021 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.471705 14809 tablet_bootstrap.cc:654] T dca04d171deb488690ea58bd75995021 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.473304 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 6f83dd9a2d13420daefdacdb0f565a7d (table default.Table_271f46df55f14ffd9b7bb48352620543 [id=44488be7335b488a9ff7cf913ce4273a]) successfully deleted
I20250115 03:06:14.473759 14811 tablet_bootstrap.cc:492] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.473845 14811 ts_tablet_manager.cc:1397] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20250115 03:06:14.474241 14811 raft_consensus.cc:357] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.474336 14811 raft_consensus.cc:383] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.474366 14811 raft_consensus.cc:738] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.474476 14811 consensus_queue.cc:260] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.474746 14811 ts_tablet_manager.cc:1428] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.475450 14811 tablet_bootstrap.cc:492] T ef002211ee1c4a149232c8e433277add P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.475723 14819 ts_tablet_manager.cc:1918] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:14.475790 14819 log.cc:1198] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/e4994733ae904c99986cabe4c6efc503
I20250115 03:06:14.476013 14819 ts_tablet_manager.cc:1939] T e4994733ae904c99986cabe4c6efc503 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.476320 14811 tablet_bootstrap.cc:654] T ef002211ee1c4a149232c8e433277add P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.477141 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet e4994733ae904c99986cabe4c6efc503 (table default.Table_db8a2673977247e0b14b7995f7e09661 [id=f7e4534c1f6f48f09e10b84c1d5c458c]) successfully deleted
I20250115 03:06:14.477550 14819 tablet_replica.cc:331] T 39245099377549dd81238eabd690b6da P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.477656 14819 raft_consensus.cc:2238] T 39245099377549dd81238eabd690b6da P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.477797 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 6629f2415b554c1c95927c721532fcfc with delete_type TABLET_DATA_DELETED (replaced by tablet 5b7022db9edc4de8909e16769a3ea136) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:14.478225 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet ec9f14c1ab014f29843a568c460a28d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ec9f14c1ab014f29843a568c460a28d6 already in progress: creating tablet
I20250115 03:06:14.478245 14823 tablet_replica.cc:331] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.478431 14823 raft_consensus.cc:2238] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.478641 14819 raft_consensus.cc:2267] T 39245099377549dd81238eabd690b6da P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.479184 14823 raft_consensus.cc:2267] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.479763 14823 ts_tablet_manager.cc:1905] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.479836 14812 tablet_bootstrap.cc:492] T f54942abdfb748f2a939993a3c2d5db0 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.479925 14812 ts_tablet_manager.cc:1397] T f54942abdfb748f2a939993a3c2d5db0 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.134s user 0.001s sys 0.000s
I20250115 03:06:14.480162 14819 ts_tablet_manager.cc:1905] T 39245099377549dd81238eabd690b6da P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.480458 14812 raft_consensus.cc:357] T f54942abdfb748f2a939993a3c2d5db0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.480568 14812 raft_consensus.cc:383] T f54942abdfb748f2a939993a3c2d5db0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.480602 14812 raft_consensus.cc:738] T f54942abdfb748f2a939993a3c2d5db0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.480712 14812 consensus_queue.cc:260] T f54942abdfb748f2a939993a3c2d5db0 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.480989 14812 ts_tablet_manager.cc:1428] T f54942abdfb748f2a939993a3c2d5db0 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.481173 14812 tablet_bootstrap.cc:492] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.481886 14820 ts_tablet_manager.cc:1918] T e379cb7a61a8415994a7419266346d8e P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:14.481958 14820 log.cc:1198] T e379cb7a61a8415994a7419266346d8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/e379cb7a61a8415994a7419266346d8e
I20250115 03:06:14.482021 14819 ts_tablet_manager.cc:1918] T 39245099377549dd81238eabd690b6da P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.482080 14812 tablet_bootstrap.cc:654] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.482093 14819 log.cc:1198] T 39245099377549dd81238eabd690b6da P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/39245099377549dd81238eabd690b6da
I20250115 03:06:14.482192 14820 ts_tablet_manager.cc:1939] T e379cb7a61a8415994a7419266346d8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.482272 14819 ts_tablet_manager.cc:1939] T 39245099377549dd81238eabd690b6da P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.482822 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 39245099377549dd81238eabd690b6da (table default.Table_aeb59441e2604aa189d7b95ea5e98ed2 [id=c121e0ff11a141d6b1c10f66e38b1006]) successfully deleted
W20250115 03:06:14.483186 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet f01c319ebd5148858de9ea1a608ddff3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f01c319ebd5148858de9ea1a608ddff3 already in progress: creating tablet
I20250115 03:06:14.483246 14809 tablet_bootstrap.cc:492] T dca04d171deb488690ea58bd75995021 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.483309 14809 ts_tablet_manager.cc:1397] T dca04d171deb488690ea58bd75995021 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.001s
I20250115 03:06:14.483362 14819 tablet_replica.cc:331] T bcf6745e549d47cdbdf97c2dff6997ad P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.483464 14819 raft_consensus.cc:2238] T bcf6745e549d47cdbdf97c2dff6997ad P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.483536 14819 raft_consensus.cc:2267] T bcf6745e549d47cdbdf97c2dff6997ad P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.483682 14809 raft_consensus.cc:357] T dca04d171deb488690ea58bd75995021 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.483772 14809 raft_consensus.cc:383] T dca04d171deb488690ea58bd75995021 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.483799 14809 raft_consensus.cc:738] T dca04d171deb488690ea58bd75995021 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.483889 14809 consensus_queue.cc:260] T dca04d171deb488690ea58bd75995021 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.483925 14819 ts_tablet_manager.cc:1905] T bcf6745e549d47cdbdf97c2dff6997ad P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.484090 14809 ts_tablet_manager.cc:1428] T dca04d171deb488690ea58bd75995021 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.485378 14809 tablet_bootstrap.cc:492] T 0580a8f7b2984162b9275d74334b1905 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.485759 15421 tablet.cc:1721] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:14.486315 14809 tablet_bootstrap.cc:654] T 0580a8f7b2984162b9275d74334b1905 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.486933 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet e379cb7a61a8415994a7419266346d8e (table default.Table_5088472ecce645e2935b8b4cb18aa458 [id=295264d1a39945ee9ff50a0997a5becd]) successfully deleted
W20250115 03:06:14.488660 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet ec9f14c1ab014f29843a568c460a28d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ec9f14c1ab014f29843a568c460a28d6 already in progress: creating tablet
I20250115 03:06:14.489801 14811 tablet_bootstrap.cc:492] T ef002211ee1c4a149232c8e433277add P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.489804 15274 tablet.cc:1721] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:14.489882 14811 ts_tablet_manager.cc:1397] T ef002211ee1c4a149232c8e433277add P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20250115 03:06:14.490314 14811 raft_consensus.cc:357] T ef002211ee1c4a149232c8e433277add P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.490404 14811 raft_consensus.cc:383] T ef002211ee1c4a149232c8e433277add P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.490435 14811 raft_consensus.cc:738] T ef002211ee1c4a149232c8e433277add P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.490741 14811 consensus_queue.cc:260] T ef002211ee1c4a149232c8e433277add P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.491055 15426 tablet.cc:1721] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:14.492107 14811 ts_tablet_manager.cc:1428] T ef002211ee1c4a149232c8e433277add P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:14.492267 14811 tablet_bootstrap.cc:492] T bda0e0238da246feb36450f40dd061e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.493389 14165 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 0580a8f7b2984162b9275d74334b1905 (table default.Table_3e704f1e03e34a2ca2377db61214c5a0 [id=0790b08615854d3688f940aba20b6520]). Expected version 1 got 0
I20250115 03:06:14.494129 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet dca04d171deb488690ea58bd75995021 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.494264 14811 tablet_bootstrap.cc:654] T bda0e0238da246feb36450f40dd061e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.495851 14823 ts_tablet_manager.cc:1918] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:14.495918 14823 log.cc:1198] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/e4994733ae904c99986cabe4c6efc503
I20250115 03:06:14.496130 14823 ts_tablet_manager.cc:1939] T e4994733ae904c99986cabe4c6efc503 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.496683 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet e4994733ae904c99986cabe4c6efc503 (table default.Table_db8a2673977247e0b14b7995f7e09661 [id=f7e4534c1f6f48f09e10b84c1d5c458c]) successfully deleted
W20250115 03:06:14.497423 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet f01c319ebd5148858de9ea1a608ddff3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f01c319ebd5148858de9ea1a608ddff3 already in progress: creating tablet
I20250115 03:06:14.498068 14809 tablet_bootstrap.cc:492] T 0580a8f7b2984162b9275d74334b1905 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.498150 14809 ts_tablet_manager.cc:1397] T 0580a8f7b2984162b9275d74334b1905 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20250115 03:06:14.498422 14819 ts_tablet_manager.cc:1918] T bcf6745e549d47cdbdf97c2dff6997ad P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.498494 14819 log.cc:1198] T bcf6745e549d47cdbdf97c2dff6997ad P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/bcf6745e549d47cdbdf97c2dff6997ad
I20250115 03:06:14.498481 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 382116b1fb6a4f888ab1d55c29917ec8
I20250115 03:06:14.498558 14809 raft_consensus.cc:357] T 0580a8f7b2984162b9275d74334b1905 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.498654 14809 raft_consensus.cc:383] T 0580a8f7b2984162b9275d74334b1905 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.498677 14819 ts_tablet_manager.cc:1939] T bcf6745e549d47cdbdf97c2dff6997ad P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.498692 14809 raft_consensus.cc:738] T 0580a8f7b2984162b9275d74334b1905 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.498811 14809 consensus_queue.cc:260] T 0580a8f7b2984162b9275d74334b1905 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.499083 14809 ts_tablet_manager.cc:1428] T 0580a8f7b2984162b9275d74334b1905 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.499248 14809 tablet_bootstrap.cc:492] T 6629f2415b554c1c95927c721532fcfc P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.499560 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet ef002211ee1c4a149232c8e433277add with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.500272 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 382116b1fb6a4f888ab1d55c29917ec8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.500998 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet f54942abdfb748f2a939993a3c2d5db0 with delete_type TABLET_DATA_DELETED (replaced by tablet 382116b1fb6a4f888ab1d55c29917ec8) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.501262 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 382116b1fb6a4f888ab1d55c29917ec8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.501464 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 382116b1fb6a4f888ab1d55c29917ec8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.502070 14809 tablet_bootstrap.cc:654] T 6629f2415b554c1c95927c721532fcfc P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.502574 14823 tablet_replica.cc:331] T f54942abdfb748f2a939993a3c2d5db0 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.502696 14823 raft_consensus.cc:2238] T f54942abdfb748f2a939993a3c2d5db0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.502784 14823 raft_consensus.cc:2267] T f54942abdfb748f2a939993a3c2d5db0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.504760 14823 ts_tablet_manager.cc:1905] T f54942abdfb748f2a939993a3c2d5db0 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.505048 14165 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_0b4f00236d814b8d98a6530e3825328f [id=ff4681cab8ce44ceb45ed76c70c1132e] alter complete (version 3)
I20250115 03:06:14.506214 14812 tablet_bootstrap.cc:492] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.506294 14812 ts_tablet_manager.cc:1397] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20250115 03:06:14.506531 14166 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 0580a8f7b2984162b9275d74334b1905 (table default.Table_3e704f1e03e34a2ca2377db61214c5a0 [id=0790b08615854d3688f940aba20b6520]). Expected version 1 got 0
I20250115 03:06:14.506697 14812 raft_consensus.cc:357] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.506794 14812 raft_consensus.cc:383] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.506824 14812 raft_consensus.cc:738] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.507140 14812 consensus_queue.cc:260] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.507413 14812 ts_tablet_manager.cc:1428] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.507571 14812 tablet_bootstrap.cc:492] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.508176 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet bcf6745e549d47cdbdf97c2dff6997ad (table default.Table_28b265c831a944c7bff1a3ee55a86c64 [id=65f79614672344d88c2ec541c43da990]) successfully deleted
I20250115 03:06:14.508600 14812 tablet_bootstrap.cc:654] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.508599 14811 tablet_bootstrap.cc:492] T bda0e0238da246feb36450f40dd061e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.508672 14811 ts_tablet_manager.cc:1397] T bda0e0238da246feb36450f40dd061e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20250115 03:06:14.509094 14811 raft_consensus.cc:357] T bda0e0238da246feb36450f40dd061e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.509192 14811 raft_consensus.cc:383] T bda0e0238da246feb36450f40dd061e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.509224 14811 raft_consensus.cc:738] T bda0e0238da246feb36450f40dd061e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.509328 14811 consensus_queue.cc:260] T bda0e0238da246feb36450f40dd061e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.509575 14811 ts_tablet_manager.cc:1428] T bda0e0238da246feb36450f40dd061e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.509999 14811 tablet_bootstrap.cc:492] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.510308 14527 tablet_service.cc:1467] Processing CreateTablet for tablet a4c3ff892f9b4dc484ffc80d7c5934c4 (DEFAULT_TABLE table=default.Table_d85908ffc4694aaabd57c6ae118cdb06 [id=f70558594b8348579a0ad5d4c8fd8885]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.510571 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4c3ff892f9b4dc484ffc80d7c5934c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.511114 14811 tablet_bootstrap.cc:654] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.512284 14664 tablet_service.cc:1467] Processing CreateTablet for tablet a4c3ff892f9b4dc484ffc80d7c5934c4 (DEFAULT_TABLE table=default.Table_d85908ffc4694aaabd57c6ae118cdb06 [id=f70558594b8348579a0ad5d4c8fd8885]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.512476 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 50df3dd199bc4283bc1836d1917988a8
I20250115 03:06:14.512503 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4c3ff892f9b4dc484ffc80d7c5934c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.513051 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 50df3dd199bc4283bc1836d1917988a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.513329 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 50df3dd199bc4283bc1836d1917988a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.513576 14399 tablet_service.cc:1467] Processing CreateTablet for tablet a4c3ff892f9b4dc484ffc80d7c5934c4 (DEFAULT_TABLE table=default.Table_d85908ffc4694aaabd57c6ae118cdb06 [id=f70558594b8348579a0ad5d4c8fd8885]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.513830 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4c3ff892f9b4dc484ffc80d7c5934c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.515697 14820 tablet_replica.cc:331] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.515866 14820 raft_consensus.cc:2238] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:14.516309 14820 raft_consensus.cc:2267] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.516448 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 50df3dd199bc4283bc1836d1917988a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:14.516700 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 6f83dd9a2d13420daefdacdb0f565a7d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f83dd9a2d13420daefdacdb0f565a7d already in progress: creating tablet
I20250115 03:06:14.516908 14820 ts_tablet_manager.cc:1905] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.517122 15153 raft_consensus.cc:491] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.517201 15153 raft_consensus.cc:513] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.517549 15153 leader_election.cc:290] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.517913 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50df3dd199bc4283bc1836d1917988a8" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.518101 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50df3dd199bc4283bc1836d1917988a8" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:14.518301 14628 leader_election.cc:343] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.518334 14823 ts_tablet_manager.cc:1918] T f54942abdfb748f2a939993a3c2d5db0 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.518410 14823 log.cc:1198] T f54942abdfb748f2a939993a3c2d5db0 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/f54942abdfb748f2a939993a3c2d5db0
I20250115 03:06:14.518602 14811 tablet_bootstrap.cc:492] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
W20250115 03:06:14.518600 14626 leader_election.cc:343] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.518676 14626 leader_election.cc:304] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.518828 14811 ts_tablet_manager.cc:1397] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20250115 03:06:14.518862 15153 raft_consensus.cc:2743] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:14.519066 14819 tablet_replica.cc:331] T dca04d171deb488690ea58bd75995021 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.519243 14811 raft_consensus.cc:357] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.519327 14811 raft_consensus.cc:383] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.519357 14811 raft_consensus.cc:738] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.519506 14819 raft_consensus.cc:2238] T dca04d171deb488690ea58bd75995021 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.519650 14819 raft_consensus.cc:2267] T dca04d171deb488690ea58bd75995021 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.519968 14811 consensus_queue.cc:260] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.518604 14823 ts_tablet_manager.cc:1939] T f54942abdfb748f2a939993a3c2d5db0 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.520697 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0b4f00236d814b8d98a6530e3825328f" } new_table_name: "default.Table_96c8d5f733034ee3a6dd8a61dabfd288" modify_external_catalogs: true
I20250115 03:06:14.521355 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet f54942abdfb748f2a939993a3c2d5db0 (table default.Table_5e6d1abc27d042dba8506da7b49a6ee3 [id=a5c946319e8a4be5ba4a01a40a9ad0b7]) successfully deleted
I20250115 03:06:14.521586 14819 ts_tablet_manager.cc:1905] T dca04d171deb488690ea58bd75995021 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.521884 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_890029e0a8f049bcaa97eb1e2b702133"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.522220 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_890029e0a8f049bcaa97eb1e2b702133 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.523109 14811 ts_tablet_manager.cc:1428] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:14.523268 14811 tablet_bootstrap.cc:492] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
W20250115 03:06:14.523396 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 382116b1fb6a4f888ab1d55c29917ec8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 382116b1fb6a4f888ab1d55c29917ec8 already in progress: creating tablet
I20250115 03:06:14.523633 14823 tablet_replica.cc:331] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.523798 14823 raft_consensus.cc:2238] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.523929 14823 raft_consensus.cc:2267] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.524138 14811 tablet_bootstrap.cc:654] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.524421 14823 ts_tablet_manager.cc:1905] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.524696 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_faa8557664e04a1c88f455c0f098a9d8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:14.525121 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_6e69d6da0dc24bfeb36a8d4a43db72e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.525390 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6e69d6da0dc24bfeb36a8d4a43db72e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.526033 14819 ts_tablet_manager.cc:1918] T dca04d171deb488690ea58bd75995021 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20250115 03:06:14.526788 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_faa8557664e04a1c88f455c0f098a9d8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.527745 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet 55f01d704b174b2ea108ea6f1df763df from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:14.526436 14809 tablet_bootstrap.cc:492] T 6629f2415b554c1c95927c721532fcfc P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.527909 14809 ts_tablet_manager.cc:1397] T 6629f2415b554c1c95927c721532fcfc P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.000s
I20250115 03:06:14.528316 14809 raft_consensus.cc:357] T 6629f2415b554c1c95927c721532fcfc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.528429 14809 raft_consensus.cc:383] T 6629f2415b554c1c95927c721532fcfc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.528461 14809 raft_consensus.cc:738] T 6629f2415b554c1c95927c721532fcfc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.528563 14809 consensus_queue.cc:260] T 6629f2415b554c1c95927c721532fcfc P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.528828 14809 ts_tablet_manager.cc:1428] T 6629f2415b554c1c95927c721532fcfc P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.529592 14809 tablet_bootstrap.cc:492] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.529664 14812 tablet_bootstrap.cc:492] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.529734 14812 ts_tablet_manager.cc:1397] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.001s
I20250115 03:06:14.530164 14812 raft_consensus.cc:357] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.530253 14812 raft_consensus.cc:383] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.530283 14812 raft_consensus.cc:738] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.530388 14820 ts_tablet_manager.cc:1918] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:14.530392 14812 consensus_queue.cc:260] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.530441 14820 log.cc:1198] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/e4994733ae904c99986cabe4c6efc503
I20250115 03:06:14.530594 14809 tablet_bootstrap.cc:654] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.530663 14812 ts_tablet_manager.cc:1428] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:14.530671 14820 ts_tablet_manager.cc:1939] T e4994733ae904c99986cabe4c6efc503 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.530834 14812 tablet_bootstrap.cc:492] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.531677 14812 tablet_bootstrap.cc:654] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.532757 14820 tablet_replica.cc:331] T 6629f2415b554c1c95927c721532fcfc P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.532857 14820 raft_consensus.cc:2238] T 6629f2415b554c1c95927c721532fcfc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.532963 14820 raft_consensus.cc:2267] T 6629f2415b554c1c95927c721532fcfc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.534462 14819 log.cc:1198] T dca04d171deb488690ea58bd75995021 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/dca04d171deb488690ea58bd75995021
I20250115 03:06:14.534735 14819 ts_tablet_manager.cc:1939] T dca04d171deb488690ea58bd75995021 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.537362 14820 ts_tablet_manager.cc:1905] T 6629f2415b554c1c95927c721532fcfc P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.537343 15426 tablet.cc:1721] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:14.538260 14166 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c (table default.Table_4bfe9fa1f7a143d684bc948b47deb8d8 [id=339b61af27cb486aaaadf8730ab21ce7]). Expected version 1 got 0
I20250115 03:06:14.540529 14664 tablet_service.cc:1467] Processing CreateTablet for tablet e4f688e4c40d4653929432b13508348a (DEFAULT_TABLE table=default.Table_cd69692684cd401a9c6f9d24e374582c [id=5beb8c6fadc742c4bd0b5171ed51d0e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.540678 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 304fab45b38e4426ae6961ae786e5659 (DEFAULT_TABLE table=default.Table_1ad8c0817ede4bbd847c01814eebfa83 [id=526370a40a0d4f7aa96c8348412d8b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.540760 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4f688e4c40d4653929432b13508348a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.540885 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 304fab45b38e4426ae6961ae786e5659. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.541589 14820 ts_tablet_manager.cc:1918] T 6629f2415b554c1c95927c721532fcfc P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.541567 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 84ce55f729224f78b420e57977419add (DEFAULT_TABLE table=default.Table_ff21d7bc463d43cfa3330f7288ba7a55 [id=5de066e9f2174fff9303cba23eeec988]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.541654 14820 log.cc:1198] T 6629f2415b554c1c95927c721532fcfc P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/6629f2415b554c1c95927c721532fcfc
I20250115 03:06:14.541772 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84ce55f729224f78b420e57977419add. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.541842 14820 ts_tablet_manager.cc:1939] T 6629f2415b554c1c95927c721532fcfc P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.535727 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 6629f2415b554c1c95927c721532fcfc with delete_type TABLET_DATA_DELETED (replaced by tablet 5b7022db9edc4de8909e16769a3ea136) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.536636 15421 tablet.cc:1721] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:14.543371 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 82be772e05d443749914004aea8af300 (DEFAULT_TABLE table=default.Table_3863dfaf57694ee7b4a9ba1e400d7c9d [id=831e8e5e33ae41368950b0b8fce37e9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.543592 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82be772e05d443749914004aea8af300. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.543637 15153 raft_consensus.cc:491] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.543656 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 6864f88bf6454ee980f624ec2f2d9fb6 (DEFAULT_TABLE table=default.Table_fa2e1dcccf5e4b34971533a5d66118d7 [id=18e0f06feeda4bca860e6f1bc6eab15b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.543694 15153 raft_consensus.cc:513] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.543850 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6864f88bf6454ee980f624ec2f2d9fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.544010 15153 leader_election.cc:290] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.538456 15401 consensus_queue.cc:1035] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.546125 15274 tablet.cc:1721] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:14.546586 14823 ts_tablet_manager.cc:1918] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.546640 14823 log.cc:1198] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/50df3dd199bc4283bc1836d1917988a8
I20250115 03:06:14.546820 14823 ts_tablet_manager.cc:1939] T 50df3dd199bc4283bc1836d1917988a8 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.539723 14401 tablet_service.cc:1467] Processing CreateTablet for tablet e4f688e4c40d4653929432b13508348a (DEFAULT_TABLE table=default.Table_cd69692684cd401a9c6f9d24e374582c [id=5beb8c6fadc742c4bd0b5171ed51d0e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.540279 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 304fab45b38e4426ae6961ae786e5659 (DEFAULT_TABLE table=default.Table_1ad8c0817ede4bbd847c01814eebfa83 [id=526370a40a0d4f7aa96c8348412d8b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.541114 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 84ce55f729224f78b420e57977419add (DEFAULT_TABLE table=default.Table_ff21d7bc463d43cfa3330f7288ba7a55 [id=5de066e9f2174fff9303cba23eeec988]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.541353 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 82be772e05d443749914004aea8af300 (DEFAULT_TABLE table=default.Table_3863dfaf57694ee7b4a9ba1e400d7c9d [id=831e8e5e33ae41368950b0b8fce37e9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.547976 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ff21d7bc463d43cfa3330f7288ba7a55" } modify_external_catalogs: true
I20250115 03:06:14.548031 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet e4994733ae904c99986cabe4c6efc503 (table default.Table_db8a2673977247e0b14b7995f7e09661 [id=f7e4534c1f6f48f09e10b84c1d5c458c]) successfully deleted
I20250115 03:06:14.548076 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ff21d7bc463d43cfa3330f7288ba7a55" } modify_external_catalogs: true
W20250115 03:06:14.548173 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet f01c319ebd5148858de9ea1a608ddff3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f01c319ebd5148858de9ea1a608ddff3 already in progress: creating tablet
I20250115 03:06:14.548276 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 6629f2415b554c1c95927c721532fcfc (table default.Table_35d1dd748a6f4e79adced77fe7b2d08d [id=f94088b0776f4be990ddcec04afbca43]) successfully deleted
I20250115 03:06:14.548305 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_35005559d2f34672bc3e70f6bbfb31d6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.548595 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_35005559d2f34672bc3e70f6bbfb31d6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.549064 14528 tablet_service.cc:1467] Processing CreateTablet for tablet 3cbc91ad00b54ede804c3629bcfe7fe6 (DEFAULT_TABLE table=default.Table_271f46df55f14ffd9b7bb48352620543 [id=44488be7335b488a9ff7cf913ce4273a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.549338 14527 tablet_service.cc:1467] Processing CreateTablet for tablet e4f688e4c40d4653929432b13508348a (DEFAULT_TABLE table=default.Table_cd69692684cd401a9c6f9d24e374582c [id=5beb8c6fadc742c4bd0b5171ed51d0e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.549676 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 304fab45b38e4426ae6961ae786e5659 (DEFAULT_TABLE table=default.Table_1ad8c0817ede4bbd847c01814eebfa83 [id=526370a40a0d4f7aa96c8348412d8b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.548040 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_75c071e03ba44140bce71e920f31715d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.549983 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_75c071e03ba44140bce71e920f31715d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.550007 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 84ce55f729224f78b420e57977419add (DEFAULT_TABLE table=default.Table_ff21d7bc463d43cfa3330f7288ba7a55 [id=5de066e9f2174fff9303cba23eeec988]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.550328 14529 tablet_service.cc:1467] Processing CreateTablet for tablet 82be772e05d443749914004aea8af300 (DEFAULT_TABLE table=default.Table_3863dfaf57694ee7b4a9ba1e400d7c9d [id=831e8e5e33ae41368950b0b8fce37e9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.550642 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 6864f88bf6454ee980f624ec2f2d9fb6 (DEFAULT_TABLE table=default.Table_fa2e1dcccf5e4b34971533a5d66118d7 [id=18e0f06feeda4bca860e6f1bc6eab15b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.550956 14537 tablet_service.cc:1467] Processing CreateTablet for tablet f70a69acbdc049beb9d98eada09d9eb8 (DEFAULT_TABLE table=default.Table_db8a2673977247e0b14b7995f7e09661 [id=f7e4534c1f6f48f09e10b84c1d5c458c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.551266 14811 tablet_bootstrap.cc:492] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.551357 14811 ts_tablet_manager.cc:1397] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.028s user 0.001s sys 0.000s
I20250115 03:06:14.551453 14820 tablet_replica.cc:331] T ef002211ee1c4a149232c8e433277add P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.551573 14820 raft_consensus.cc:2238] T ef002211ee1c4a149232c8e433277add P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.551823 14811 raft_consensus.cc:357] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.551921 14811 raft_consensus.cc:383] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.551952 14811 raft_consensus.cc:738] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.552129 14811 consensus_queue.cc:260] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.552533 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 84ce55f729224f78b420e57977419add
I20250115 03:06:14.552392 14811 ts_tablet_manager.cc:1428] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.553543 14811 tablet_bootstrap.cc:492] T 0d25aa6a292b454f8f451d6493d940c3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.553661 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet dca04d171deb488690ea58bd75995021 (table default.Table_3bd81cb8bc2d4b74ae3dbcbdb74aae8b [id=05f9fd2ebec34d2299ea59c47a74c828]) successfully deleted
I20250115 03:06:14.553988 14396 tablet_service.cc:1467] Processing CreateTablet for tablet f70a69acbdc049beb9d98eada09d9eb8 (DEFAULT_TABLE table=default.Table_db8a2673977247e0b14b7995f7e09661 [id=f7e4534c1f6f48f09e10b84c1d5c458c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.554150 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 84ce55f729224f78b420e57977419add with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.554231 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c76ab180b26487896f4360efe47b2c7" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.554397 14665 tablet_service.cc:1467] Processing CreateTablet for tablet f70a69acbdc049beb9d98eada09d9eb8 (DEFAULT_TABLE table=default.Table_db8a2673977247e0b14b7995f7e09661 [id=f7e4534c1f6f48f09e10b84c1d5c458c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.554598 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f70a69acbdc049beb9d98eada09d9eb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.554761 14820 raft_consensus.cc:2267] T ef002211ee1c4a149232c8e433277add P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.556313 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 84ce55f729224f78b420e57977419add with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.555907 14820 ts_tablet_manager.cc:1905] T ef002211ee1c4a149232c8e433277add P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.553064 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c76ab180b26487896f4360efe47b2c7" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:14.557227 14626 leader_election.cc:343] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:14.557488 14628 leader_election.cc:343] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.557551 14628 leader_election.cc:304] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.557725 15153 raft_consensus.cc:2743] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:14.558070 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f70a69acbdc049beb9d98eada09d9eb8. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:14.558053 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 382116b1fb6a4f888ab1d55c29917ec8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 382116b1fb6a4f888ab1d55c29917ec8 already in progress: creating tablet
I20250115 03:06:14.558259 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 50df3dd199bc4283bc1836d1917988a8 (table default.Table_d75a227f24d644ceb2bda90d1f6ae998 [id=c72398a7988c48cc99ae0d244be1ac1a]) successfully deleted
I20250115 03:06:14.558367 14809 tablet_bootstrap.cc:492] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
W20250115 03:06:14.558384 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 84ce55f729224f78b420e57977419add because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84ce55f729224f78b420e57977419add already in progress: creating tablet
I20250115 03:06:14.558440 14809 ts_tablet_manager.cc:1397] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.000s
I20250115 03:06:14.558398 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 6864f88bf6454ee980f624ec2f2d9fb6 (DEFAULT_TABLE table=default.Table_fa2e1dcccf5e4b34971533a5d66118d7 [id=18e0f06feeda4bca860e6f1bc6eab15b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.558600 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6864f88bf6454ee980f624ec2f2d9fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.558877 14809 raft_consensus.cc:357] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.558966 14809 raft_consensus.cc:383] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.558995 14809 raft_consensus.cc:738] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.559105 14809 consensus_queue.cc:260] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.559327 14809 ts_tablet_manager.cc:1428] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:14.561093 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82be772e05d443749914004aea8af300. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.561690 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 3cbc91ad00b54ede804c3629bcfe7fe6 (DEFAULT_TABLE table=default.Table_271f46df55f14ffd9b7bb48352620543 [id=44488be7335b488a9ff7cf913ce4273a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.561892 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cbc91ad00b54ede804c3629bcfe7fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.563274 14819 tablet_replica.cc:331] T 6629f2415b554c1c95927c721532fcfc P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.563457 14819 raft_consensus.cc:2238] T 6629f2415b554c1c95927c721532fcfc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.563594 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4f688e4c40d4653929432b13508348a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.563731 14819 raft_consensus.cc:2267] T 6629f2415b554c1c95927c721532fcfc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.563926 15390 consensus_queue.cc:1035] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.564385 14819 ts_tablet_manager.cc:1905] T 6629f2415b554c1c95927c721532fcfc P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.565548 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84ce55f729224f78b420e57977419add. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.566990 15390 raft_consensus.cc:491] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.567055 15390 raft_consensus.cc:513] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.567363 15390 leader_election.cc:290] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.567721 14809 tablet_bootstrap.cc:492] T 29ef42586d164315b5fd54ba91227bad P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
W20250115 03:06:14.567925 14358 leader_election.cc:343] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: ef002211ee1c4a149232c8e433277add
I20250115 03:06:14.568287 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef002211ee1c4a149232c8e433277add" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.568442 14819 ts_tablet_manager.cc:1918] T 6629f2415b554c1c95927c721532fcfc P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.568565 14819 log.cc:1198] T 6629f2415b554c1c95927c721532fcfc P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/6629f2415b554c1c95927c721532fcfc
I20250115 03:06:14.568578 14809 tablet_bootstrap.cc:654] T 29ef42586d164315b5fd54ba91227bad P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:14.568758 14356 leader_election.cc:343] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: Tablet not RUNNING: STOPPED
I20250115 03:06:14.568810 14356 leader_election.cc:304] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:14.568917 14819 ts_tablet_manager.cc:1939] T 6629f2415b554c1c95927c721532fcfc P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.569001 15390 raft_consensus.cc:2743] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:14.569461 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 50df3dd199bc4283bc1836d1917988a8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 50df3dd199bc4283bc1836d1917988a8 already in progress: creating tablet
I20250115 03:06:14.569591 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 304fab45b38e4426ae6961ae786e5659. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.570590 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 6629f2415b554c1c95927c721532fcfc (table default.Table_35d1dd748a6f4e79adced77fe7b2d08d [id=f94088b0776f4be990ddcec04afbca43]) successfully deleted
I20250115 03:06:14.571448 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 3cbc91ad00b54ede804c3629bcfe7fe6 (DEFAULT_TABLE table=default.Table_271f46df55f14ffd9b7bb48352620543 [id=44488be7335b488a9ff7cf913ce4273a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.571643 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cbc91ad00b54ede804c3629bcfe7fe6. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:14.571135 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 84ce55f729224f78b420e57977419add because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84ce55f729224f78b420e57977419add already in progress: creating tablet
I20250115 03:06:14.574877 14554 raft_consensus.cc:1270] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:14.575229 14162 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_96c8d5f733034ee3a6dd8a61dabfd288 [id=ff4681cab8ce44ceb45ed76c70c1132e] alter complete (version 4)
I20250115 03:06:14.575285 15153 raft_consensus.cc:491] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.575357 15153 raft_consensus.cc:513] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.575680 15153 leader_election.cc:290] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.575934 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5504a9b8a4ea47e2b4cfd98c92e25d5c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.576009 14689 raft_consensus.cc:1270] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250115 03:06:14.576303 14628 leader_election.cc:343] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.576315 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5504a9b8a4ea47e2b4cfd98c92e25d5c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.576573 15401 consensus_queue.cc:1035] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:14.576606 14626 leader_election.cc:343] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.576664 14626 leader_election.cc:304] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.576828 15153 raft_consensus.cc:2743] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:14.577179 15390 consensus_queue.cc:1035] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.581012 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet ef002211ee1c4a149232c8e433277add with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.581279 14819 tablet_replica.cc:331] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.581647 14166 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a7ef00d45deb49578042744e5f66f14e" } modify_external_catalogs: true
I20250115 03:06:14.581667 14819 raft_consensus.cc:2238] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.581753 14166 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a7ef00d45deb49578042744e5f66f14e" } modify_external_catalogs: true
I20250115 03:06:14.581841 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet 84ce55f729224f78b420e57977419add with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.581897 14819 raft_consensus.cc:2267] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.584441 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84ce55f729224f78b420e57977419add. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.584581 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82be772e05d443749914004aea8af300. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.584745 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cbc91ad00b54ede804c3629bcfe7fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.585335 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4f688e4c40d4653929432b13508348a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.587324 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 304fab45b38e4426ae6961ae786e5659. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.587617 14858 raft_consensus.cc:491] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.587678 14858 raft_consensus.cc:513] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.587981 14858 leader_election.cc:290] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.588384 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "673f9e6cabdd4320b777181c787d15c7" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:14.588671 14493 leader_election.cc:343] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.588888 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6864f88bf6454ee980f624ec2f2d9fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.588882 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet 4cc6e937615b4aca8e9625d233452229 from {username='slave'} at 127.0.0.1:60432
W20250115 03:06:14.588994 14493 leader_election.cc:343] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 673f9e6cabdd4320b777181c787d15c7
I20250115 03:06:14.589056 14493 leader_election.cc:304] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.589179 14858 raft_consensus.cc:2743] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:14.589286 14819 ts_tablet_manager.cc:1905] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.590245 14812 tablet_bootstrap.cc:492] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.590319 14812 ts_tablet_manager.cc:1397] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.060s user 0.000s sys 0.001s
I20250115 03:06:14.590734 14812 raft_consensus.cc:357] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.590845 14812 raft_consensus.cc:383] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.590873 14812 raft_consensus.cc:738] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.590970 14812 consensus_queue.cc:260] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.591219 14812 ts_tablet_manager.cc:1428] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.591276 14400 tablet_service.cc:1467] Processing CreateTablet for tablet b1c0f6a5611647eb884c77f7a6d1cd4e (DEFAULT_TABLE table=default.Table_890029e0a8f049bcaa97eb1e2b702133 [id=384b6711c99840f094550519558797fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.591395 14812 tablet_bootstrap.cc:492] T 2dffa87697284dca8f07719f1c861bd4 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.591568 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1c0f6a5611647eb884c77f7a6d1cd4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.592278 14812 tablet_bootstrap.cc:654] T 2dffa87697284dca8f07719f1c861bd4 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.593573 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 7f67cbb929cd4ae589a14606aa168b46
I20250115 03:06:14.593784 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f70a69acbdc049beb9d98eada09d9eb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.594101 14811 tablet_bootstrap.cc:654] T 0d25aa6a292b454f8f451d6493d940c3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.594220 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 4cc6e937615b4aca8e9625d233452229
I20250115 03:06:14.594326 15151 consensus_queue.cc:1035] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.594442 14168 catalog_manager.cc:6596] ReplaceTablet: tablet 4cc6e937615b4aca8e9625d233452229 deleted and replaced by tablet acb43d614111477abb10def48a1dfb3b
I20250115 03:06:14.596055 14820 ts_tablet_manager.cc:1918] T ef002211ee1c4a149232c8e433277add P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.596264 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cc6e937615b4aca8e9625d233452229 with delete_type TABLET_DATA_DELETED (replaced by tablet acb43d614111477abb10def48a1dfb3b) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.596174 14820 log.cc:1198] T ef002211ee1c4a149232c8e433277add P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/ef002211ee1c4a149232c8e433277add
I20250115 03:06:14.596657 14528 tablet_service.cc:1467] Processing CreateTablet for tablet b1c0f6a5611647eb884c77f7a6d1cd4e (DEFAULT_TABLE table=default.Table_890029e0a8f049bcaa97eb1e2b702133 [id=384b6711c99840f094550519558797fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.596907 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1c0f6a5611647eb884c77f7a6d1cd4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.599104 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_90aedfcdb1274e1497cf36620b980768"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.599406 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_90aedfcdb1274e1497cf36620b980768 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.600171 14812 tablet_bootstrap.cc:492] T 2dffa87697284dca8f07719f1c861bd4 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.600240 14812 ts_tablet_manager.cc:1397] T 2dffa87697284dca8f07719f1c861bd4 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:14.600651 14812 raft_consensus.cc:357] T 2dffa87697284dca8f07719f1c861bd4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.600736 14812 raft_consensus.cc:383] T 2dffa87697284dca8f07719f1c861bd4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.600704 15421 tablet.cc:1721] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.600780 14812 raft_consensus.cc:738] T 2dffa87697284dca8f07719f1c861bd4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.600875 14812 consensus_queue.cc:260] T 2dffa87697284dca8f07719f1c861bd4 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.601182 14812 ts_tablet_manager.cc:1428] T 2dffa87697284dca8f07719f1c861bd4 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.601338 14812 tablet_bootstrap.cc:492] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.602186 14812 tablet_bootstrap.cc:654] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.603320 14530 tablet_service.cc:1467] Processing CreateTablet for tablet a1c93bfd3da44f9f8a8d585f2d21f1be (DEFAULT_TABLE table=default.Table_6e69d6da0dc24bfeb36a8d4a43db72e2 [id=64d20b1792044c5e8add735aaf75a6d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.603523 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1c93bfd3da44f9f8a8d585f2d21f1be. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.603618 15152 mvcc.cc:204] Tried to move back new op lower bound from 7114384894255448064 to 7114384892843053056. Current Snapshot: MvccSnapshot[applied={T|T < 7114384894255448064}]
I20250115 03:06:14.604827 15274 tablet.cc:1721] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.604995 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f67cbb929cd4ae589a14606aa168b46 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.605414 14809 tablet_bootstrap.cc:492] T 29ef42586d164315b5fd54ba91227bad P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.605501 14809 ts_tablet_manager.cc:1397] T 29ef42586d164315b5fd54ba91227bad P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.000s
I20250115 03:06:14.605906 14809 raft_consensus.cc:357] T 29ef42586d164315b5fd54ba91227bad P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.605993 14809 raft_consensus.cc:383] T 29ef42586d164315b5fd54ba91227bad P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.606022 14809 raft_consensus.cc:738] T 29ef42586d164315b5fd54ba91227bad P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.606261 14809 consensus_queue.cc:260] T 29ef42586d164315b5fd54ba91227bad P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.606813 14809 ts_tablet_manager.cc:1428] T 29ef42586d164315b5fd54ba91227bad P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.607008 14809 tablet_bootstrap.cc:492] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.607911 14809 tablet_bootstrap.cc:654] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.609036 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f67cbb929cd4ae589a14606aa168b46 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.609573 15151 raft_consensus.cc:491] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.609618 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cc6e937615b4aca8e9625d233452229 with delete_type TABLET_DATA_DELETED (replaced by tablet acb43d614111477abb10def48a1dfb3b) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.609650 15151 raft_consensus.cc:513] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.609843 14820 ts_tablet_manager.cc:1939] T ef002211ee1c4a149232c8e433277add P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.609985 15151 leader_election.cc:290] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.610105 15426 tablet.cc:1721] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.610154 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cc6e937615b4aca8e9625d233452229 with delete_type TABLET_DATA_DELETED (replaced by tablet acb43d614111477abb10def48a1dfb3b) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.610188 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d25aa6a292b454f8f451d6493d940c3" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.610340 14672 tablet_service.cc:1467] Processing CreateTablet for tablet b1c0f6a5611647eb884c77f7a6d1cd4e (DEFAULT_TABLE table=default.Table_890029e0a8f049bcaa97eb1e2b702133 [id=384b6711c99840f094550519558797fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.610468 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f67cbb929cd4ae589a14606aa168b46 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.610559 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1c0f6a5611647eb884c77f7a6d1cd4e. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:14.610473 14628 leader_election.cc:343] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.611029 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d25aa6a292b454f8f451d6493d940c3" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:14.611366 14626 leader_election.cc:343] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.611425 14626 leader_election.cc:304] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.611624 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet ef002211ee1c4a149232c8e433277add (table default.Table_f957fe3487ec4546a46a8392108003f0 [id=fb3a764b51e74e08a281d128e21466e3]) successfully deleted
W20250115 03:06:14.611764 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 382116b1fb6a4f888ab1d55c29917ec8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 382116b1fb6a4f888ab1d55c29917ec8 already in progress: creating tablet
W20250115 03:06:14.611866 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 50df3dd199bc4283bc1836d1917988a8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 50df3dd199bc4283bc1836d1917988a8 already in progress: creating tablet
W20250115 03:06:14.611948 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 84ce55f729224f78b420e57977419add because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 84ce55f729224f78b420e57977419add already in progress: creating tablet
W20250115 03:06:14.612025 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 4cc6e937615b4aca8e9625d233452229 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4cc6e937615b4aca8e9625d233452229 already in progress: creating tablet
W20250115 03:06:14.612121 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 7f67cbb929cd4ae589a14606aa168b46 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f67cbb929cd4ae589a14606aa168b46 already in progress: creating tablet
I20250115 03:06:14.612779 15151 raft_consensus.cc:2743] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:14.612761 14400 tablet_service.cc:1467] Processing CreateTablet for tablet a1c93bfd3da44f9f8a8d585f2d21f1be (DEFAULT_TABLE table=default.Table_6e69d6da0dc24bfeb36a8d4a43db72e2 [id=64d20b1792044c5e8add735aaf75a6d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.612995 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1c93bfd3da44f9f8a8d585f2d21f1be. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.613654 14819 ts_tablet_manager.cc:1918] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.613730 14819 log.cc:1198] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/ef002211ee1c4a149232c8e433277add
I20250115 03:06:14.613916 14819 ts_tablet_manager.cc:1939] T ef002211ee1c4a149232c8e433277add P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.614856 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b44636b0d90d4972aceaa7d169a3a477" } new_table_name: "default.Table_50c659a42a154d46a26879df0101bf23" modify_external_catalogs: true
I20250115 03:06:14.615667 14811 tablet_bootstrap.cc:492] T 0d25aa6a292b454f8f451d6493d940c3 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.615748 14811 ts_tablet_manager.cc:1397] T 0d25aa6a292b454f8f451d6493d940c3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.062s user 0.000s sys 0.004s
I20250115 03:06:14.616150 14811 raft_consensus.cc:357] T 0d25aa6a292b454f8f451d6493d940c3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.616236 14811 raft_consensus.cc:383] T 0d25aa6a292b454f8f451d6493d940c3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.616266 14811 raft_consensus.cc:738] T 0d25aa6a292b454f8f451d6493d940c3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.616353 14811 consensus_queue.cc:260] T 0d25aa6a292b454f8f451d6493d940c3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.616632 14811 ts_tablet_manager.cc:1428] T 0d25aa6a292b454f8f451d6493d940c3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.616792 14811 tablet_bootstrap.cc:492] T 6f83dd9a2d13420daefdacdb0f565a7d P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.617656 14811 tablet_bootstrap.cc:654] T 6f83dd9a2d13420daefdacdb0f565a7d P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.618389 14812 tablet_bootstrap.cc:492] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.618459 14812 ts_tablet_manager.cc:1397] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20250115 03:06:14.618860 14812 raft_consensus.cc:357] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.618964 14812 raft_consensus.cc:383] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.618997 14812 raft_consensus.cc:738] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
W20250115 03:06:14.619114 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 4cc6e937615b4aca8e9625d233452229 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4cc6e937615b4aca8e9625d233452229 already in progress: creating tablet
I20250115 03:06:14.619107 14812 consensus_queue.cc:260] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:14.619230 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 7f67cbb929cd4ae589a14606aa168b46 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f67cbb929cd4ae589a14606aa168b46 already in progress: creating tablet
I20250115 03:06:14.619390 14812 ts_tablet_manager.cc:1428] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.619462 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet ef002211ee1c4a149232c8e433277add (table default.Table_f957fe3487ec4546a46a8392108003f0 [id=fb3a764b51e74e08a281d128e21466e3]) successfully deleted
I20250115 03:06:14.619575 14812 tablet_bootstrap.cc:492] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.620174 15390 raft_consensus.cc:491] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.620246 15390 raft_consensus.cc:513] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.620404 14812 tablet_bootstrap.cc:654] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.620559 15390 leader_election.cc:290] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.620994 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bda0e0238da246feb36450f40dd061e5" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:14.621021 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 7f67cbb929cd4ae589a14606aa168b46 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7f67cbb929cd4ae589a14606aa168b46 already in progress: creating tablet
I20250115 03:06:14.621091 14557 raft_consensus.cc:2463] T bda0e0238da246feb36450f40dd061e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
W20250115 03:06:14.621137 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 4cc6e937615b4aca8e9625d233452229 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4cc6e937615b4aca8e9625d233452229 already in progress: creating tablet
I20250115 03:06:14.621336 14356 leader_election.cc:304] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:14.621477 15390 raft_consensus.cc:2798] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.621443 14672 tablet_service.cc:1467] Processing CreateTablet for tablet a1c93bfd3da44f9f8a8d585f2d21f1be (DEFAULT_TABLE table=default.Table_6e69d6da0dc24bfeb36a8d4a43db72e2 [id=64d20b1792044c5e8add735aaf75a6d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.621524 15390 raft_consensus.cc:491] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.621563 15390 raft_consensus.cc:3054] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.621646 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1c93bfd3da44f9f8a8d585f2d21f1be. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.622421 15390 raft_consensus.cc:513] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.622512 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bda0e0238da246feb36450f40dd061e5" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:14.622611 14689 raft_consensus.cc:2463] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:14.622737 15390 leader_election.cc:290] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.623132 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bda0e0238da246feb36450f40dd061e5" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:14.623220 14557 raft_consensus.cc:3054] T bda0e0238da246feb36450f40dd061e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.624030 14557 raft_consensus.cc:2463] T bda0e0238da246feb36450f40dd061e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:14.624500 14356 leader_election.cc:304] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:14.624501 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bda0e0238da246feb36450f40dd061e5" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:14.624593 14689 raft_consensus.cc:3054] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.624642 15390 raft_consensus.cc:2798] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:14.624708 15390 raft_consensus.cc:695] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:14.624825 15390 consensus_queue.cc:237] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.625411 14689 raft_consensus.cc:2463] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:14.625698 15153 consensus_queue.cc:1035] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.629194 15426 tablet.cc:1721] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:14.630210 15421 tablet.cc:1721] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:14.632954 14809 tablet_bootstrap.cc:492] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.633076 14809 ts_tablet_manager.cc:1397] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20250115 03:06:14.633515 14809 raft_consensus.cc:357] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.633661 14809 raft_consensus.cc:383] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.633729 14809 raft_consensus.cc:738] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.633899 14809 consensus_queue.cc:260] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.634255 14809 ts_tablet_manager.cc:1428] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.634263 15274 tablet.cc:1721] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:14.634447 14809 tablet_bootstrap.cc:492] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.635776 14809 tablet_bootstrap.cc:654] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.636348 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_ec8ee0d483474271a3b369a2bcde593d [id=d8084de64a264d28a3370971617a694c] alter complete (version 1)
I20250115 03:06:14.638283 14809 tablet_bootstrap.cc:492] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.638341 14811 tablet_bootstrap.cc:492] T 6f83dd9a2d13420daefdacdb0f565a7d P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.638352 14809 ts_tablet_manager.cc:1397] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250115 03:06:14.638459 14811 ts_tablet_manager.cc:1397] T 6f83dd9a2d13420daefdacdb0f565a7d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20250115 03:06:14.638707 14809 raft_consensus.cc:357] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.638828 14809 raft_consensus.cc:383] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.638885 14809 raft_consensus.cc:738] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.638904 14811 raft_consensus.cc:357] T 6f83dd9a2d13420daefdacdb0f565a7d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.639006 14811 raft_consensus.cc:383] T 6f83dd9a2d13420daefdacdb0f565a7d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.638996 14809 consensus_queue.cc:260] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.639048 14811 raft_consensus.cc:738] T 6f83dd9a2d13420daefdacdb0f565a7d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.639184 14811 consensus_queue.cc:260] T 6f83dd9a2d13420daefdacdb0f565a7d P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.639292 14809 ts_tablet_manager.cc:1428] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.639437 14809 tablet_bootstrap.cc:492] T 50df3dd199bc4283bc1836d1917988a8 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.639454 14811 ts_tablet_manager.cc:1428] T 6f83dd9a2d13420daefdacdb0f565a7d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.639626 14811 tablet_bootstrap.cc:492] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.640530 14811 tablet_bootstrap.cc:654] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.642024 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_fa83f65f02b04ad491cee3b2a617927f" } modify_external_catalogs: true
I20250115 03:06:14.642179 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_fa83f65f02b04ad491cee3b2a617927f" } modify_external_catalogs: true
I20250115 03:06:14.643239 14812 tablet_bootstrap.cc:492] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.643313 14812 ts_tablet_manager.cc:1397] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.000s
I20250115 03:06:14.643720 14812 raft_consensus.cc:357] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.643815 14812 raft_consensus.cc:383] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.643847 14812 raft_consensus.cc:738] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.643981 14812 consensus_queue.cc:260] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.644345 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 673f9e6cabdd4320b777181c787d15c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.644490 14820 tablet_replica.cc:331] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.644603 14820 raft_consensus.cc:2238] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.644685 14820 raft_consensus.cc:2267] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.645989 14812 ts_tablet_manager.cc:1428] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:14.647264 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_8350ec7f96f743449d3122b98ad087d7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:14.647404 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d1e81884e3a94c6abc91f7a6e9646351" } new_table_name: "default.Table_d49d96cab47241d4836768cbc9cd552b" modify_external_catalogs: true
W20250115 03:06:14.647570 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8350ec7f96f743449d3122b98ad087d7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.648317 14812 tablet_bootstrap.cc:492] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.649307 14812 tablet_bootstrap.cc:654] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.650255 14811 tablet_bootstrap.cc:492] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.650328 14811 ts_tablet_manager.cc:1397] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20250115 03:06:14.650697 14811 raft_consensus.cc:357] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.650807 14811 raft_consensus.cc:383] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.650838 14811 raft_consensus.cc:738] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.650946 14811 consensus_queue.cc:260] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.651175 14811 ts_tablet_manager.cc:1428] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:14.651257 14820 ts_tablet_manager.cc:1905] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.651360 14811 tablet_bootstrap.cc:492] T 55dcc7ac82ac44ab9d85055720760d3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.651575 14809 tablet_bootstrap.cc:654] T 50df3dd199bc4283bc1836d1917988a8 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.652022 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet 68974a1815cb482d8b28d54f42634608 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:14.652169 14811 tablet_bootstrap.cc:654] T 55dcc7ac82ac44ab9d85055720760d3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.652436 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet f21761ff87ac43d5b9b47b5905fa5867 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:14.653602 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_39e0d37b84fc42c0807a4cbffb7bae0a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:14.653772 14812 tablet_bootstrap.cc:492] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.653854 14820 ts_tablet_manager.cc:1918] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.653878 14812 ts_tablet_manager.cc:1397] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20250115 03:06:14.653906 14817 raft_consensus.cc:491] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20250115 03:06:14.653929 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_39e0d37b84fc42c0807a4cbffb7bae0a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.653982 14817 raft_consensus.cc:513] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.654297 14817 leader_election.cc:290] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.654315 14812 raft_consensus.cc:357] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.654404 14812 raft_consensus.cc:383] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.654453 14812 raft_consensus.cc:738] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.654788 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ef42586d164315b5fd54ba91227bad" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:14.654932 14689 raft_consensus.cc:2463] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:14.654973 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ef42586d164315b5fd54ba91227bad" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.655016 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_849d2ef8d13549b7bb6bfb03314d998b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:14.655071 14421 raft_consensus.cc:2463] T 29ef42586d164315b5fd54ba91227bad P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:14.653916 14820 log.cc:1198] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/673f9e6cabdd4320b777181c787d15c7
W20250115 03:06:14.655279 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_849d2ef8d13549b7bb6bfb03314d998b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.655388 14493 leader_election.cc:304] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:14.655476 14820 ts_tablet_manager.cc:1939] T 673f9e6cabdd4320b777181c787d15c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.656014 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 673f9e6cabdd4320b777181c787d15c7 (table default.Table_42a1977fa70d4f16b43bf861b66524a7 [id=c9b28587ba934521bd4158dfaf416f6e]) successfully deleted
I20250115 03:06:14.656294 14817 raft_consensus.cc:2798] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.656354 14817 raft_consensus.cc:491] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.656397 14817 raft_consensus.cc:3054] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.657215 14817 raft_consensus.cc:513] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.657487 14817 leader_election.cc:290] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:14.657938 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 13ce129d8a9245baab7c772f0f5fbaa7 consensus state has no leader
I20250115 03:06:14.658020 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ef42586d164315b5fd54ba91227bad" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:14.658110 14689 raft_consensus.cc:3054] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.658110 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_8ecc7de5c6b3497fb2d931c10ec6b373"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:14.654594 14812 consensus_queue.cc:260] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:14.658362 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8ecc7de5c6b3497fb2d931c10ec6b373 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.658967 14689 raft_consensus.cc:2463] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:14.659162 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ef42586d164315b5fd54ba91227bad" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:14.659245 14421 raft_consensus.cc:3054] T 29ef42586d164315b5fd54ba91227bad P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.660071 14421 raft_consensus.cc:2463] T 29ef42586d164315b5fd54ba91227bad P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:14.660316 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet b561abc942014fb79f22fd6a9fec38d0
I20250115 03:06:14.660342 14493 leader_election.cc:304] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:14.660586 14817 raft_consensus.cc:2798] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:14.660656 14817 raft_consensus.cc:695] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:14.660710 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet b561abc942014fb79f22fd6a9fec38d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.660769 14817 consensus_queue.cc:237] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.661162 14819 tablet_replica.cc:331] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.661276 14819 raft_consensus.cc:2238] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:14.660172 14812 ts_tablet_manager.cc:1428] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
I20250115 03:06:14.661433 14819 raft_consensus.cc:2267] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.661581 14812 tablet_bootstrap.cc:492] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.662437 14812 tablet_bootstrap.cc:654] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.662909 14819 ts_tablet_manager.cc:1905] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.663136 14858 consensus_queue.cc:1035] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.663292 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet f21761ff87ac43d5b9b47b5905fa5867
I20250115 03:06:14.663553 14171 catalog_manager.cc:6596] ReplaceTablet: tablet f21761ff87ac43d5b9b47b5905fa5867 deleted and replaced by tablet 3a0bb9b5093b454d921bf1150d631565
I20250115 03:06:14.664026 15153 raft_consensus.cc:491] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.664100 15153 raft_consensus.cc:513] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.664426 15153 leader_election.cc:290] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.664762 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet f21761ff87ac43d5b9b47b5905fa5867 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a0bb9b5093b454d921bf1150d631565) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.664902 14819 ts_tablet_manager.cc:1918] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:14.665007 14819 log.cc:1198] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/b561abc942014fb79f22fd6a9fec38d0
I20250115 03:06:14.665064 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet f21761ff87ac43d5b9b47b5905fa5867 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a0bb9b5093b454d921bf1150d631565) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.665213 14823 tablet_replica.cc:331] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.665341 14823 raft_consensus.cc:2238] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.665442 14823 raft_consensus.cc:2267] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.665611 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "080224910d554389b646e2c250fada2d" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:14.665936 14626 leader_election.cc:343] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.666085 15453 raft_consensus.cc:491] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.666152 15453 raft_consensus.cc:513] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.666323 14823 ts_tablet_manager.cc:1905] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.667888 15453 leader_election.cc:290] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.668324 14823 ts_tablet_manager.cc:1918] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:14.668337 14809 tablet_bootstrap.cc:492] T 50df3dd199bc4283bc1836d1917988a8 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.668382 14823 log.cc:1198] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/f21761ff87ac43d5b9b47b5905fa5867
I20250115 03:06:14.668407 14809 ts_tablet_manager.cc:1397] T 50df3dd199bc4283bc1836d1917988a8 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.000s
I20250115 03:06:14.668625 14823 ts_tablet_manager.cc:1939] T f21761ff87ac43d5b9b47b5905fa5867 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.668825 14809 raft_consensus.cc:357] T 50df3dd199bc4283bc1836d1917988a8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.668912 14809 raft_consensus.cc:383] T 50df3dd199bc4283bc1836d1917988a8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.668962 14809 raft_consensus.cc:738] T 50df3dd199bc4283bc1836d1917988a8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.669072 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet b561abc942014fb79f22fd6a9fec38d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.669060 14809 consensus_queue.cc:260] T 50df3dd199bc4283bc1836d1917988a8 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.669298 14809 ts_tablet_manager.cc:1428] T 50df3dd199bc4283bc1836d1917988a8 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:14.669458 14809 tablet_bootstrap.cc:492] T 6f83dd9a2d13420daefdacdb0f565a7d P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.669642 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb809ec664784386aa5f365bc27e0a1a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.670303 14823 tablet_replica.cc:331] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.670347 14809 tablet_bootstrap.cc:654] T 6f83dd9a2d13420daefdacdb0f565a7d P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.670409 14823 raft_consensus.cc:2238] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.670511 14823 raft_consensus.cc:2267] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.671432 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb809ec664784386aa5f365bc27e0a1a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.672164 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "080224910d554389b646e2c250fada2d" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:14.672439 14628 leader_election.cc:343] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:14.672574 14628 leader_election.cc:343] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.672624 14628 leader_election.cc:304] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.672797 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet f21761ff87ac43d5b9b47b5905fa5867 (table default.Table_53c9a73ae8ef459c987964efac552c96 [id=37d965d8a5eb408ea3f33313bd22c152]) successfully deleted
W20250115 03:06:14.673008 14626 leader_election.cc:343] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.673017 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet f21761ff87ac43d5b9b47b5905fa5867 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a0bb9b5093b454d921bf1150d631565) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.673066 14626 leader_election.cc:304] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.673136 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet b561abc942014fb79f22fd6a9fec38d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.673619 14823 ts_tablet_manager.cc:1905] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.673910 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_507ab8820cba47a0bb879411fd28f671" } new_table_name: "default.Table_3e7259eabe7749149fb7ae6c52f729a3" modify_external_catalogs: true
I20250115 03:06:14.674103 15153 raft_consensus.cc:2743] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:14.674222 14820 tablet_replica.cc:331] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.674218 15153 raft_consensus.cc:2743] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:14.674341 14820 raft_consensus.cc:2238] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.674453 14820 raft_consensus.cc:2267] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.674943 14820 ts_tablet_manager.cc:1905] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.675755 14819 ts_tablet_manager.cc:1939] T b561abc942014fb79f22fd6a9fec38d0 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.676462 14858 consensus_queue.cc:1035] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.677024 14820 ts_tablet_manager.cc:1918] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:14.677088 14820 log.cc:1198] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/f21761ff87ac43d5b9b47b5905fa5867
I20250115 03:06:14.677178 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet b561abc942014fb79f22fd6a9fec38d0 (table default.Table_fa83f65f02b04ad491cee3b2a617927f [id=46f096547b3a4de9a46522b978ac4b8c]) successfully deleted
I20250115 03:06:14.677217 14809 tablet_bootstrap.cc:492] T 6f83dd9a2d13420daefdacdb0f565a7d P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.677284 14809 ts_tablet_manager.cc:1397] T 6f83dd9a2d13420daefdacdb0f565a7d P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20250115 03:06:14.677270 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet 9808c7f96a3147848da5f78948895699 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:14.677299 14820 ts_tablet_manager.cc:1939] T f21761ff87ac43d5b9b47b5905fa5867 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.677661 14809 raft_consensus.cc:357] T 6f83dd9a2d13420daefdacdb0f565a7d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.677765 14809 raft_consensus.cc:383] T 6f83dd9a2d13420daefdacdb0f565a7d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.677799 14809 raft_consensus.cc:738] T 6f83dd9a2d13420daefdacdb0f565a7d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.677846 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet f21761ff87ac43d5b9b47b5905fa5867 (table default.Table_53c9a73ae8ef459c987964efac552c96 [id=37d965d8a5eb408ea3f33313bd22c152]) successfully deleted
I20250115 03:06:14.677909 14809 consensus_queue.cc:260] T 6f83dd9a2d13420daefdacdb0f565a7d P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.678175 14809 ts_tablet_manager.cc:1428] T 6f83dd9a2d13420daefdacdb0f565a7d P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:14.678333 14809 tablet_bootstrap.cc:492] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.678048 14820 tablet_replica.cc:331] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.679987 14819 tablet_replica.cc:331] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.680101 14819 raft_consensus.cc:2238] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:14.680135 14809 tablet_bootstrap.cc:654] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.680279 14819 raft_consensus.cc:2267] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.680615 14823 ts_tablet_manager.cc:1918] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:14.680671 14823 log.cc:1198] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/b561abc942014fb79f22fd6a9fec38d0
I20250115 03:06:14.680910 14823 ts_tablet_manager.cc:1939] T b561abc942014fb79f22fd6a9fec38d0 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.679821 14820 raft_consensus.cc:2238] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.681957 14820 raft_consensus.cc:2267] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.682658 14820 ts_tablet_manager.cc:1905] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.684844 14170 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c (table default.Table_4bfe9fa1f7a143d684bc948b47deb8d8 [id=339b61af27cb486aaaadf8730ab21ce7]). Expected version 1 got 0
I20250115 03:06:14.685087 14820 ts_tablet_manager.cc:1918] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:14.685211 14820 log.cc:1198] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/b561abc942014fb79f22fd6a9fec38d0
I20250115 03:06:14.685489 14820 ts_tablet_manager.cc:1939] T b561abc942014fb79f22fd6a9fec38d0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.686425 14809 tablet_bootstrap.cc:492] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.686496 14809 ts_tablet_manager.cc:1397] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20250115 03:06:14.686827 14819 ts_tablet_manager.cc:1905] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.687058 14809 raft_consensus.cc:357] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.687179 14809 raft_consensus.cc:383] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.687212 14809 raft_consensus.cc:738] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.687345 14809 consensus_queue.cc:260] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.687624 14809 ts_tablet_manager.cc:1428] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:14.687849 14809 tablet_bootstrap.cc:492] T 673f9e6cabdd4320b777181c787d15c7 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.686254 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet b561abc942014fb79f22fd6a9fec38d0 (table default.Table_fa83f65f02b04ad491cee3b2a617927f [id=46f096547b3a4de9a46522b978ac4b8c]) successfully deleted
I20250115 03:06:14.689145 14809 tablet_bootstrap.cc:654] T 673f9e6cabdd4320b777181c787d15c7 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.689344 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3e7259eabe7749149fb7ae6c52f729a3" } new_table_name: "default.Table_a75300c14c1f40d68ecd81adc691f38a" modify_external_catalogs: true
I20250115 03:06:14.690281 14809 tablet_bootstrap.cc:492] T 673f9e6cabdd4320b777181c787d15c7 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.690395 14809 ts_tablet_manager.cc:1397] T 673f9e6cabdd4320b777181c787d15c7 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:14.690829 14809 raft_consensus.cc:357] T 673f9e6cabdd4320b777181c787d15c7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.691048 14809 raft_consensus.cc:383] T 673f9e6cabdd4320b777181c787d15c7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.691128 14809 raft_consensus.cc:738] T 673f9e6cabdd4320b777181c787d15c7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.691330 14809 consensus_queue.cc:260] T 673f9e6cabdd4320b777181c787d15c7 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.691645 14809 ts_tablet_manager.cc:1428] T 673f9e6cabdd4320b777181c787d15c7 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.691855 14809 tablet_bootstrap.cc:492] T 080224910d554389b646e2c250fada2d P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.692751 14809 tablet_bootstrap.cc:654] T 080224910d554389b646e2c250fada2d P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.693898 14809 tablet_bootstrap.cc:492] T 080224910d554389b646e2c250fada2d P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.693975 14809 ts_tablet_manager.cc:1397] T 080224910d554389b646e2c250fada2d P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:14.694387 14809 raft_consensus.cc:357] T 080224910d554389b646e2c250fada2d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.694471 14809 raft_consensus.cc:383] T 080224910d554389b646e2c250fada2d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.694500 14809 raft_consensus.cc:738] T 080224910d554389b646e2c250fada2d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.694607 14809 consensus_queue.cc:260] T 080224910d554389b646e2c250fada2d P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.694855 14809 ts_tablet_manager.cc:1428] T 080224910d554389b646e2c250fada2d P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.695060 14809 tablet_bootstrap.cc:492] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.695904 14809 tablet_bootstrap.cc:654] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.681491 14812 tablet_bootstrap.cc:492] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.696499 14812 ts_tablet_manager.cc:1397] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.035s user 0.000s sys 0.002s
I20250115 03:06:14.696983 14812 raft_consensus.cc:357] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.697077 14812 raft_consensus.cc:383] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.697108 14812 raft_consensus.cc:738] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.697206 14812 consensus_queue.cc:260] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.697446 14812 ts_tablet_manager.cc:1428] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.697610 14812 tablet_bootstrap.cc:492] T 37351252b608492e9acbc818b2754731 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.698594 14812 tablet_bootstrap.cc:654] T 37351252b608492e9acbc818b2754731 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.699728 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 68974a1815cb482d8b28d54f42634608
I20250115 03:06:14.700001 14164 catalog_manager.cc:6596] ReplaceTablet: tablet 68974a1815cb482d8b28d54f42634608 deleted and replaced by tablet 420b7c8d43a04bae972eed2014d5634c
I20250115 03:06:14.700151 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 68974a1815cb482d8b28d54f42634608 with delete_type TABLET_DATA_DELETED (replaced by tablet 420b7c8d43a04bae972eed2014d5634c) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.700359 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 68974a1815cb482d8b28d54f42634608 with delete_type TABLET_DATA_DELETED (replaced by tablet 420b7c8d43a04bae972eed2014d5634c) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.700804 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet b561abc942014fb79f22fd6a9fec38d0 (table default.Table_fa83f65f02b04ad491cee3b2a617927f [id=46f096547b3a4de9a46522b978ac4b8c]) successfully deleted
I20250115 03:06:14.700906 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 68974a1815cb482d8b28d54f42634608 with delete_type TABLET_DATA_DELETED (replaced by tablet 420b7c8d43a04bae972eed2014d5634c) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:14.701136 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 68974a1815cb482d8b28d54f42634608 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68974a1815cb482d8b28d54f42634608 already in progress: creating tablet
I20250115 03:06:14.702132 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_85ea53c1d8b245cc94e6e81e157242aa" } modify_external_catalogs: true
I20250115 03:06:14.702226 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_85ea53c1d8b245cc94e6e81e157242aa" } modify_external_catalogs: true
I20250115 03:06:14.694629 14819 ts_tablet_manager.cc:1918] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:14.703544 14819 log.cc:1198] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/f21761ff87ac43d5b9b47b5905fa5867
I20250115 03:06:14.703799 14819 ts_tablet_manager.cc:1939] T f21761ff87ac43d5b9b47b5905fa5867 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
W20250115 03:06:14.704327 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 68974a1815cb482d8b28d54f42634608 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68974a1815cb482d8b28d54f42634608 already in progress: creating tablet
I20250115 03:06:14.704469 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet f21761ff87ac43d5b9b47b5905fa5867 (table default.Table_53c9a73ae8ef459c987964efac552c96 [id=37d965d8a5eb408ea3f33313bd22c152]) successfully deleted
W20250115 03:06:14.704619 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 68974a1815cb482d8b28d54f42634608 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 68974a1815cb482d8b28d54f42634608 already in progress: creating tablet
I20250115 03:06:14.705487 14812 tablet_bootstrap.cc:492] T 37351252b608492e9acbc818b2754731 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.705570 14812 ts_tablet_manager.cc:1397] T 37351252b608492e9acbc818b2754731 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20250115 03:06:14.705966 14812 raft_consensus.cc:357] T 37351252b608492e9acbc818b2754731 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.706055 14812 raft_consensus.cc:383] T 37351252b608492e9acbc818b2754731 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.706084 14812 raft_consensus.cc:738] T 37351252b608492e9acbc818b2754731 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.706151 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 9808c7f96a3147848da5f78948895699
I20250115 03:06:14.706184 14812 consensus_queue.cc:260] T 37351252b608492e9acbc818b2754731 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.706410 14163 catalog_manager.cc:6596] ReplaceTablet: tablet 9808c7f96a3147848da5f78948895699 deleted and replaced by tablet 341b24d9b63f4761b2013035023fcd98
I20250115 03:06:14.706441 14812 ts_tablet_manager.cc:1428] T 37351252b608492e9acbc818b2754731 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.706851 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 9808c7f96a3147848da5f78948895699 with delete_type TABLET_DATA_DELETED (replaced by tablet 341b24d9b63f4761b2013035023fcd98) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.706897 14809 tablet_bootstrap.cc:492] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.706967 14809 ts_tablet_manager.cc:1397] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
W20250115 03:06:14.707235 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 9808c7f96a3147848da5f78948895699 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9808c7f96a3147848da5f78948895699 already in progress: creating tablet
I20250115 03:06:14.707365 14809 raft_consensus.cc:357] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.707456 14809 raft_consensus.cc:383] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.707489 14809 raft_consensus.cc:738] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.707675 14809 consensus_queue.cc:260] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.707705 15274 tablet.cc:1721] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.707818 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 9808c7f96a3147848da5f78948895699 with delete_type TABLET_DATA_DELETED (replaced by tablet 341b24d9b63f4761b2013035023fcd98) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.708078 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 9808c7f96a3147848da5f78948895699 with delete_type TABLET_DATA_DELETED (replaced by tablet 341b24d9b63f4761b2013035023fcd98) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.708963 15426 tablet.cc:1721] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.709374 15421 tablet.cc:1721] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.711140 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet acb43d614111477abb10def48a1dfb3b
I20250115 03:06:14.712186 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet acb43d614111477abb10def48a1dfb3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.712314 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet acb43d614111477abb10def48a1dfb3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.712802 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet acb43d614111477abb10def48a1dfb3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.712881 14809 ts_tablet_manager.cc:1428] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:14.713070 14809 tablet_bootstrap.cc:492] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
W20250115 03:06:14.713737 14156 catalog_manager.cc:4839] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet acb43d614111477abb10def48a1dfb3b because the tablet was not found. No further retry: Not found: Tablet not found: acb43d614111477abb10def48a1dfb3b
I20250115 03:06:14.713930 14809 tablet_bootstrap.cc:654] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.713994 14812 tablet_bootstrap.cc:492] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.714818 14812 tablet_bootstrap.cc:654] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:14.715601 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 9808c7f96a3147848da5f78948895699 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9808c7f96a3147848da5f78948895699 already in progress: creating tablet
W20250115 03:06:14.715760 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet acb43d614111477abb10def48a1dfb3b because the tablet was not found. No further retry: Not found: Tablet not found: acb43d614111477abb10def48a1dfb3b
I20250115 03:06:14.717156 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e050a17d73d342fda8108a99a665f222" } modify_external_catalogs: true
I20250115 03:06:14.717253 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e050a17d73d342fda8108a99a665f222" } modify_external_catalogs: true
I20250115 03:06:14.717223 14853 consensus_queue.cc:1035] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.717861 14853 raft_consensus.cc:491] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.717941 14853 raft_consensus.cc:513] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.718240 14853 leader_election.cc:290] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:14.718544 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0580a8f7b2984162b9275d74334b1905" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:14.718648 14689 raft_consensus.cc:2463] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:14.718899 14493 leader_election.cc:304] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:14.718865 14166 catalog_manager.cc:5526] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:14.719077 14853 raft_consensus.cc:2798] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.719123 14853 raft_consensus.cc:491] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.719156 14853 raft_consensus.cc:3054] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.719945 14853 raft_consensus.cc:513] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.720238 14853 leader_election.cc:290] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:14.720600 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0580a8f7b2984162b9275d74334b1905" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:14.720702 14689 raft_consensus.cc:3054] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.720842 14330 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet acb43d614111477abb10def48a1dfb3b
I20250115 03:06:14.721591 14689 raft_consensus.cc:2463] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:14.721768 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet acb43d614111477abb10def48a1dfb3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.722018 14493 leader_election.cc:304] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
W20250115 03:06:14.722131 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet acb43d614111477abb10def48a1dfb3b because the tablet was not found. No further retry: Not found: Tablet not found: acb43d614111477abb10def48a1dfb3b
I20250115 03:06:14.723163 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 506198162e0b4e24924c408542707c4c (DEFAULT_TABLE table=default.Table_75c071e03ba44140bce71e920f31715d [id=13ea98de708e409ba7b66e29bb49416d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.723330 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet acb43d614111477abb10def48a1dfb3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.723397 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 506198162e0b4e24924c408542707c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.723489 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet acb43d614111477abb10def48a1dfb3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:14.724171 14156 catalog_manager.cc:4839] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet acb43d614111477abb10def48a1dfb3b because the tablet was not found. No further retry: Not found: Tablet not found: acb43d614111477abb10def48a1dfb3b
I20250115 03:06:14.724464 15274 tablet.cc:1721] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:14.724541 14817 raft_consensus.cc:2798] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:14.724615 14817 raft_consensus.cc:695] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:14.724731 14817 consensus_queue.cc:237] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.726047 15421 tablet.cc:1721] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:14.726416 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0580a8f7b2984162b9275d74334b1905" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:14.726526 14421 raft_consensus.cc:3054] T 0580a8f7b2984162b9275d74334b1905 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.727317 14421 raft_consensus.cc:2463] T 0580a8f7b2984162b9275d74334b1905 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:14.727476 14858 consensus_queue.cc:1035] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.727556 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0580a8f7b2984162b9275d74334b1905" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.727641 14422 raft_consensus.cc:2371] T 0580a8f7b2984162b9275d74334b1905 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1. Re-sending same reply.
I20250115 03:06:14.727487 15426 tablet.cc:1721] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:14.728318 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 506198162e0b4e24924c408542707c4c (DEFAULT_TABLE table=default.Table_75c071e03ba44140bce71e920f31715d [id=13ea98de708e409ba7b66e29bb49416d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.728533 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 506198162e0b4e24924c408542707c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.729326 15453 raft_consensus.cc:491] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.729398 15453 raft_consensus.cc:513] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.729743 15453 leader_election.cc:290] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
W20250115 03:06:14.729872 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 9808c7f96a3147848da5f78948895699 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9808c7f96a3147848da5f78948895699 already in progress: creating tablet
W20250115 03:06:14.729985 14158 catalog_manager.cc:4839] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet acb43d614111477abb10def48a1dfb3b because the tablet was not found. No further retry: Not found: Tablet not found: acb43d614111477abb10def48a1dfb3b
W20250115 03:06:14.730100 14158 catalog_manager.cc:4839] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet acb43d614111477abb10def48a1dfb3b because the tablet was not found. No further retry: Not found: Tablet not found: acb43d614111477abb10def48a1dfb3b
I20250115 03:06:14.730445 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01aa12e3d8fa463cbaee1287e7b74dd4" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:14.730741 14628 leader_election.cc:343] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.730932 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6e69d6da0dc24bfeb36a8d4a43db72e2" } new_table_name: "default.Table_40b1953a4a364eb1ae58e3387fc65a91" modify_external_catalogs: true
I20250115 03:06:14.731657 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet db1aa206e9994de6b9a355a621cb2ab3
I20250115 03:06:14.731701 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01aa12e3d8fa463cbaee1287e7b74dd4" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.733364 15453 raft_consensus.cc:491] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.733451 15453 raft_consensus.cc:513] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.733793 15453 leader_election.cc:290] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.734006 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a56ae8a3e3e4f57b13e762cef21f753" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.734084 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet db1aa206e9994de6b9a355a621cb2ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:14.734273 14628 leader_election.cc:343] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.734710 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 745344db22da4102bad39c2f121548a1 (DEFAULT_TABLE table=default.Table_faa8557664e04a1c88f455c0f098a9d8 [id=f22f08e9af4a4496b100e7d4b5982121]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.734779 14400 tablet_service.cc:1467] Processing CreateTablet for tablet acb43d614111477abb10def48a1dfb3b (DEFAULT_TABLE table=default.Table_85ea53c1d8b245cc94e6e81e157242aa [id=4d8e724e03dc4e2ab9a6e0281159aca8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.734910 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 745344db22da4102bad39c2f121548a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.735219 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acb43d614111477abb10def48a1dfb3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.736500 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 506198162e0b4e24924c408542707c4c (DEFAULT_TABLE table=default.Table_75c071e03ba44140bce71e920f31715d [id=13ea98de708e409ba7b66e29bb49416d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.736721 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 506198162e0b4e24924c408542707c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.736845 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 48aa039bf0364c02abb7f9677eae49e5 (DEFAULT_TABLE table=default.Table_35005559d2f34672bc3e70f6bbfb31d6 [id=d73a4eb552114e20887f9599f244e6ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.737057 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48aa039bf0364c02abb7f9677eae49e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.737272 15390 raft_consensus.cc:491] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.737341 15390 raft_consensus.cc:513] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.737649 15390 leader_election.cc:290] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.737998 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d25aa6a292b454f8f451d6493d940c3" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:14.738095 14689 raft_consensus.cc:2463] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:14.738521 14358 leader_election.cc:304] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:14.738845 15390 raft_consensus.cc:2798] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.738901 15390 raft_consensus.cc:491] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.738937 15390 raft_consensus.cc:3054] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.739549 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_50c659a42a154d46a26879df0101bf23 [id=f8f2a7ff6fdc40468de6ae9ba28c1427] alter complete (version 3)
I20250115 03:06:14.739740 15390 raft_consensus.cc:513] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.739811 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a56ae8a3e3e4f57b13e762cef21f753" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.739917 14530 tablet_service.cc:1467] Processing CreateTablet for tablet acb43d614111477abb10def48a1dfb3b (DEFAULT_TABLE table=default.Table_85ea53c1d8b245cc94e6e81e157242aa [id=4d8e724e03dc4e2ab9a6e0281159aca8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.740027 15390 leader_election.cc:290] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.740129 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acb43d614111477abb10def48a1dfb3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.740245 14660 tablet_service.cc:1467] Processing CreateTablet for tablet acb43d614111477abb10def48a1dfb3b (DEFAULT_TABLE table=default.Table_85ea53c1d8b245cc94e6e81e157242aa [id=4d8e724e03dc4e2ab9a6e0281159aca8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.740463 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet acb43d614111477abb10def48a1dfb3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.741211 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 745344db22da4102bad39c2f121548a1 (DEFAULT_TABLE table=default.Table_faa8557664e04a1c88f455c0f098a9d8 [id=f22f08e9af4a4496b100e7d4b5982121]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.741420 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 745344db22da4102bad39c2f121548a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.742029 14529 tablet_service.cc:1467] Processing CreateTablet for tablet 745344db22da4102bad39c2f121548a1 (DEFAULT_TABLE table=default.Table_faa8557664e04a1c88f455c0f098a9d8 [id=f22f08e9af4a4496b100e7d4b5982121]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.742225 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 745344db22da4102bad39c2f121548a1. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:14.743148 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet db1aa206e9994de6b9a355a621cb2ab3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet db1aa206e9994de6b9a355a621cb2ab3 already in progress: creating tablet
I20250115 03:06:14.743330 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a56ae8a3e3e4f57b13e762cef21f753 with delete_type TABLET_DATA_DELETED (replaced by tablet 5dd115f83c544c4bba0d08fd82220952) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.743506 14823 tablet_replica.cc:331] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.743638 14823 raft_consensus.cc:2238] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.743741 14823 raft_consensus.cc:2267] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.744253 14823 ts_tablet_manager.cc:1905] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.744546 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d25aa6a292b454f8f451d6493d940c3" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:14.744647 14691 raft_consensus.cc:3054] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.744742 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dffa87697284dca8f07719f1c861bd4 with delete_type TABLET_DATA_DELETED (replaced by tablet a1cbe29c08c7477e8944b7e108d46e7a) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.745452 14691 raft_consensus.cc:2463] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:14.745672 14358 leader_election.cc:304] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
W20250115 03:06:14.744546 14163 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a1c93bfd3da44f9f8a8d585f2d21f1be consensus state has no leader
I20250115 03:06:14.745839 15390 raft_consensus.cc:2798] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:14.745906 15390 raft_consensus.cc:695] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:14.746012 15390 consensus_queue.cc:237] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.746754 14823 ts_tablet_manager.cc:1918] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.746816 14823 log.cc:1198] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/4a56ae8a3e3e4f57b13e762cef21f753
I20250115 03:06:14.747014 14823 ts_tablet_manager.cc:1939] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.747505 14823 tablet_replica.cc:331] T 2dffa87697284dca8f07719f1c861bd4 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.747599 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 4a56ae8a3e3e4f57b13e762cef21f753 (table default.Table_13681f62285b46bc88883e9c804c888e [id=1b7d3a80c5424e5692c2e6c556919342]) successfully deleted
I20250115 03:06:14.748174 14823 raft_consensus.cc:2238] T 2dffa87697284dca8f07719f1c861bd4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.748279 14823 raft_consensus.cc:2267] T 2dffa87697284dca8f07719f1c861bd4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.748634 14823 ts_tablet_manager.cc:1905] T 2dffa87697284dca8f07719f1c861bd4 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.749579 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 48aa039bf0364c02abb7f9677eae49e5 (DEFAULT_TABLE table=default.Table_35005559d2f34672bc3e70f6bbfb31d6 [id=d73a4eb552114e20887f9599f244e6ed]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:14.749641 14626 leader_election.cc:343] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.749711 14626 leader_election.cc:304] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.749776 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48aa039bf0364c02abb7f9677eae49e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.750617 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet 5ad5197958ce45de8b9d78174aaae689 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:14.751318 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f960fb76bf0d4e3288988b6a3abb2e8c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:14.751473 15325 raft_consensus.cc:2743] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:14.751770 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_a0c249d69b904390952a780b10e1369b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.752074 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a0c249d69b904390952a780b10e1369b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:14.752715 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f960fb76bf0d4e3288988b6a3abb2e8c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.753666 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_eea85759fdf64d0bbfabd1e6cb16b125"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.753926 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eea85759fdf64d0bbfabd1e6cb16b125 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:14.754587 14626 leader_election.cc:343] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.754674 14626 leader_election.cc:304] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
W20250115 03:06:14.754755 14626 raft_consensus.cc:2681] T 4a56ae8a3e3e4f57b13e762cef21f753 P 5c0dae7aff4b44c699275b95ba639770: unable to run election callback: Service unavailable: Thread pool token was shut down
I20250115 03:06:14.757159 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 5ad5197958ce45de8b9d78174aaae689
I20250115 03:06:14.757319 14809 tablet_bootstrap.cc:492] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.757395 14809 ts_tablet_manager.cc:1397] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.044s user 0.001s sys 0.000s
I20250115 03:06:14.757411 14163 catalog_manager.cc:6596] ReplaceTablet: tablet 5ad5197958ce45de8b9d78174aaae689 deleted and replaced by tablet f8b90f854a5e4221a24986f8c71ca300
I20250115 03:06:14.758227 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ad5197958ce45de8b9d78174aaae689 with delete_type TABLET_DATA_DELETED (replaced by tablet f8b90f854a5e4221a24986f8c71ca300) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.758215 14809 raft_consensus.cc:357] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.758303 14809 raft_consensus.cc:383] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.758332 14809 raft_consensus.cc:738] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.758471 14809 consensus_queue.cc:260] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.758690 14820 tablet_replica.cc:331] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.758845 14820 raft_consensus.cc:2238] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.758968 14820 raft_consensus.cc:2267] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.759383 14809 ts_tablet_manager.cc:1428] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:14.759559 14809 tablet_bootstrap.cc:492] T 4a56ae8a3e3e4f57b13e762cef21f753 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.760147 14820 ts_tablet_manager.cc:1905] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.760259 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 48aa039bf0364c02abb7f9677eae49e5 (DEFAULT_TABLE table=default.Table_35005559d2f34672bc3e70f6bbfb31d6 [id=d73a4eb552114e20887f9599f244e6ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.760471 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48aa039bf0364c02abb7f9677eae49e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.760601 14809 tablet_bootstrap.cc:654] T 4a56ae8a3e3e4f57b13e762cef21f753 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.761488 14823 ts_tablet_manager.cc:1918] T 2dffa87697284dca8f07719f1c861bd4 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.761605 14823 log.cc:1198] T 2dffa87697284dca8f07719f1c861bd4 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/2dffa87697284dca8f07719f1c861bd4
I20250115 03:06:14.761819 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ad5197958ce45de8b9d78174aaae689 with delete_type TABLET_DATA_DELETED (replaced by tablet f8b90f854a5e4221a24986f8c71ca300) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.761868 14823 ts_tablet_manager.cc:1939] T 2dffa87697284dca8f07719f1c861bd4 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.762259 14820 ts_tablet_manager.cc:1918] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:14.762328 14820 log.cc:1198] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/5ad5197958ce45de8b9d78174aaae689
I20250115 03:06:14.762346 14529 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f83dd9a2d13420daefdacdb0f565a7d with delete_type TABLET_DATA_DELETED (replaced by tablet 3cbc91ad00b54ede804c3629bcfe7fe6) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.762512 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 2dffa87697284dca8f07719f1c861bd4 (table default.Table_7e53fa73287c43e3a9a0e65908b42593 [id=1f983a10a8704e22b8a2b96ef4e06fbc]) successfully deleted
I20250115 03:06:14.762588 14820 ts_tablet_manager.cc:1939] T 5ad5197958ce45de8b9d78174aaae689 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.762714 14823 tablet_replica.cc:331] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.762688 14162 catalog_manager.cc:5526] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:14.762856 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 0580a8f7b2984162b9275d74334b1905 (table default.Table_3e704f1e03e34a2ca2377db61214c5a0 [id=0790b08615854d3688f940aba20b6520]). Expected version 1 got 0
I20250115 03:06:14.762914 14823 raft_consensus.cc:2238] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.763017 14823 raft_consensus.cc:2267] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.762972 14162 catalog_manager.cc:5526] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:14.763335 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ad5197958ce45de8b9d78174aaae689 with delete_type TABLET_DATA_DELETED (replaced by tablet f8b90f854a5e4221a24986f8c71ca300) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.763514 14819 tablet_replica.cc:331] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.763638 14819 raft_consensus.cc:2238] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:14.763808 14819 raft_consensus.cc:2267] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.763957 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet db1aa206e9994de6b9a355a621cb2ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.765128 14819 ts_tablet_manager.cc:1905] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.763088 14820 tablet_replica.cc:331] T 6f83dd9a2d13420daefdacdb0f565a7d P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.765484 14820 raft_consensus.cc:2238] T 6f83dd9a2d13420daefdacdb0f565a7d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.765585 14820 raft_consensus.cc:2267] T 6f83dd9a2d13420daefdacdb0f565a7d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.763731 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d25aa6a292b454f8f451d6493d940c3" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.765733 14557 raft_consensus.cc:2463] T 0d25aa6a292b454f8f451d6493d940c3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:14.766199 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 5ad5197958ce45de8b9d78174aaae689 (table default.Table_ec8ee0d483474271a3b369a2bcde593d [id=d8084de64a264d28a3370971617a694c]) successfully deleted
I20250115 03:06:14.767220 14819 ts_tablet_manager.cc:1918] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:14.767285 14819 log.cc:1198] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/5ad5197958ce45de8b9d78174aaae689
I20250115 03:06:14.767513 14819 ts_tablet_manager.cc:1939] T 5ad5197958ce45de8b9d78174aaae689 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.767619 14820 ts_tablet_manager.cc:1905] T 6f83dd9a2d13420daefdacdb0f565a7d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.768349 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_a75300c14c1f40d68ecd81adc691f38a [id=38bfb0375c334d16b97c7251d6ddd76b] alter complete (version 2)
I20250115 03:06:14.763895 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet db1aa206e9994de6b9a355a621cb2ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.764766 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d25aa6a292b454f8f451d6493d940c3" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:14.773738 14556 raft_consensus.cc:3054] T 0d25aa6a292b454f8f451d6493d940c3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.773967 14861 raft_consensus.cc:491] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.774029 14861 raft_consensus.cc:513] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.774325 14861 leader_election.cc:290] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.774534 14556 raft_consensus.cc:2463] T 0d25aa6a292b454f8f451d6493d940c3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:14.774720 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c76ab180b26487896f4360efe47b2c7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:14.774830 14691 raft_consensus.cc:2463] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:14.775095 14358 leader_election.cc:304] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:14.775172 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c76ab180b26487896f4360efe47b2c7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.775241 14861 raft_consensus.cc:2798] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.775278 14861 raft_consensus.cc:491] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.775311 14861 raft_consensus.cc:3054] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:14.775461 14356 leader_election.cc:343] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.775964 14861 raft_consensus.cc:513] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.776070 14820 ts_tablet_manager.cc:1918] T 6f83dd9a2d13420daefdacdb0f565a7d P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.776152 14820 log.cc:1198] T 6f83dd9a2d13420daefdacdb0f565a7d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/6f83dd9a2d13420daefdacdb0f565a7d
I20250115 03:06:14.776257 14861 leader_election.cc:290] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.776351 14820 ts_tablet_manager.cc:1939] T 6f83dd9a2d13420daefdacdb0f565a7d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.776587 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c76ab180b26487896f4360efe47b2c7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:14.776823 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c76ab180b26487896f4360efe47b2c7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:14.776916 14691 raft_consensus.cc:3054] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.777405 14812 tablet_bootstrap.cc:492] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.777491 14812 ts_tablet_manager.cc:1397] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.064s user 0.001s sys 0.000s
I20250115 03:06:14.777782 14691 raft_consensus.cc:2463] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:14.777866 14823 ts_tablet_manager.cc:1905] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.777966 14812 raft_consensus.cc:357] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.778026 14358 leader_election.cc:304] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:14.778054 14812 raft_consensus.cc:383] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.778090 14812 raft_consensus.cc:738] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
W20250115 03:06:14.778131 14356 leader_election.cc:343] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.778183 14861 raft_consensus.cc:2798] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:14.778205 14812 consensus_queue.cc:260] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.778244 14861 raft_consensus.cc:695] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:14.778352 14861 consensus_queue.cc:237] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.778476 14812 ts_tablet_manager.cc:1428] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.778651 14812 tablet_bootstrap.cc:492] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.779143 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 6f83dd9a2d13420daefdacdb0f565a7d (table default.Table_271f46df55f14ffd9b7bb48352620543 [id=44488be7335b488a9ff7cf913ce4273a]) successfully deleted
I20250115 03:06:14.779603 14812 tablet_bootstrap.cc:654] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.779636 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 5ad5197958ce45de8b9d78174aaae689 (table default.Table_ec8ee0d483474271a3b369a2bcde593d [id=d8084de64a264d28a3370971617a694c]) successfully deleted
W20250115 03:06:14.781366 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet db1aa206e9994de6b9a355a621cb2ab3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet db1aa206e9994de6b9a355a621cb2ab3 already in progress: creating tablet
I20250115 03:06:14.782253 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_929dc98c434d495da9e1e46de64f5ef9" } modify_external_catalogs: true
I20250115 03:06:14.782341 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_929dc98c434d495da9e1e46de64f5ef9" } modify_external_catalogs: true
I20250115 03:06:14.782392 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 37351252b608492e9acbc818b2754731 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.782954 14811 tablet_bootstrap.cc:492] T 55dcc7ac82ac44ab9d85055720760d3d P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.783035 14811 ts_tablet_manager.cc:1397] T 55dcc7ac82ac44ab9d85055720760d3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.132s user 0.000s sys 0.001s
I20250115 03:06:14.783409 14811 raft_consensus.cc:357] T 55dcc7ac82ac44ab9d85055720760d3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.783502 14811 raft_consensus.cc:383] T 55dcc7ac82ac44ab9d85055720760d3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.783535 14811 raft_consensus.cc:738] T 55dcc7ac82ac44ab9d85055720760d3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.783630 14811 consensus_queue.cc:260] T 55dcc7ac82ac44ab9d85055720760d3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.783862 14811 ts_tablet_manager.cc:1428] T 55dcc7ac82ac44ab9d85055720760d3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.784000 14811 tablet_bootstrap.cc:492] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.784819 14811 tablet_bootstrap.cc:654] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.785346 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ad312d1f941c45b5a0dc5324967d5105"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.785634 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ad312d1f941c45b5a0dc5324967d5105 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.786976 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_62fa8b23f4c44bb0938d6363bc2086ef"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:14.787133 14809 tablet_bootstrap.cc:492] T 4a56ae8a3e3e4f57b13e762cef21f753 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.787200 14809 ts_tablet_manager.cc:1397] T 4a56ae8a3e3e4f57b13e762cef21f753 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.028s user 0.001s sys 0.001s
W20250115 03:06:14.787254 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_62fa8b23f4c44bb0938d6363bc2086ef in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.787611 14809 raft_consensus.cc:357] T 4a56ae8a3e3e4f57b13e762cef21f753 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.787698 14809 raft_consensus.cc:383] T 4a56ae8a3e3e4f57b13e762cef21f753 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.787730 14809 raft_consensus.cc:738] T 4a56ae8a3e3e4f57b13e762cef21f753 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.787859 14809 consensus_queue.cc:260] T 4a56ae8a3e3e4f57b13e762cef21f753 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.788621 14853 raft_consensus.cc:491] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.788700 14853 raft_consensus.cc:513] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.789032 14853 leader_election.cc:290] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.789050 14809 ts_tablet_manager.cc:1428] T 4a56ae8a3e3e4f57b13e762cef21f753 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:14.789201 14809 tablet_bootstrap.cc:492] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
W20250115 03:06:14.789755 14493 leader_election.cc:343] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 50df3dd199bc4283bc1836d1917988a8
I20250115 03:06:14.790066 14809 tablet_bootstrap.cc:654] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.790367 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50df3dd199bc4283bc1836d1917988a8" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.790467 14421 raft_consensus.cc:2463] T 50df3dd199bc4283bc1836d1917988a8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:14.791597 14493 leader_election.cc:304] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770
I20250115 03:06:14.791908 14853 raft_consensus.cc:2798] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.791956 14853 raft_consensus.cc:491] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.791990 14853 raft_consensus.cc:3054] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.792078 14164 catalog_manager.cc:5526] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:14.792852 14853 raft_consensus.cc:513] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.792950 14823 ts_tablet_manager.cc:1918] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:14.793009 14823 log.cc:1198] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/5ad5197958ce45de8b9d78174aaae689
I20250115 03:06:14.793197 14853 leader_election.cc:290] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.793361 14823 ts_tablet_manager.cc:1939] T 5ad5197958ce45de8b9d78174aaae689 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
W20250115 03:06:14.793877 14493 leader_election.cc:343] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 50df3dd199bc4283bc1836d1917988a8
I20250115 03:06:14.793913 14170 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 5dd115f83c544c4bba0d08fd82220952 (table default.Table_13681f62285b46bc88883e9c804c888e [id=1b7d3a80c5424e5692c2e6c556919342]). Expected version 2 got 0
I20250115 03:06:14.793970 14823 tablet_replica.cc:331] T 37351252b608492e9acbc818b2754731 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.794078 14823 raft_consensus.cc:2238] T 37351252b608492e9acbc818b2754731 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.794114 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 5ad5197958ce45de8b9d78174aaae689 (table default.Table_ec8ee0d483474271a3b369a2bcde593d [id=d8084de64a264d28a3370971617a694c]) successfully deleted
I20250115 03:06:14.794168 14823 raft_consensus.cc:2267] T 37351252b608492e9acbc818b2754731 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:14.794257 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet db1aa206e9994de6b9a355a621cb2ab3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet db1aa206e9994de6b9a355a621cb2ab3 already in progress: creating tablet
I20250115 03:06:14.794631 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50df3dd199bc4283bc1836d1917988a8" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:14.794741 14421 raft_consensus.cc:3054] T 50df3dd199bc4283bc1836d1917988a8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.794777 14823 ts_tablet_manager.cc:1905] T 37351252b608492e9acbc818b2754731 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.795554 14421 raft_consensus.cc:2463] T 50df3dd199bc4283bc1836d1917988a8 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:14.795821 14493 leader_election.cc:304] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770
I20250115 03:06:14.796015 14858 raft_consensus.cc:2798] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:14.796080 14858 raft_consensus.cc:695] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:14.796186 14858 consensus_queue.cc:237] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.796721 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 50df3dd199bc4283bc1836d1917988a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.796880 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6bb3a06c1ae749e593cdc3f2b3992eaf" } modify_external_catalogs: true
I20250115 03:06:14.796990 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6bb3a06c1ae749e593cdc3f2b3992eaf" } modify_external_catalogs: true
I20250115 03:06:14.801937 14820 tablet_replica.cc:331] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:14.802086 14820 raft_consensus.cc:2238] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:14.802246 14820 pending_rounds.cc:62] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Trying to abort 1 pending ops.
I20250115 03:06:14.802325 14820 pending_rounds.cc:69] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384895166705664 op_type: NO_OP noop_request { }
I20250115 03:06:14.802417 14820 raft_consensus.cc:2883] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:14.802517 14820 raft_consensus.cc:2267] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.805178 14166 master_service.cc:946] ReplaceTablet: received request to replace tablet 37351252b608492e9acbc818b2754731 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:14.806015 14811 tablet_bootstrap.cc:492] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.806097 14811 ts_tablet_manager.cc:1397] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.002s
I20250115 03:06:14.806530 14811 raft_consensus.cc:357] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.806622 14811 raft_consensus.cc:383] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.806651 14811 raft_consensus.cc:738] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.806751 14811 consensus_queue.cc:260] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.807010 14811 ts_tablet_manager.cc:1428] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:14.807166 14811 tablet_bootstrap.cc:492] T f54942abdfb748f2a939993a3c2d5db0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.807969 14812 tablet_bootstrap.cc:492] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.808037 14811 tablet_bootstrap.cc:654] T f54942abdfb748f2a939993a3c2d5db0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.808043 14812 ts_tablet_manager.cc:1397] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.000s
I20250115 03:06:14.808420 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_fe0a1f3ae7a04225a5a482c33a15c579"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:14.808499 14812 raft_consensus.cc:357] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.808584 14812 raft_consensus.cc:383] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.808612 14812 raft_consensus.cc:738] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
W20250115 03:06:14.808728 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe0a1f3ae7a04225a5a482c33a15c579 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.808712 14812 consensus_queue.cc:260] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.808984 14812 ts_tablet_manager.cc:1428] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.809155 14812 tablet_bootstrap.cc:492] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.810140 14812 tablet_bootstrap.cc:654] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.810714 14809 tablet_bootstrap.cc:492] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.810794 14809 ts_tablet_manager.cc:1397] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20250115 03:06:14.810886 14059 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu with pid 14210
I20250115 03:06:14.811012 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 080224910d554389b646e2c250fada2d
I20250115 03:06:14.813777 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 080224910d554389b646e2c250fada2d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.814383 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 080224910d554389b646e2c250fada2d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.814513 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f83dd9a2d13420daefdacdb0f565a7d with delete_type TABLET_DATA_DELETED (replaced by tablet 3cbc91ad00b54ede804c3629bcfe7fe6) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.814531 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 50df3dd199bc4283bc1836d1917988a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.814617 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 673f9e6cabdd4320b777181c787d15c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.814723 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet f54942abdfb748f2a939993a3c2d5db0 with delete_type TABLET_DATA_DELETED (replaced by tablet 382116b1fb6a4f888ab1d55c29917ec8) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.814759 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 080224910d554389b646e2c250fada2d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.814298 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 99d9180ed9b24363a4c44d01ea1d6e9b
I20250115 03:06:14.815794 14819 tablet_replica.cc:331] T 080224910d554389b646e2c250fada2d P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.815910 14819 raft_consensus.cc:2238] T 080224910d554389b646e2c250fada2d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.815996 14819 raft_consensus.cc:2267] T 080224910d554389b646e2c250fada2d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.816172 14809 raft_consensus.cc:357] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.816313 14809 raft_consensus.cc:383] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.816375 14809 raft_consensus.cc:738] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.816426 14819 ts_tablet_manager.cc:1905] T 080224910d554389b646e2c250fada2d P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.816551 14809 consensus_queue.cc:260] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.818049 14809 ts_tablet_manager.cc:1428] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:14.818980 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 99d9180ed9b24363a4c44d01ea1d6e9b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.818504 14819 ts_tablet_manager.cc:1918] T 080224910d554389b646e2c250fada2d P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.819087 14819 log.cc:1198] T 080224910d554389b646e2c250fada2d P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/080224910d554389b646e2c250fada2d
I20250115 03:06:14.819288 14819 ts_tablet_manager.cc:1939] T 080224910d554389b646e2c250fada2d P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.819617 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 99d9180ed9b24363a4c44d01ea1d6e9b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.819743 14819 tablet_replica.cc:331] T 6f83dd9a2d13420daefdacdb0f565a7d P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.819804 14809 tablet_bootstrap.cc:492] T 2dffa87697284dca8f07719f1c861bd4 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.819845 14819 raft_consensus.cc:2238] T 6f83dd9a2d13420daefdacdb0f565a7d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.819924 14819 raft_consensus.cc:2267] T 6f83dd9a2d13420daefdacdb0f565a7d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.818578 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 99d9180ed9b24363a4c44d01ea1d6e9b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.820223 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 080224910d554389b646e2c250fada2d (table default.Table_929dc98c434d495da9e1e46de64f5ef9 [id=ade8ca55065d405b994a155664fa3eb1]) successfully deleted
I20250115 03:06:14.820254 15390 raft_consensus.cc:491] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.820312 15390 raft_consensus.cc:513] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.820557 14819 ts_tablet_manager.cc:1905] T 6f83dd9a2d13420daefdacdb0f565a7d P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.820636 15390 leader_election.cc:290] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.820698 14809 tablet_bootstrap.cc:654] T 2dffa87697284dca8f07719f1c861bd4 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.820986 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01aa12e3d8fa463cbaee1287e7b74dd4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:14.821103 14689 raft_consensus.cc:2463] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:14.821355 14358 leader_election.cc:304] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:14.821733 15390 raft_consensus.cc:2798] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.821781 15390 raft_consensus.cc:491] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.821815 15390 raft_consensus.cc:3054] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.821830 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01aa12e3d8fa463cbaee1287e7b74dd4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.822252 14819 ts_tablet_manager.cc:1918] T 6f83dd9a2d13420daefdacdb0f565a7d P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.822304 14819 log.cc:1198] T 6f83dd9a2d13420daefdacdb0f565a7d P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/6f83dd9a2d13420daefdacdb0f565a7d
I20250115 03:06:14.822474 14819 ts_tablet_manager.cc:1939] T 6f83dd9a2d13420daefdacdb0f565a7d P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.822636 15390 raft_consensus.cc:513] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.822952 15390 leader_election.cc:290] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.823122 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 6f83dd9a2d13420daefdacdb0f565a7d (table default.Table_271f46df55f14ffd9b7bb48352620543 [id=44488be7335b488a9ff7cf913ce4273a]) successfully deleted
I20250115 03:06:14.823370 14819 tablet_replica.cc:331] T 50df3dd199bc4283bc1836d1917988a8 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.823467 14819 raft_consensus.cc:2238] T 50df3dd199bc4283bc1836d1917988a8 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
W20250115 03:06:14.823544 14356 leader_election.cc:343] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.823627 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01aa12e3d8fa463cbaee1287e7b74dd4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:14.823545 14819 raft_consensus.cc:2267] T 50df3dd199bc4283bc1836d1917988a8 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.823665 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01aa12e3d8fa463cbaee1287e7b74dd4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:14.823709 14689 raft_consensus.cc:3054] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:14.823951 14356 leader_election.cc:343] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.824553 14689 raft_consensus.cc:2463] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:14.825088 15453 raft_consensus.cc:491] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.825165 15453 raft_consensus.cc:513] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.825495 15453 leader_election.cc:290] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.825841 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d3157859e684da98bb473d5799b24a6" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:14.826153 14626 leader_election.cc:343] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.826689 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f590b49e96644f96934046c984688857" } new_table_name: "default.Table_47ede7f24148438ba5b165a821f676bc" modify_external_catalogs: true
I20250115 03:06:14.827118 14823 ts_tablet_manager.cc:1918] T 37351252b608492e9acbc818b2754731 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.828192 14823 log.cc:1198] T 37351252b608492e9acbc818b2754731 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/37351252b608492e9acbc818b2754731
I20250115 03:06:14.828399 14823 ts_tablet_manager.cc:1939] T 37351252b608492e9acbc818b2754731 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.828976 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 37351252b608492e9acbc818b2754731 (table default.Table_e557ad0830f44aec955ce14c80b107e1 [id=0b8e801e3c8f40a6a5e226fbd824379e]) successfully deleted
I20250115 03:06:14.829154 14823 tablet_replica.cc:331] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:14.830104 14823 raft_consensus.cc:2238] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.830205 14823 raft_consensus.cc:2267] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.830193 14170 catalog_manager.cc:5526] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:14.832599 14823 ts_tablet_manager.cc:1905] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.833214 14165 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 481705dc9ae841659520a6c909879f9b (table default.Table_cab1959a159a46a6b86daad3647b89b3 [id=fe32f8f8243b4803876ed2b371e9e602]). Expected version 1 got 0
I20250115 03:06:14.835284 14165 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 481705dc9ae841659520a6c909879f9b (table default.Table_cab1959a159a46a6b86daad3647b89b3 [id=fe32f8f8243b4803876ed2b371e9e602]). Expected version 1 got 0
W20250115 03:06:14.835498 14626 connection.cc:537] client connection to 127.13.186.252:41855 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20250115 03:06:14.835975 14156 connection.cc:537] client connection to 127.13.186.252:41855 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20250115 03:06:14.837328 14672 tablet_service.cc:1467] Processing CreateTablet for tablet eeff2cd0825c4419aacfc73ebe267a8e (DEFAULT_TABLE table=default.Table_8ecc7de5c6b3497fb2d931c10ec6b373 [id=021f5685630741a8880d2471f7d2d1d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.837579 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeff2cd0825c4419aacfc73ebe267a8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.837815 14823 ts_tablet_manager.cc:1918] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.838151 14537 tablet_service.cc:1467] Processing CreateTablet for tablet eeff2cd0825c4419aacfc73ebe267a8e (DEFAULT_TABLE table=default.Table_8ecc7de5c6b3497fb2d931c10ec6b373 [id=021f5685630741a8880d2471f7d2d1d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.838343 14823 log.cc:1198] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/080224910d554389b646e2c250fada2d
I20250115 03:06:14.838387 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeff2cd0825c4419aacfc73ebe267a8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.838618 14823 ts_tablet_manager.cc:1939] T 080224910d554389b646e2c250fada2d P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:14.839339 14529 tablet_service.cc:1467] Processing CreateTablet for tablet df44c5f5a9ae4d1aa7647dcbe2303a39 (DEFAULT_TABLE table=default.Table_8350ec7f96f743449d3122b98ad087d7 [id=8bf1da8d633940258affd0e11c334786]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.839423 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 080224910d554389b646e2c250fada2d (table default.Table_929dc98c434d495da9e1e46de64f5ef9 [id=ade8ca55065d405b994a155664fa3eb1]) successfully deleted
I20250115 03:06:14.839576 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df44c5f5a9ae4d1aa7647dcbe2303a39. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:14.839586 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 99d9180ed9b24363a4c44d01ea1d6e9b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 99d9180ed9b24363a4c44d01ea1d6e9b already in progress: creating tablet
W20250115 03:06:14.840605 15491 negotiation.cc:337] Failed RPC negotiation. Trace:
0115 03:06:14.836509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.186.252:41855 (local address 127.0.0.1:55680)
0115 03:06:14.839875 (+ 3366us) negotiation.cc:107] Waiting for socket to connect
0115 03:06:14.839924 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.186.252:41855: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":3296,"thread_start_us":54,"threads_started":1}
W20250115 03:06:14.840818 14156 consensus_peers.cc:487] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 -> Peer 2db8cff032af46db986bf52b21b52246 (127.13.186.252:41855): Couldn't send request to peer 2db8cff032af46db986bf52b21b52246. Status: Network error: Client connection negotiation failed: client connection to 127.13.186.252:41855: connect: Connection reset by peer (error 104). This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:14.824210 14819 ts_tablet_manager.cc:1905] T 50df3dd199bc4283bc1836d1917988a8 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.824831 14358 leader_election.cc:304] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:14.842820 14861 raft_consensus.cc:2798] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:14.842895 14861 raft_consensus.cc:695] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:14.843001 14861 consensus_queue.cc:237] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.843317 14819 ts_tablet_manager.cc:1918] T 50df3dd199bc4283bc1836d1917988a8 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.843372 14819 log.cc:1198] T 50df3dd199bc4283bc1836d1917988a8 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/50df3dd199bc4283bc1836d1917988a8
I20250115 03:06:14.843562 14819 ts_tablet_manager.cc:1939] T 50df3dd199bc4283bc1836d1917988a8 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.843868 15274 tablet.cc:1721] T 1ef14443e7fd464a8ab82eb33819dac3 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.844086 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 50df3dd199bc4283bc1836d1917988a8 (table default.Table_d75a227f24d644ceb2bda90d1f6ae998 [id=c72398a7988c48cc99ae0d244be1ac1a]) successfully deleted
I20250115 03:06:14.825965 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d3157859e684da98bb473d5799b24a6" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:14.844822 14628 leader_election.cc:343] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.844892 14628 leader_election.cc:304] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.845309 15153 raft_consensus.cc:2743] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:14.845804 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 33380937a3b7428995b06e477a896c02 (DEFAULT_TABLE table=default.Table_90aedfcdb1274e1497cf36620b980768 [id=9e4f5de45eb24beabef550d15dcad8bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.845834 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 420b7c8d43a04bae972eed2014d5634c (DEFAULT_TABLE table=default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.846045 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33380937a3b7428995b06e477a896c02. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.846948 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 420b7c8d43a04bae972eed2014d5634c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.847564 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 33380937a3b7428995b06e477a896c02 (DEFAULT_TABLE table=default.Table_90aedfcdb1274e1497cf36620b980768 [id=9e4f5de45eb24beabef550d15dcad8bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.847823 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33380937a3b7428995b06e477a896c02. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.847923 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 420b7c8d43a04bae972eed2014d5634c (DEFAULT_TABLE table=default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.848690 15426 tablet.cc:1721] T 1ef14443e7fd464a8ab82eb33819dac3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.848788 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 3a0bb9b5093b454d921bf1150d631565 (DEFAULT_TABLE table=default.Table_53c9a73ae8ef459c987964efac552c96 [id=37d965d8a5eb408ea3f33313bd22c152]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.849515 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 420b7c8d43a04bae972eed2014d5634c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.850219 14664 tablet_service.cc:1467] Processing CreateTablet for tablet f8b90f854a5e4221a24986f8c71ca300 (DEFAULT_TABLE table=default.Table_ec8ee0d483474271a3b369a2bcde593d [id=d8084de64a264d28a3370971617a694c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.850415 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8b90f854a5e4221a24986f8c71ca300. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.850630 14396 tablet_service.cc:1467] Processing CreateTablet for tablet f8b90f854a5e4221a24986f8c71ca300 (DEFAULT_TABLE table=default.Table_ec8ee0d483474271a3b369a2bcde593d [id=d8084de64a264d28a3370971617a694c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.850821 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8b90f854a5e4221a24986f8c71ca300. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.850963 15421 tablet.cc:1721] T 1ef14443e7fd464a8ab82eb33819dac3 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.851379 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a0bb9b5093b454d921bf1150d631565. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.831450 15390 raft_consensus.cc:491] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20250115 03:06:14.835925 14356 connection.cc:537] client connection to 127.13.186.252:41855 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20250115 03:06:14.853399 15390 raft_consensus.cc:513] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.837328 14394 tablet_service.cc:1467] Processing CreateTablet for tablet eeff2cd0825c4419aacfc73ebe267a8e (DEFAULT_TABLE table=default.Table_8ecc7de5c6b3497fb2d931c10ec6b373 [id=021f5685630741a8880d2471f7d2d1d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.853675 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeff2cd0825c4419aacfc73ebe267a8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.853705 15390 leader_election.cc:290] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.835587 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a56ae8a3e3e4f57b13e762cef21f753 with delete_type TABLET_DATA_DELETED (replaced by tablet 5dd115f83c544c4bba0d08fd82220952) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.838236 14809 tablet_bootstrap.cc:492] T 2dffa87697284dca8f07719f1c861bd4 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.854169 14809 ts_tablet_manager.cc:1397] T 2dffa87697284dca8f07719f1c861bd4 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.000s
I20250115 03:06:14.854588 14809 raft_consensus.cc:357] T 2dffa87697284dca8f07719f1c861bd4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.854683 14809 raft_consensus.cc:383] T 2dffa87697284dca8f07719f1c861bd4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.854715 14809 raft_consensus.cc:738] T 2dffa87697284dca8f07719f1c861bd4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.854815 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5504a9b8a4ea47e2b4cfd98c92e25d5c" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:14.854817 14809 consensus_queue.cc:260] T 2dffa87697284dca8f07719f1c861bd4 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.854921 14689 raft_consensus.cc:2463] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:14.855068 14809 ts_tablet_manager.cc:1428] T 2dffa87697284dca8f07719f1c861bd4 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.837739 14400 tablet_service.cc:1467] Processing CreateTablet for tablet df44c5f5a9ae4d1aa7647dcbe2303a39 (DEFAULT_TABLE table=default.Table_8350ec7f96f743449d3122b98ad087d7 [id=8bf1da8d633940258affd0e11c334786]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.855261 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 3a0bb9b5093b454d921bf1150d631565 (DEFAULT_TABLE table=default.Table_53c9a73ae8ef459c987964efac552c96 [id=37d965d8a5eb408ea3f33313bd22c152]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.855386 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df44c5f5a9ae4d1aa7647dcbe2303a39. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.855542 14358 leader_election.cc:304] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:14.855227 14809 tablet_bootstrap.cc:492] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.856278 15401 raft_consensus.cc:491] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.856343 15401 raft_consensus.cc:513] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.856645 15401 leader_election.cc:290] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.856742 14809 tablet_bootstrap.cc:654] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.856745 15401 raft_consensus.cc:2798] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.856814 15401 raft_consensus.cc:491] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.856860 15401 raft_consensus.cc:3054] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.857535 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5504a9b8a4ea47e2b4cfd98c92e25d5c" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.857692 15401 raft_consensus.cc:513] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:14.857805 14356 leader_election.cc:343] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.857892 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55dcc7ac82ac44ab9d85055720760d3d" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.857990 14556 raft_consensus.cc:2463] T 55dcc7ac82ac44ab9d85055720760d3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:14.857995 15401 leader_election.cc:290] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.858211 14356 leader_election.cc:304] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:14.858300 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5504a9b8a4ea47e2b4cfd98c92e25d5c" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:14.858553 15401 raft_consensus.cc:2798] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.858610 15401 raft_consensus.cc:491] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.858594 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5504a9b8a4ea47e2b4cfd98c92e25d5c" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:14.858650 15401 raft_consensus.cc:3054] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.858680 14691 raft_consensus.cc:3054] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.859359 14170 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c (table default.Table_4bfe9fa1f7a143d684bc948b47deb8d8 [id=339b61af27cb486aaaadf8730ab21ce7]). Expected version 1 got 0
I20250115 03:06:14.859369 15401 raft_consensus.cc:513] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:14.859400 14691 raft_consensus.cc:2463] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:14.859643 15401 leader_election.cc:290] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
W20250115 03:06:14.859946 14356 leader_election.cc:343] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.853309 14396 tablet_service.cc:1467] Processing CreateTablet for tablet b0a5d554764249f8a7eefdc943271090 (DEFAULT_TABLE table=default.Table_39e0d37b84fc42c0807a4cbffb7bae0a [id=4b6ce1381c024498a78d91e346f488f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.860347 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55dcc7ac82ac44ab9d85055720760d3d" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:14.860414 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0a5d554764249f8a7eefdc943271090. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.860441 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55dcc7ac82ac44ab9d85055720760d3d" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:14.860209 14358 leader_election.cc:304] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:14.860666 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55dcc7ac82ac44ab9d85055720760d3d" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:14.860740 14556 raft_consensus.cc:3054] T 55dcc7ac82ac44ab9d85055720760d3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.861063 14861 raft_consensus.cc:2798] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:14.861145 14861 raft_consensus.cc:695] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:14.861268 14861 consensus_queue.cc:237] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.861524 14556 raft_consensus.cc:2463] T 55dcc7ac82ac44ab9d85055720760d3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:14.861757 14356 leader_election.cc:304] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:14.862062 14885 raft_consensus.cc:2798] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:14.862138 14885 raft_consensus.cc:695] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:14.862241 14885 consensus_queue.cc:237] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
W20250115 03:06:14.853020 15486 consensus_queue.cc:1159] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" status { last_received { term: 1 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 1 } }
I20250115 03:06:14.862826 14820 ts_tablet_manager.cc:1905] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.864713 14820 ts_tablet_manager.cc:1918] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:14.860445 14689 raft_consensus.cc:2463] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:14.864784 14820 log.cc:1198] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/50df3dd199bc4283bc1836d1917988a8
I20250115 03:06:14.865123 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a0bb9b5093b454d921bf1150d631565. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.865290 14820 ts_tablet_manager.cc:1939] T 50df3dd199bc4283bc1836d1917988a8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:14.867298 14672 tablet_service.cc:1467] Processing CreateTablet for tablet df44c5f5a9ae4d1aa7647dcbe2303a39 (DEFAULT_TABLE table=default.Table_8350ec7f96f743449d3122b98ad087d7 [id=8bf1da8d633940258affd0e11c334786]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.867497 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df44c5f5a9ae4d1aa7647dcbe2303a39. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.869426 14809 tablet_bootstrap.cc:492] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.869510 14809 ts_tablet_manager.cc:1397] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20250115 03:06:14.869948 14809 raft_consensus.cc:357] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.870035 14809 raft_consensus.cc:383] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.870066 14809 raft_consensus.cc:738] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.870174 14809 consensus_queue.cc:260] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.870416 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 341b24d9b63f4761b2013035023fcd98 (DEFAULT_TABLE table=default.Table_af3419632bb0478bab77f1fa32e9fb52 [id=369c42ec23ec4a27a71a54be23a410a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.870460 14809 ts_tablet_manager.cc:1428] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.870448 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 054251151c294e06905092f2a9306849 (DEFAULT_TABLE table=default.Table_849d2ef8d13549b7bb6bfb03314d998b [id=54fd71a4b30d41b7b59b8dfc261a616d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.870620 14809 tablet_bootstrap.cc:492] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.870663 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054251151c294e06905092f2a9306849. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.870767 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 341b24d9b63f4761b2013035023fcd98. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.871495 14809 tablet_bootstrap.cc:654] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.872231 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_129ccce41ec44747b8b0a8c0df193bf8" } new_table_name: "default.Table_632fc9d76fb3482c8a5e87feda0ceeed" modify_external_catalogs: true
I20250115 03:06:14.872617 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 420b7c8d43a04bae972eed2014d5634c (DEFAULT_TABLE table=default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.872819 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 420b7c8d43a04bae972eed2014d5634c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.872835 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 3a0bb9b5093b454d921bf1150d631565 (DEFAULT_TABLE table=default.Table_53c9a73ae8ef459c987964efac552c96 [id=37d965d8a5eb408ea3f33313bd22c152]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.873100 14528 tablet_service.cc:1467] Processing CreateTablet for tablet f8b90f854a5e4221a24986f8c71ca300 (DEFAULT_TABLE table=default.Table_ec8ee0d483474271a3b369a2bcde593d [id=d8084de64a264d28a3370971617a694c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.873247 14536 tablet_service.cc:1467] Processing CreateTablet for tablet b0a5d554764249f8a7eefdc943271090 (DEFAULT_TABLE table=default.Table_39e0d37b84fc42c0807a4cbffb7bae0a [id=4b6ce1381c024498a78d91e346f488f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.873442 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 054251151c294e06905092f2a9306849 (DEFAULT_TABLE table=default.Table_849d2ef8d13549b7bb6bfb03314d998b [id=54fd71a4b30d41b7b59b8dfc261a616d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.873097 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a0bb9b5093b454d921bf1150d631565. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.873976 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0a5d554764249f8a7eefdc943271090. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.875325 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 054251151c294e06905092f2a9306849 (DEFAULT_TABLE table=default.Table_849d2ef8d13549b7bb6bfb03314d998b [id=54fd71a4b30d41b7b59b8dfc261a616d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.869495 14819 tablet_replica.cc:331] T 673f9e6cabdd4320b777181c787d15c7 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.873574 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 341b24d9b63f4761b2013035023fcd98 (DEFAULT_TABLE table=default.Table_af3419632bb0478bab77f1fa32e9fb52 [id=369c42ec23ec4a27a71a54be23a410a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.875689 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 341b24d9b63f4761b2013035023fcd98. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.875916 14819 raft_consensus.cc:2238] T 673f9e6cabdd4320b777181c787d15c7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.876011 14819 raft_consensus.cc:2267] T 673f9e6cabdd4320b777181c787d15c7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.876772 14529 tablet_service.cc:1467] Processing CreateTablet for tablet 33380937a3b7428995b06e477a896c02 (DEFAULT_TABLE table=default.Table_90aedfcdb1274e1497cf36620b980768 [id=9e4f5de45eb24beabef550d15dcad8bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.876991 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33380937a3b7428995b06e477a896c02. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.877274 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054251151c294e06905092f2a9306849. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.878942 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8b90f854a5e4221a24986f8c71ca300. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.879092 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_3e911fd952934f79bee759f184cbcce0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.879396 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3e911fd952934f79bee759f184cbcce0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:14.880023 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b1644ba8616349dc9d10a790b6089571 consensus state has no leader
I20250115 03:06:14.882408 14812 tablet_bootstrap.cc:492] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.882505 14812 ts_tablet_manager.cc:1397] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.073s user 0.002s sys 0.000s
I20250115 03:06:14.882947 14812 raft_consensus.cc:357] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.883039 14812 raft_consensus.cc:383] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.883071 14812 raft_consensus.cc:738] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.883183 14812 consensus_queue.cc:260] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.883452 14812 ts_tablet_manager.cc:1428] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:14.884064 14812 tablet_bootstrap.cc:492] T 55f01d704b174b2ea108ea6f1df763df P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.885200 14812 tablet_bootstrap.cc:654] T 55f01d704b174b2ea108ea6f1df763df P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.885836 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054251151c294e06905092f2a9306849. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.886024 15325 raft_consensus.cc:491] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.886111 15325 raft_consensus.cc:513] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.886406 15325 leader_election.cc:290] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:14.887060 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b22aa85ce34b38b0d6a5e2d38f0a1c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:14.887408 14628 leader_election.cc:343] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.888643 14811 tablet_bootstrap.cc:492] T f54942abdfb748f2a939993a3c2d5db0 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.888722 14811 ts_tablet_manager.cc:1397] T f54942abdfb748f2a939993a3c2d5db0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.082s user 0.001s sys 0.001s
I20250115 03:06:14.889207 14811 raft_consensus.cc:357] T f54942abdfb748f2a939993a3c2d5db0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.889302 14811 raft_consensus.cc:383] T f54942abdfb748f2a939993a3c2d5db0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.889302 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b22aa85ce34b38b0d6a5e2d38f0a1c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.889637 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 341b24d9b63f4761b2013035023fcd98 (DEFAULT_TABLE table=default.Table_af3419632bb0478bab77f1fa32e9fb52 [id=369c42ec23ec4a27a71a54be23a410a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.889355 14811 raft_consensus.cc:738] T f54942abdfb748f2a939993a3c2d5db0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.889856 14811 consensus_queue.cc:260] T f54942abdfb748f2a939993a3c2d5db0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.890082 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 341b24d9b63f4761b2013035023fcd98. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.890110 14811 ts_tablet_manager.cc:1428] T f54942abdfb748f2a939993a3c2d5db0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.890254 14811 tablet_bootstrap.cc:492] T 080224910d554389b646e2c250fada2d P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.891136 14811 tablet_bootstrap.cc:654] T 080224910d554389b646e2c250fada2d P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:14.891988 14626 leader_election.cc:343] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.892053 14626 leader_election.cc:304] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:14.892210 15325 raft_consensus.cc:2743] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:14.892263 14811 tablet_bootstrap.cc:492] T 080224910d554389b646e2c250fada2d P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.892331 14811 ts_tablet_manager.cc:1397] T 080224910d554389b646e2c250fada2d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:14.892728 14811 raft_consensus.cc:357] T 080224910d554389b646e2c250fada2d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.892815 14811 raft_consensus.cc:383] T 080224910d554389b646e2c250fada2d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.892845 14811 raft_consensus.cc:738] T 080224910d554389b646e2c250fada2d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.872241 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 50df3dd199bc4283bc1836d1917988a8 (table default.Table_d75a227f24d644ceb2bda90d1f6ae998 [id=c72398a7988c48cc99ae0d244be1ac1a]) successfully deleted
W20250115 03:06:14.892988 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 080224910d554389b646e2c250fada2d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 080224910d554389b646e2c250fada2d already in progress: creating tablet
I20250115 03:06:14.892972 14811 consensus_queue.cc:260] T 080224910d554389b646e2c250fada2d P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:14.893102 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 99d9180ed9b24363a4c44d01ea1d6e9b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 99d9180ed9b24363a4c44d01ea1d6e9b already in progress: creating tablet
I20250115 03:06:14.893224 14811 ts_tablet_manager.cc:1428] T 080224910d554389b646e2c250fada2d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.893373 14811 tablet_bootstrap.cc:492] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.894241 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_638e39bcf2aa4131ac57afbe63ac9b73" } modify_external_catalogs: true
I20250115 03:06:14.894320 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_65f0fd5511f64bfeb6539f9019a4873c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:14.894418 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_04eafaab3f094b2998237e6ca1440386"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:14.894474 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet b0a5d554764249f8a7eefdc943271090 from {username='slave'} at 127.0.0.1:60432
W20250115 03:06:14.894661 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65f0fd5511f64bfeb6539f9019a4873c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:14.894734 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_04eafaab3f094b2998237e6ca1440386 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.895336 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_bac546b42308426c9fd8a4ac15b547b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.895628 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bac546b42308426c9fd8a4ac15b547b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.895782 14672 tablet_service.cc:1467] Processing CreateTablet for tablet b0a5d554764249f8a7eefdc943271090 (DEFAULT_TABLE table=default.Table_39e0d37b84fc42c0807a4cbffb7bae0a [id=4b6ce1381c024498a78d91e346f488f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.896006 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0a5d554764249f8a7eefdc943271090. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.895347 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_13681f62285b46bc88883e9c804c888e" } new_table_name: "default.Table_6f23f5e22da844029c46908c62aeeb23" modify_external_catalogs: true
I20250115 03:06:14.897014 14689 raft_consensus.cc:1270] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:14.897152 14421 raft_consensus.cc:1270] T 0580a8f7b2984162b9275d74334b1905 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:14.897358 14853 consensus_queue.cc:1035] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.897964 14853 consensus_queue.cc:1035] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:14.894333 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_638e39bcf2aa4131ac57afbe63ac9b73" } modify_external_catalogs: true
I20250115 03:06:14.901779 15426 tablet.cc:1721] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.901865 14811 tablet_bootstrap.cc:654] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.902788 15274 tablet.cc:1721] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.903846 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet b0a5d554764249f8a7eefdc943271090
I20250115 03:06:14.904101 14171 catalog_manager.cc:6596] ReplaceTablet: tablet b0a5d554764249f8a7eefdc943271090 deleted and replaced by tablet d895f75911b7456a88e7a5159db7d60b
W20250115 03:06:14.907295 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5dd115f83c544c4bba0d08fd82220952 consensus state has no leader
I20250115 03:06:14.908707 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet b0a5d554764249f8a7eefdc943271090 with delete_type TABLET_DATA_DELETED (replaced by tablet d895f75911b7456a88e7a5159db7d60b) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:14.910732 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet b0a5d554764249f8a7eefdc943271090 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b0a5d554764249f8a7eefdc943271090 already in progress: creating tablet
I20250115 03:06:14.913980 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2d802ab3dfcf4a54ae7c59c4c98d8446" } new_table_name: "default.Table_bbfcc61694f44215aa5736bcc075ec0e" modify_external_catalogs: true
I20250115 03:06:14.914690 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet b0a5d554764249f8a7eefdc943271090 with delete_type TABLET_DATA_DELETED (replaced by tablet d895f75911b7456a88e7a5159db7d60b) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.914913 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet b0a5d554764249f8a7eefdc943271090 with delete_type TABLET_DATA_DELETED (replaced by tablet d895f75911b7456a88e7a5159db7d60b) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.915160 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3863dfaf57694ee7b4a9ba1e400d7c9d" } modify_external_catalogs: true
I20250115 03:06:14.915266 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3863dfaf57694ee7b4a9ba1e400d7c9d" } modify_external_catalogs: true
W20250115 03:06:14.915144 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet b0a5d554764249f8a7eefdc943271090 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b0a5d554764249f8a7eefdc943271090 already in progress: creating tablet
I20250115 03:06:14.916396 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet 47b08f6e7dec4c5a871a49c79421209e from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:14.917495 14819 ts_tablet_manager.cc:1905] T 673f9e6cabdd4320b777181c787d15c7 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.920277 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet 914d0f374564460480f93aff40fee17f from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:14.920444 14819 ts_tablet_manager.cc:1918] T 673f9e6cabdd4320b777181c787d15c7 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.920511 14819 log.cc:1198] T 673f9e6cabdd4320b777181c787d15c7 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/673f9e6cabdd4320b777181c787d15c7
I20250115 03:06:14.920703 14819 ts_tablet_manager.cc:1939] T 673f9e6cabdd4320b777181c787d15c7 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.923012 15274 tablet.cc:1721] T c4a63742df234d30bdac0173693159e4 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.923269 15426 tablet.cc:1721] T c4a63742df234d30bdac0173693159e4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.929123 14809 tablet_bootstrap.cc:492] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.929224 14809 ts_tablet_manager.cc:1397] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.059s user 0.002s sys 0.000s
I20250115 03:06:14.929600 14809 raft_consensus.cc:357] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.929709 14809 raft_consensus.cc:383] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.929750 14809 raft_consensus.cc:738] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.929894 14809 consensus_queue.cc:260] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.930065 14861 raft_consensus.cc:491] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.930125 14861 raft_consensus.cc:513] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.930426 14861 leader_election.cc:290] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.930416 15421 tablet.cc:1721] T 0580a8f7b2984162b9275d74334b1905 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.930835 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f54942abdfb748f2a939993a3c2d5db0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.930938 14556 raft_consensus.cc:2463] T f54942abdfb748f2a939993a3c2d5db0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
W20250115 03:06:14.931149 14358 leader_election.cc:343] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: f54942abdfb748f2a939993a3c2d5db0
I20250115 03:06:14.931282 14809 ts_tablet_manager.cc:1428] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:14.931313 14356 leader_election.cc:304] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770
I20250115 03:06:14.931519 15401 raft_consensus.cc:2798] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.931566 15401 raft_consensus.cc:491] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.931547 14969 tablet.cc:1721] T c4a63742df234d30bdac0173693159e4 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:14.931604 15401 raft_consensus.cc:3054] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.931948 14809 tablet_bootstrap.cc:492] T 99ef00eb188a4f628ecbf8fd8906b6b5 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.932265 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 673f9e6cabdd4320b777181c787d15c7 (table default.Table_42a1977fa70d4f16b43bf861b66524a7 [id=c9b28587ba934521bd4158dfaf416f6e]) successfully deleted
I20250115 03:06:14.932471 14819 tablet_replica.cc:331] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.932760 15401 raft_consensus.cc:513] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.932986 14809 tablet_bootstrap.cc:654] T 99ef00eb188a4f628ecbf8fd8906b6b5 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.933386 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f54942abdfb748f2a939993a3c2d5db0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:14.933478 14556 raft_consensus.cc:3054] T f54942abdfb748f2a939993a3c2d5db0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.933554 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b1775fbfc0eb4cf4a59a0f9a853a081e" } modify_external_catalogs: true
I20250115 03:06:14.933653 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b1775fbfc0eb4cf4a59a0f9a853a081e" } modify_external_catalogs: true
I20250115 03:06:14.934293 14556 raft_consensus.cc:2463] T f54942abdfb748f2a939993a3c2d5db0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:14.932969 14819 raft_consensus.cc:2238] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.935601 14819 raft_consensus.cc:2267] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.933096 15401 leader_election.cc:290] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:14.934031 14358 leader_election.cc:343] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: f54942abdfb748f2a939993a3c2d5db0
I20250115 03:06:14.936046 14356 leader_election.cc:304] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770
W20250115 03:06:14.936131 14356 raft_consensus.cc:2681] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75: unable to run election callback: Service unavailable: Thread pool token was shut down
I20250115 03:06:14.937387 14809 tablet_bootstrap.cc:492] T 99ef00eb188a4f628ecbf8fd8906b6b5 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.937474 14809 ts_tablet_manager.cc:1397] T 99ef00eb188a4f628ecbf8fd8906b6b5 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20250115 03:06:14.937906 14809 raft_consensus.cc:357] T 99ef00eb188a4f628ecbf8fd8906b6b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.937994 14809 raft_consensus.cc:383] T 99ef00eb188a4f628ecbf8fd8906b6b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.938019 14809 raft_consensus.cc:738] T 99ef00eb188a4f628ecbf8fd8906b6b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.938110 14809 consensus_queue.cc:260] T 99ef00eb188a4f628ecbf8fd8906b6b5 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.938323 14809 ts_tablet_manager.cc:1428] T 99ef00eb188a4f628ecbf8fd8906b6b5 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:14.938603 14809 tablet_bootstrap.cc:492] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.939421 14809 tablet_bootstrap.cc:654] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.940233 14809 tablet_bootstrap.cc:492] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.940299 14809 ts_tablet_manager.cc:1397] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20250115 03:06:14.940649 14809 raft_consensus.cc:357] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.940726 14809 raft_consensus.cc:383] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.940752 14809 raft_consensus.cc:738] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.940968 14809 consensus_queue.cc:260] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.942034 14809 ts_tablet_manager.cc:1428] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:14.942206 14809 tablet_bootstrap.cc:492] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.937407 14819 ts_tablet_manager.cc:1905] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.943238 14812 tablet_bootstrap.cc:492] T 55f01d704b174b2ea108ea6f1df763df P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.943320 14812 ts_tablet_manager.cc:1397] T 55f01d704b174b2ea108ea6f1df763df P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.059s user 0.002s sys 0.000s
I20250115 03:06:14.943738 14812 raft_consensus.cc:357] T 55f01d704b174b2ea108ea6f1df763df P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.943838 14809 tablet_bootstrap.cc:654] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.943917 14812 raft_consensus.cc:383] T 55f01d704b174b2ea108ea6f1df763df P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.943990 14812 raft_consensus.cc:738] T 55f01d704b174b2ea108ea6f1df763df P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.944168 14812 consensus_queue.cc:260] T 55f01d704b174b2ea108ea6f1df763df P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.945652 14809 tablet_bootstrap.cc:492] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.945717 14809 ts_tablet_manager.cc:1397] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250115 03:06:14.945766 14812 ts_tablet_manager.cc:1428] T 55f01d704b174b2ea108ea6f1df763df P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:14.946072 14809 raft_consensus.cc:357] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.946169 14809 raft_consensus.cc:383] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.946195 14809 raft_consensus.cc:738] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.946290 14809 consensus_queue.cc:260] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.946499 14809 ts_tablet_manager.cc:1428] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.946511 14819 ts_tablet_manager.cc:1918] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.946571 14819 log.cc:1198] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/f54942abdfb748f2a939993a3c2d5db0
I20250115 03:06:14.946624 14809 tablet_bootstrap.cc:492] T 37351252b608492e9acbc818b2754731 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.946764 14819 ts_tablet_manager.cc:1939] T f54942abdfb748f2a939993a3c2d5db0 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.947284 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet f54942abdfb748f2a939993a3c2d5db0 (table default.Table_5e6d1abc27d042dba8506da7b49a6ee3 [id=a5c946319e8a4be5ba4a01a40a9ad0b7]) successfully deleted
I20250115 03:06:14.947352 14809 tablet_bootstrap.cc:654] T 37351252b608492e9acbc818b2754731 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.947397 14812 tablet_bootstrap.cc:492] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.948277 14812 tablet_bootstrap.cc:654] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:14.948881 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 99d9180ed9b24363a4c44d01ea1d6e9b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 99d9180ed9b24363a4c44d01ea1d6e9b already in progress: creating tablet
I20250115 03:06:14.949331 14812 tablet_bootstrap.cc:492] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.949447 14812 ts_tablet_manager.cc:1397] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:14.949483 14811 tablet_bootstrap.cc:492] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.949622 14811 ts_tablet_manager.cc:1397] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.056s user 0.002s sys 0.000s
I20250115 03:06:14.949877 14819 tablet_replica.cc:331] T 4a56ae8a3e3e4f57b13e762cef21f753 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.949986 14819 raft_consensus.cc:2238] T 4a56ae8a3e3e4f57b13e762cef21f753 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.950073 14819 raft_consensus.cc:2267] T 4a56ae8a3e3e4f57b13e762cef21f753 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.950114 14812 raft_consensus.cc:357] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.950268 14812 raft_consensus.cc:383] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.950248 14811 raft_consensus.cc:357] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.950343 14812 raft_consensus.cc:738] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.950423 14819 ts_tablet_manager.cc:1905] T 4a56ae8a3e3e4f57b13e762cef21f753 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.950531 14812 consensus_queue.cc:260] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.950601 14811 raft_consensus.cc:383] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.950651 14809 tablet_bootstrap.cc:492] T 37351252b608492e9acbc818b2754731 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.950651 14811 raft_consensus.cc:738] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.950735 14809 ts_tablet_manager.cc:1397] T 37351252b608492e9acbc818b2754731 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:14.950785 14811 consensus_queue.cc:260] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.951076 14809 raft_consensus.cc:357] T 37351252b608492e9acbc818b2754731 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.951164 14809 raft_consensus.cc:383] T 37351252b608492e9acbc818b2754731 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.951191 14809 raft_consensus.cc:738] T 37351252b608492e9acbc818b2754731 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.951282 14809 consensus_queue.cc:260] T 37351252b608492e9acbc818b2754731 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.951521 14809 ts_tablet_manager.cc:1428] T 37351252b608492e9acbc818b2754731 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:14.951663 14809 tablet_bootstrap.cc:492] T 382116b1fb6a4f888ab1d55c29917ec8 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.952342 14809 tablet_bootstrap.cc:654] T 382116b1fb6a4f888ab1d55c29917ec8 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.952383 14811 ts_tablet_manager.cc:1428] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:14.952867 14811 tablet_bootstrap.cc:492] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.953238 14171 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_20a4cbca90414ebdac47ec2b786c52eb" } new_table_name: "default.Table_cbfbacbaeee4488aba1a8be104138404" modify_external_catalogs: true
I20250115 03:06:14.953297 14812 ts_tablet_manager.cc:1428] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:14.953933 14812 tablet_bootstrap.cc:492] T 914d0f374564460480f93aff40fee17f P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.954159 14811 tablet_bootstrap.cc:654] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.954751 14812 tablet_bootstrap.cc:654] T 914d0f374564460480f93aff40fee17f P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.955574 14812 tablet_bootstrap.cc:492] T 914d0f374564460480f93aff40fee17f P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:14.955654 14812 ts_tablet_manager.cc:1397] T 914d0f374564460480f93aff40fee17f P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:14.956035 14811 tablet_bootstrap.cc:492] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:14.956035 14812 raft_consensus.cc:357] T 914d0f374564460480f93aff40fee17f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.956132 14811 ts_tablet_manager.cc:1397] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:14.956146 14812 raft_consensus.cc:383] T 914d0f374564460480f93aff40fee17f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.956199 14812 raft_consensus.cc:738] T 914d0f374564460480f93aff40fee17f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.956342 14812 consensus_queue.cc:260] T 914d0f374564460480f93aff40fee17f P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.956540 14811 raft_consensus.cc:357] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.956796 14812 ts_tablet_manager.cc:1428] T 914d0f374564460480f93aff40fee17f P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.956887 14811 raft_consensus.cc:383] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.956976 14811 raft_consensus.cc:738] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.956992 14812 tablet_bootstrap.cc:492] T 53248628d9584f8b886166c1e8ec5c3c P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:14.957108 14811 consensus_queue.cc:260] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.957391 14811 ts_tablet_manager.cc:1428] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.958076 14812 tablet_bootstrap.cc:654] T 53248628d9584f8b886166c1e8ec5c3c P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.958225 14811 tablet_bootstrap.cc:492] T 4a56ae8a3e3e4f57b13e762cef21f753 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:14.958902 14809 tablet_bootstrap.cc:492] T 382116b1fb6a4f888ab1d55c29917ec8 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.958984 14809 ts_tablet_manager.cc:1397] T 382116b1fb6a4f888ab1d55c29917ec8 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:14.959192 14811 tablet_bootstrap.cc:654] T 4a56ae8a3e3e4f57b13e762cef21f753 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.959322 14809 raft_consensus.cc:357] T 382116b1fb6a4f888ab1d55c29917ec8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.959424 14809 raft_consensus.cc:383] T 382116b1fb6a4f888ab1d55c29917ec8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.959470 14809 raft_consensus.cc:738] T 382116b1fb6a4f888ab1d55c29917ec8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.959578 14809 consensus_queue.cc:260] T 382116b1fb6a4f888ab1d55c29917ec8 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.959821 14809 ts_tablet_manager.cc:1428] T 382116b1fb6a4f888ab1d55c29917ec8 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:14.959981 14809 tablet_bootstrap.cc:492] T f978e85cf659450ca0039368d58b49e5 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.960777 14809 tablet_bootstrap.cc:654] T f978e85cf659450ca0039368d58b49e5 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:14.963447 14171 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ccf6b58c55d747da89c1c218a70ce09f consensus state has no leader
I20250115 03:06:14.963495 14819 ts_tablet_manager.cc:1918] T 4a56ae8a3e3e4f57b13e762cef21f753 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.964042 14819 log.cc:1198] T 4a56ae8a3e3e4f57b13e762cef21f753 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/4a56ae8a3e3e4f57b13e762cef21f753
I20250115 03:06:14.964241 14819 ts_tablet_manager.cc:1939] T 4a56ae8a3e3e4f57b13e762cef21f753 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.964756 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 4a56ae8a3e3e4f57b13e762cef21f753 (table default.Table_6f23f5e22da844029c46908c62aeeb23 [id=1b7d3a80c5424e5692c2e6c556919342]) successfully deleted
I20250115 03:06:14.965420 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet acb43d614111477abb10def48a1dfb3b from {username='slave'} at 127.0.0.1:60432
W20250115 03:06:14.966468 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet b0a5d554764249f8a7eefdc943271090 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b0a5d554764249f8a7eefdc943271090 already in progress: creating tablet
I20250115 03:06:14.967095 14809 tablet_bootstrap.cc:492] T f978e85cf659450ca0039368d58b49e5 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:14.967178 14809 ts_tablet_manager.cc:1397] T f978e85cf659450ca0039368d58b49e5 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:14.967583 14809 raft_consensus.cc:357] T f978e85cf659450ca0039368d58b49e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.967666 14809 raft_consensus.cc:383] T f978e85cf659450ca0039368d58b49e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:14.967572 14170 catalog_manager.cc:5526] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:14.967698 14809 raft_consensus.cc:738] T f978e85cf659450ca0039368d58b49e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:14.967833 14809 consensus_queue.cc:260] T f978e85cf659450ca0039368d58b49e5 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.967826 14170 catalog_manager.cc:5526] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:14.968019 14170 catalog_manager.cc:5526] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:14.968166 14170 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c (table default.Table_4bfe9fa1f7a143d684bc948b47deb8d8 [id=339b61af27cb486aaaadf8730ab21ce7]). Expected version 1 got 0
I20250115 03:06:14.968606 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_47ede7f24148438ba5b165a821f676bc [id=fd7ff6d50c944f3a80089030d9fc895d] alter complete (version 1)
I20250115 03:06:14.968830 14809 ts_tablet_manager.cc:1428] T f978e85cf659450ca0039368d58b49e5 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:14.969066 14809 tablet_bootstrap.cc:492] T 5dd115f83c544c4bba0d08fd82220952 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:14.969909 14809 tablet_bootstrap.cc:654] T 5dd115f83c544c4bba0d08fd82220952 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:14.972678 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 2d78a7793f84424bad42a1aba89eab92
I20250115 03:06:14.972887 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e303383ddf3845828de6837219313014"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.973212 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e303383ddf3845828de6837219313014 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.973279 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d78a7793f84424bad42a1aba89eab92 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.973505 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d78a7793f84424bad42a1aba89eab92 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.973656 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d78a7793f84424bad42a1aba89eab92 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:14.973744 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 2d78a7793f84424bad42a1aba89eab92 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d78a7793f84424bad42a1aba89eab92 already in progress: creating tablet
W20250115 03:06:14.974119 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 2d78a7793f84424bad42a1aba89eab92 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d78a7793f84424bad42a1aba89eab92 already in progress: creating tablet
I20250115 03:06:14.974175 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_1c52ca26880a48289f5916fabdb05ac4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.974252 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 2d78a7793f84424bad42a1aba89eab92 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d78a7793f84424bad42a1aba89eab92 already in progress: creating tablet
I20250115 03:06:14.974400 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dffa87697284dca8f07719f1c861bd4 with delete_type TABLET_DATA_DELETED (replaced by tablet a1cbe29c08c7477e8944b7e108d46e7a) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:14.974453 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1c52ca26880a48289f5916fabdb05ac4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.974552 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_863155db5aee457a9ada887faf6deb85"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:14.974813 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_863155db5aee457a9ada887faf6deb85 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.975705 14819 tablet_replica.cc:331] T 2dffa87697284dca8f07719f1c861bd4 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:14.975833 14819 raft_consensus.cc:2238] T 2dffa87697284dca8f07719f1c861bd4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:14.975915 14819 raft_consensus.cc:2267] T 2dffa87697284dca8f07719f1c861bd4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:14.976369 14819 ts_tablet_manager.cc:1905] T 2dffa87697284dca8f07719f1c861bd4 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:14.978524 14819 ts_tablet_manager.cc:1918] T 2dffa87697284dca8f07719f1c861bd4 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:14.978641 14819 log.cc:1198] T 2dffa87697284dca8f07719f1c861bd4 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/2dffa87697284dca8f07719f1c861bd4
I20250115 03:06:14.978901 14819 ts_tablet_manager.cc:1939] T 2dffa87697284dca8f07719f1c861bd4 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:14.979472 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 2dffa87697284dca8f07719f1c861bd4 (table default.Table_7e53fa73287c43e3a9a0e65908b42593 [id=1f983a10a8704e22b8a2b96ef4e06fbc]) successfully deleted
I20250115 03:06:14.979893 14527 tablet_service.cc:1467] Processing CreateTablet for tablet a75b9401cb4b4f8b92a4779c3704eb46 (DEFAULT_TABLE table=default.Table_a0c249d69b904390952a780b10e1369b [id=22d05297fb9545aeab39b5b62248d0d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.980130 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a75b9401cb4b4f8b92a4779c3704eb46. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.980226 14393 tablet_service.cc:1467] Processing CreateTablet for tablet a75b9401cb4b4f8b92a4779c3704eb46 (DEFAULT_TABLE table=default.Table_a0c249d69b904390952a780b10e1369b [id=22d05297fb9545aeab39b5b62248d0d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.980429 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a75b9401cb4b4f8b92a4779c3704eb46. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.981243 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 90277f7d279e4c7a83c43f40766cd11b (DEFAULT_TABLE table=default.Table_62fa8b23f4c44bb0938d6363bc2086ef [id=a888fcc40089456fba95b6e634ae85f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.981463 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90277f7d279e4c7a83c43f40766cd11b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.982338 14672 tablet_service.cc:1467] Processing CreateTablet for tablet a75b9401cb4b4f8b92a4779c3704eb46 (DEFAULT_TABLE table=default.Table_a0c249d69b904390952a780b10e1369b [id=22d05297fb9545aeab39b5b62248d0d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.982429 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 90277f7d279e4c7a83c43f40766cd11b (DEFAULT_TABLE table=default.Table_62fa8b23f4c44bb0938d6363bc2086ef [id=a888fcc40089456fba95b6e634ae85f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.982561 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a75b9401cb4b4f8b92a4779c3704eb46. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.982721 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90277f7d279e4c7a83c43f40766cd11b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.983383 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 90277f7d279e4c7a83c43f40766cd11b (DEFAULT_TABLE table=default.Table_62fa8b23f4c44bb0938d6363bc2086ef [id=a888fcc40089456fba95b6e634ae85f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.983614 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90277f7d279e4c7a83c43f40766cd11b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.984676 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 5986558350244e7aa4ddcd5cccb7d676 (DEFAULT_TABLE table=default.Table_f960fb76bf0d4e3288988b6a3abb2e8c [id=6ab343bee1254fb19a3024ebafa44464]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.984784 14393 tablet_service.cc:1467] Processing CreateTablet for tablet d28ed8538c4a44d18279371b826c22a5 (DEFAULT_TABLE table=default.Table_fe0a1f3ae7a04225a5a482c33a15c579 [id=c99c3573d2394eada9631427ac38322d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.984871 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5986558350244e7aa4ddcd5cccb7d676. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.984886 14527 tablet_service.cc:1467] Processing CreateTablet for tablet d28ed8538c4a44d18279371b826c22a5 (DEFAULT_TABLE table=default.Table_fe0a1f3ae7a04225a5a482c33a15c579 [id=c99c3573d2394eada9631427ac38322d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.984997 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d28ed8538c4a44d18279371b826c22a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.985114 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d28ed8538c4a44d18279371b826c22a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.985493 14663 tablet_service.cc:1467] Processing CreateTablet for tablet d28ed8538c4a44d18279371b826c22a5 (DEFAULT_TABLE table=default.Table_fe0a1f3ae7a04225a5a482c33a15c579 [id=c99c3573d2394eada9631427ac38322d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.985692 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d28ed8538c4a44d18279371b826c22a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.987519 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 5986558350244e7aa4ddcd5cccb7d676 (DEFAULT_TABLE table=default.Table_f960fb76bf0d4e3288988b6a3abb2e8c [id=6ab343bee1254fb19a3024ebafa44464]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.987582 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 5986558350244e7aa4ddcd5cccb7d676 (DEFAULT_TABLE table=default.Table_f960fb76bf0d4e3288988b6a3abb2e8c [id=6ab343bee1254fb19a3024ebafa44464]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.987746 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5986558350244e7aa4ddcd5cccb7d676. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.987764 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5986558350244e7aa4ddcd5cccb7d676. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.988117 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 6d10731df02f4e9682b8de6962256635 (DEFAULT_TABLE table=default.Table_ad312d1f941c45b5a0dc5324967d5105 [id=79eb8c9694b74a31a4253b33d064e230]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.988302 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d10731df02f4e9682b8de6962256635. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.989662 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 6d10731df02f4e9682b8de6962256635 (DEFAULT_TABLE table=default.Table_ad312d1f941c45b5a0dc5324967d5105 [id=79eb8c9694b74a31a4253b33d064e230]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.989957 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d10731df02f4e9682b8de6962256635. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.990782 15325 raft_consensus.cc:491] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.990855 15325 raft_consensus.cc:513] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.991186 15325 leader_election.cc:290] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:14.991611 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_6df09ae8a979496abf73d384649b81b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:14.991626 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f978e85cf659450ca0039368d58b49e5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:14.991748 14422 raft_consensus.cc:2463] T f978e85cf659450ca0039368d58b49e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
W20250115 03:06:14.991905 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6df09ae8a979496abf73d384649b81b5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:14.991943 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 79579515fb7e45a8b16fd36dbc6a965a
I20250115 03:06:14.991936 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f978e85cf659450ca0039368d58b49e5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:14.991952 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 6d10731df02f4e9682b8de6962256635 (DEFAULT_TABLE table=default.Table_ad312d1f941c45b5a0dc5324967d5105 [id=79eb8c9694b74a31a4253b33d064e230]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.992161 14628 leader_election.cc:304] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:14.992456 15325 raft_consensus.cc:2798] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.992511 15325 raft_consensus.cc:491] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.992547 15325 raft_consensus.cc:3054] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:14.992667 14626 leader_election.cc:343] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.992836 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 79579515fb7e45a8b16fd36dbc6a965a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:14.993207 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 13f36a563f13403eab372deb10b67451 (DEFAULT_TABLE table=default.Table_eea85759fdf64d0bbfabd1e6cb16b125 [id=61f3625fed2b4a70b5331d6c6ac73f1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.993247 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet 79579515fb7e45a8b16fd36dbc6a965a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:14.993377 15325 raft_consensus.cc:513] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:14.993520 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 79579515fb7e45a8b16fd36dbc6a965a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79579515fb7e45a8b16fd36dbc6a965a already in progress: creating tablet
I20250115 03:06:14.993690 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13f36a563f13403eab372deb10b67451. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.993690 15325 leader_election.cc:290] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
W20250115 03:06:14.993811 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 79579515fb7e45a8b16fd36dbc6a965a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79579515fb7e45a8b16fd36dbc6a965a already in progress: creating tablet
I20250115 03:06:14.993954 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 79579515fb7e45a8b16fd36dbc6a965a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.994036 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f978e85cf659450ca0039368d58b49e5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:14.994129 14422 raft_consensus.cc:3054] T f978e85cf659450ca0039368d58b49e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.994396 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d10731df02f4e9682b8de6962256635. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.994745 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 82be772e05d443749914004aea8af300
I20250115 03:06:14.994938 14422 raft_consensus.cc:2463] T f978e85cf659450ca0039368d58b49e5 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:14.995115 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f978e85cf659450ca0039368d58b49e5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:14.995395 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet 82be772e05d443749914004aea8af300 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:14.995479 14626 leader_election.cc:343] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.995715 14628 leader_election.cc:304] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:14.995920 15401 raft_consensus.cc:491] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:14.995954 15325 raft_consensus.cc:2798] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
W20250115 03:06:14.995981 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 82be772e05d443749914004aea8af300 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82be772e05d443749914004aea8af300 already in progress: creating tablet
I20250115 03:06:14.996026 15325 raft_consensus.cc:695] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:14.996011 15401 raft_consensus.cc:513] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.996136 15325 consensus_queue.cc:237] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.996294 15401 leader_election.cc:290] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.996737 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb809ec664784386aa5f365bc27e0a1a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:14.997145 14356 leader_election.cc:343] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:14.997637 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb809ec664784386aa5f365bc27e0a1a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:14.997740 14689 raft_consensus.cc:2463] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:14.997982 14358 leader_election.cc:304] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
W20250115 03:06:14.998034 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 79579515fb7e45a8b16fd36dbc6a965a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 79579515fb7e45a8b16fd36dbc6a965a already in progress: creating tablet
I20250115 03:06:14.998143 15401 raft_consensus.cc:2798] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:14.998193 15401 raft_consensus.cc:491] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:14.998224 15401 raft_consensus.cc:3054] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:14.998276 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 82be772e05d443749914004aea8af300 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:14.998837 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 13f36a563f13403eab372deb10b67451 (DEFAULT_TABLE table=default.Table_eea85759fdf64d0bbfabd1e6cb16b125 [id=61f3625fed2b4a70b5331d6c6ac73f1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:14.998951 15401 raft_consensus.cc:513] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:14.999030 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13f36a563f13403eab372deb10b67451. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:14.999238 15401 leader_election.cc:290] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:14.999466 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb809ec664784386aa5f365bc27e0a1a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:14.999555 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb809ec664784386aa5f365bc27e0a1a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:14.999641 14689 raft_consensus.cc:3054] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:14.999742 14356 leader_election.cc:343] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.000375 14689 raft_consensus.cc:2463] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:15.000546 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 914d0f374564460480f93aff40fee17f
I20250115 03:06:15.000627 14358 leader_election.cc:304] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:15.000788 14164 catalog_manager.cc:6596] ReplaceTablet: tablet 914d0f374564460480f93aff40fee17f deleted and replaced by tablet bf6f61dc685a48bdb367278773bf1d90
I20250115 03:06:15.000811 15401 raft_consensus.cc:2798] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:15.000876 15401 raft_consensus.cc:695] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:15.001005 15401 consensus_queue.cc:237] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.001655 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 82be772e05d443749914004aea8af300 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:15.001739 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 82be772e05d443749914004aea8af300 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82be772e05d443749914004aea8af300 already in progress: creating tablet
I20250115 03:06:15.002020 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 13f36a563f13403eab372deb10b67451 (DEFAULT_TABLE table=default.Table_eea85759fdf64d0bbfabd1e6cb16b125 [id=61f3625fed2b4a70b5331d6c6ac73f1c]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:15.002051 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 82be772e05d443749914004aea8af300 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82be772e05d443749914004aea8af300 already in progress: creating tablet
I20250115 03:06:15.002149 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 2d3157859e684da98bb473d5799b24a6 (table default.Table_02c01532217e4a8d9092ecb9bed7c891 [id=265c248361624c3f9e5ece27144d8e04]). Expected version 1 got 0
I20250115 03:06:15.002197 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 914d0f374564460480f93aff40fee17f with delete_type TABLET_DATA_DELETED (replaced by tablet bf6f61dc685a48bdb367278773bf1d90) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.002338 14823 tablet_replica.cc:331] T 914d0f374564460480f93aff40fee17f P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.002452 14823 raft_consensus.cc:2238] T 914d0f374564460480f93aff40fee17f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.002529 14823 raft_consensus.cc:2267] T 914d0f374564460480f93aff40fee17f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.002555 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13f36a563f13403eab372deb10b67451. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.003185 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 2d3157859e684da98bb473d5799b24a6 (table default.Table_02c01532217e4a8d9092ecb9bed7c891 [id=265c248361624c3f9e5ece27144d8e04]). Expected version 1 got 0
I20250115 03:06:15.003392 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 481705dc9ae841659520a6c909879f9b (table default.Table_cab1959a159a46a6b86daad3647b89b3 [id=fe32f8f8243b4803876ed2b371e9e602]). Expected version 1 got 0
I20250115 03:06:15.004977 14823 ts_tablet_manager.cc:1905] T 914d0f374564460480f93aff40fee17f P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.005556 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 914d0f374564460480f93aff40fee17f with delete_type TABLET_DATA_DELETED (replaced by tablet bf6f61dc685a48bdb367278773bf1d90) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.006872 14823 ts_tablet_manager.cc:1918] T 914d0f374564460480f93aff40fee17f P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.006932 14823 log.cc:1198] T 914d0f374564460480f93aff40fee17f P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/914d0f374564460480f93aff40fee17f
I20250115 03:06:15.007107 14823 ts_tablet_manager.cc:1939] T 914d0f374564460480f93aff40fee17f P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.007290 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_79d7aa1c665d4695a2a17501157b3807"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.007560 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_79d7aa1c665d4695a2a17501157b3807 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.008221 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 914d0f374564460480f93aff40fee17f (table default.Table_b0ca39935d224741840950333b63bcfe [id=08996d7663c94ed1acb94b675012934c]) successfully deleted
W20250115 03:06:15.008664 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 914d0f374564460480f93aff40fee17f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 914d0f374564460480f93aff40fee17f already in progress: creating tablet
I20250115 03:06:15.009004 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 914d0f374564460480f93aff40fee17f with delete_type TABLET_DATA_DELETED (replaced by tablet bf6f61dc685a48bdb367278773bf1d90) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.009125 15453 raft_consensus.cc:491] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.009209 15453 raft_consensus.cc:513] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
W20250115 03:06:15.009495 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 914d0f374564460480f93aff40fee17f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 914d0f374564460480f93aff40fee17f already in progress: creating tablet
I20250115 03:06:15.009531 15453 leader_election.cc:290] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.009959 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dd115f83c544c4bba0d08fd82220952" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.010016 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dd115f83c544c4bba0d08fd82220952" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:15.010308 14628 leader_election.cc:343] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:15.010936 14626 leader_election.cc:343] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.011001 14626 leader_election.cc:304] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:15.011123 15453 raft_consensus.cc:2743] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:15.013923 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_65f0fd5511f64bfeb6539f9019a4873c" } modify_external_catalogs: true
I20250115 03:06:15.014016 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_65f0fd5511f64bfeb6539f9019a4873c" } modify_external_catalogs: true
I20250115 03:06:15.014066 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_3e704f1e03e34a2ca2377db61214c5a0 [id=0790b08615854d3688f940aba20b6520] alter complete (version 1)
I20250115 03:06:15.014443 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet 55f01d704b174b2ea108ea6f1df763df from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:15.017069 14165 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_bbfcc61694f44215aa5736bcc075ec0e [id=8d381e00c93343238da5478825775d04] alter complete (version 1)
I20250115 03:06:15.018046 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 37351252b608492e9acbc818b2754731 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.018102 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_544b91f835734146acd099765f743a9d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:15.018247 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 382116b1fb6a4f888ab1d55c29917ec8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:15.018399 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_544b91f835734146acd099765f743a9d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.019698 14819 tablet_replica.cc:331] T 37351252b608492e9acbc818b2754731 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.019835 14819 raft_consensus.cc:2238] T 37351252b608492e9acbc818b2754731 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.019914 14819 raft_consensus.cc:2267] T 37351252b608492e9acbc818b2754731 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.020004 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 080224910d554389b646e2c250fada2d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.020161 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet f54942abdfb748f2a939993a3c2d5db0 with delete_type TABLET_DATA_DELETED (replaced by tablet 382116b1fb6a4f888ab1d55c29917ec8) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.020365 14812 tablet_bootstrap.cc:492] T 53248628d9584f8b886166c1e8ec5c3c P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.020424 14812 ts_tablet_manager.cc:1397] T 53248628d9584f8b886166c1e8ec5c3c P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.063s user 0.001s sys 0.000s
I20250115 03:06:15.020474 14819 ts_tablet_manager.cc:1905] T 37351252b608492e9acbc818b2754731 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.020633 14820 tablet_replica.cc:331] T 080224910d554389b646e2c250fada2d P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.020750 14820 raft_consensus.cc:2238] T 080224910d554389b646e2c250fada2d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.020838 14820 raft_consensus.cc:2267] T 080224910d554389b646e2c250fada2d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.020802 14812 raft_consensus.cc:357] T 53248628d9584f8b886166c1e8ec5c3c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.020889 14812 raft_consensus.cc:383] T 53248628d9584f8b886166c1e8ec5c3c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.020917 14812 raft_consensus.cc:738] T 53248628d9584f8b886166c1e8ec5c3c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.021039 14812 consensus_queue.cc:260] T 53248628d9584f8b886166c1e8ec5c3c P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.021258 14820 ts_tablet_manager.cc:1905] T 080224910d554389b646e2c250fada2d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.021286 14812 ts_tablet_manager.cc:1428] T 53248628d9584f8b886166c1e8ec5c3c P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:15.021456 14812 tablet_bootstrap.cc:492] T 4cc6e937615b4aca8e9625d233452229 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.022418 14812 tablet_bootstrap.cc:654] T 4cc6e937615b4aca8e9625d233452229 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.023212 14820 ts_tablet_manager.cc:1918] T 080224910d554389b646e2c250fada2d P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.023269 14820 log.cc:1198] T 080224910d554389b646e2c250fada2d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/080224910d554389b646e2c250fada2d
I20250115 03:06:15.023452 14820 ts_tablet_manager.cc:1939] T 080224910d554389b646e2c250fada2d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.023933 14820 tablet_replica.cc:331] T f54942abdfb748f2a939993a3c2d5db0 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.023999 14819 ts_tablet_manager.cc:1918] T 37351252b608492e9acbc818b2754731 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.024034 14820 raft_consensus.cc:2238] T f54942abdfb748f2a939993a3c2d5db0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.024051 14819 log.cc:1198] T 37351252b608492e9acbc818b2754731 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/37351252b608492e9acbc818b2754731
I20250115 03:06:15.024116 14820 raft_consensus.cc:2267] T f54942abdfb748f2a939993a3c2d5db0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.024099 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 080224910d554389b646e2c250fada2d (table default.Table_929dc98c434d495da9e1e46de64f5ef9 [id=ade8ca55065d405b994a155664fa3eb1]) successfully deleted
I20250115 03:06:15.024230 14819 ts_tablet_manager.cc:1939] T 37351252b608492e9acbc818b2754731 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.024674 14820 ts_tablet_manager.cc:1905] T f54942abdfb748f2a939993a3c2d5db0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.024955 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 914d0f374564460480f93aff40fee17f with delete_type TABLET_DATA_DELETED (replaced by tablet bf6f61dc685a48bdb367278773bf1d90) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.025086 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f01d704b174b2ea108ea6f1df763df with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.025332 14823 tablet_replica.cc:331] T 55f01d704b174b2ea108ea6f1df763df P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.025446 14823 raft_consensus.cc:2238] T 55f01d704b174b2ea108ea6f1df763df P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.025480 15453 raft_consensus.cc:491] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.025532 14823 raft_consensus.cc:2267] T 55f01d704b174b2ea108ea6f1df763df P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.025549 15453 raft_consensus.cc:513] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.025849 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 382116b1fb6a4f888ab1d55c29917ec8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.026062 15453 leader_election.cc:290] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.026209 14819 tablet_replica.cc:331] T 382116b1fb6a4f888ab1d55c29917ec8 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.026371 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "382116b1fb6a4f888ab1d55c29917ec8" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:15.026485 14422 raft_consensus.cc:2463] T 382116b1fb6a4f888ab1d55c29917ec8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:15.026515 14820 ts_tablet_manager.cc:1918] T f54942abdfb748f2a939993a3c2d5db0 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.026600 14820 log.cc:1198] T f54942abdfb748f2a939993a3c2d5db0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/f54942abdfb748f2a939993a3c2d5db0
I20250115 03:06:15.026615 14819 raft_consensus.cc:2238] T 382116b1fb6a4f888ab1d55c29917ec8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.026702 14819 raft_consensus.cc:2267] T 382116b1fb6a4f888ab1d55c29917ec8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.026515 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "382116b1fb6a4f888ab1d55c29917ec8" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.026782 14820 ts_tablet_manager.cc:1939] T f54942abdfb748f2a939993a3c2d5db0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.027004 14628 leader_election.cc:304] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.027117 14819 ts_tablet_manager.cc:1905] T 382116b1fb6a4f888ab1d55c29917ec8 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.027139 15453 raft_consensus.cc:2798] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:15.027175 15453 raft_consensus.cc:491] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:15.027207 15453 raft_consensus.cc:3054] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:15.027256 14626 leader_election.cc:343] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.027629 14823 ts_tablet_manager.cc:1905] T 55f01d704b174b2ea108ea6f1df763df P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.028031 15453 raft_consensus.cc:513] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.028326 15453 leader_election.cc:290] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.029062 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "382116b1fb6a4f888ab1d55c29917ec8" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:15.029076 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "382116b1fb6a4f888ab1d55c29917ec8" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:15.029357 14626 leader_election.cc:343] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.029402 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet f54942abdfb748f2a939993a3c2d5db0 (table default.Table_5e6d1abc27d042dba8506da7b49a6ee3 [id=a5c946319e8a4be5ba4a01a40a9ad0b7]) successfully deleted
W20250115 03:06:15.029543 14158 catalog_manager.cc:4839] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 914d0f374564460480f93aff40fee17f because the tablet was not found. No further retry: Not found: Tablet not found: 914d0f374564460480f93aff40fee17f
I20250115 03:06:15.029649 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 37351252b608492e9acbc818b2754731 (table default.Table_e557ad0830f44aec955ce14c80b107e1 [id=0b8e801e3c8f40a6a5e226fbd824379e]) successfully deleted
I20250115 03:06:15.029703 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_7e53fa73287c43e3a9a0e65908b42593" } modify_external_catalogs: true
I20250115 03:06:15.029783 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_7e53fa73287c43e3a9a0e65908b42593" } modify_external_catalogs: true
W20250115 03:06:15.030072 14628 leader_election.cc:343] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: Tablet not RUNNING: STOPPED
I20250115 03:06:15.030130 14628 leader_election.cc:304] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:15.030277 14166 catalog_manager.cc:5526] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:15.030731 15453 raft_consensus.cc:2743] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250115 03:06:15.030973 14823 ts_tablet_manager.cc:1918] T 55f01d704b174b2ea108ea6f1df763df P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.031051 14823 log.cc:1198] T 55f01d704b174b2ea108ea6f1df763df P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/55f01d704b174b2ea108ea6f1df763df
I20250115 03:06:15.031108 14165 catalog_manager.cc:5526] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:15.031517 14823 ts_tablet_manager.cc:1939] T 55f01d704b174b2ea108ea6f1df763df P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.031695 14819 ts_tablet_manager.cc:1918] T 382116b1fb6a4f888ab1d55c29917ec8 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.031761 14819 log.cc:1198] T 382116b1fb6a4f888ab1d55c29917ec8 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/382116b1fb6a4f888ab1d55c29917ec8
I20250115 03:06:15.031948 14819 ts_tablet_manager.cc:1939] T 382116b1fb6a4f888ab1d55c29917ec8 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.031971 14394 tablet_service.cc:1467] Processing CreateTablet for tablet dcdb5282e41c43dbb235ae78465d19f9 (DEFAULT_TABLE table=default.Table_863155db5aee457a9ada887faf6deb85 [id=b298dce7ab664327893633f821ed4d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.032058 14823 tablet_replica.cc:331] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.032168 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcdb5282e41c43dbb235ae78465d19f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.032209 14823 raft_consensus.cc:2238] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.032290 14823 raft_consensus.cc:2267] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.033943 14527 tablet_service.cc:1467] Processing CreateTablet for tablet dcdb5282e41c43dbb235ae78465d19f9 (DEFAULT_TABLE table=default.Table_863155db5aee457a9ada887faf6deb85 [id=b298dce7ab664327893633f821ed4d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.034045 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 382116b1fb6a4f888ab1d55c29917ec8 (table default.Table_5e6d1abc27d042dba8506da7b49a6ee3 [id=a5c946319e8a4be5ba4a01a40a9ad0b7]) successfully deleted
I20250115 03:06:15.034204 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcdb5282e41c43dbb235ae78465d19f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.034214 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 55f01d704b174b2ea108ea6f1df763df (table default.Table_64d7a4f0d2694a65a7fb86e55e1255e0 [id=9dcc0154ed914304a15cf96049afaf52]) successfully deleted
I20250115 03:06:15.034524 14823 ts_tablet_manager.cc:1905] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.035838 14823 ts_tablet_manager.cc:1918] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.035902 14823 log.cc:1198] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/382116b1fb6a4f888ab1d55c29917ec8
I20250115 03:06:15.036080 14823 ts_tablet_manager.cc:1939] T 382116b1fb6a4f888ab1d55c29917ec8 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.036588 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 382116b1fb6a4f888ab1d55c29917ec8 (table default.Table_5e6d1abc27d042dba8506da7b49a6ee3 [id=a5c946319e8a4be5ba4a01a40a9ad0b7]) successfully deleted
I20250115 03:06:15.037336 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 2bffef32339a4aa6b2ef7fb5661d035c (DEFAULT_TABLE table=default.Table_e303383ddf3845828de6837219313014 [id=654dfdc4477b476f80e6af5b066713c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.037550 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bffef32339a4aa6b2ef7fb5661d035c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.037569 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 2bffef32339a4aa6b2ef7fb5661d035c (DEFAULT_TABLE table=default.Table_e303383ddf3845828de6837219313014 [id=654dfdc4477b476f80e6af5b066713c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.037750 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 341180618b4748eabc568888e2e6a7df
I20250115 03:06:15.037773 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bffef32339a4aa6b2ef7fb5661d035c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.038543 14663 tablet_service.cc:1467] Processing CreateTablet for tablet dcdb5282e41c43dbb235ae78465d19f9 (DEFAULT_TABLE table=default.Table_863155db5aee457a9ada887faf6deb85 [id=b298dce7ab664327893633f821ed4d89]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.038630 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 341180618b4748eabc568888e2e6a7df with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.038766 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcdb5282e41c43dbb235ae78465d19f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.038781 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 341180618b4748eabc568888e2e6a7df with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.039156 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 341180618b4748eabc568888e2e6a7df with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:15.039758 14158 catalog_manager.cc:4839] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 341180618b4748eabc568888e2e6a7df because the tablet was not found. No further retry: Not found: Tablet not found: 341180618b4748eabc568888e2e6a7df
W20250115 03:06:15.040302 14156 catalog_manager.cc:4839] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 341180618b4748eabc568888e2e6a7df because the tablet was not found. No further retry: Not found: Tablet not found: 341180618b4748eabc568888e2e6a7df
I20250115 03:06:15.040396 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 6110b6c4911d4932b4cac4d86598a5c6 (DEFAULT_TABLE table=default.Table_3e911fd952934f79bee759f184cbcce0 [id=0cbd9b74bdc84d9291bd5abee773d650]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.040628 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6110b6c4911d4932b4cac4d86598a5c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.041522 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 2bffef32339a4aa6b2ef7fb5661d035c (DEFAULT_TABLE table=default.Table_e303383ddf3845828de6837219313014 [id=654dfdc4477b476f80e6af5b066713c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.041780 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bffef32339a4aa6b2ef7fb5661d035c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.042366 14858 raft_consensus.cc:491] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.042455 14858 raft_consensus.cc:513] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.042747 14858 leader_election.cc:290] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:15.043151 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01aa12e3d8fa463cbaee1287e7b74dd4" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:15.043370 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01aa12e3d8fa463cbaee1287e7b74dd4" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:15.043481 14689 raft_consensus.cc:2388] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in current term 1: Already voted for candidate e8df7247ee9c43e0aa18216da2661f75 in this term.
I20250115 03:06:15.043860 14493 leader_election.cc:304] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:15.044059 14858 raft_consensus.cc:3054] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.044832 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet a1cbe29c08c7477e8944b7e108d46e7a
I20250115 03:06:15.044873 14858 raft_consensus.cc:2743] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:15.045018 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 341180618b4748eabc568888e2e6a7df because the tablet was not found. No further retry: Not found: Tablet not found: 341180618b4748eabc568888e2e6a7df
I20250115 03:06:15.045586 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet a1cbe29c08c7477e8944b7e108d46e7a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.045720 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 6110b6c4911d4932b4cac4d86598a5c6 (DEFAULT_TABLE table=default.Table_3e911fd952934f79bee759f184cbcce0 [id=0cbd9b74bdc84d9291bd5abee773d650]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.045915 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6110b6c4911d4932b4cac4d86598a5c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.046443 14812 tablet_bootstrap.cc:492] T 4cc6e937615b4aca8e9625d233452229 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.046528 14812 ts_tablet_manager.cc:1397] T 4cc6e937615b4aca8e9625d233452229 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.001s
I20250115 03:06:15.046934 14812 raft_consensus.cc:357] T 4cc6e937615b4aca8e9625d233452229 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.047027 14812 raft_consensus.cc:383] T 4cc6e937615b4aca8e9625d233452229 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.047039 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet a1cbe29c08c7477e8944b7e108d46e7a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.047061 14812 raft_consensus.cc:738] T 4cc6e937615b4aca8e9625d233452229 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.047192 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet a1cbe29c08c7477e8944b7e108d46e7a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.047170 14812 consensus_queue.cc:260] T 4cc6e937615b4aca8e9625d233452229 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.047421 14812 ts_tablet_manager.cc:1428] T 4cc6e937615b4aca8e9625d233452229 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:15.047624 14812 tablet_bootstrap.cc:492] T d4210020188d4f21aed9bb85f3b1f2a1 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.048638 14812 tablet_bootstrap.cc:654] T d4210020188d4f21aed9bb85f3b1f2a1 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:15.049026 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet a1cbe29c08c7477e8944b7e108d46e7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a1cbe29c08c7477e8944b7e108d46e7a already in progress: creating tablet
I20250115 03:06:15.049225 15401 consensus_queue.cc:1035] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.051384 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 53248628d9584f8b886166c1e8ec5c3c with delete_type TABLET_DATA_DELETED (replaced by tablet 7f67cbb929cd4ae589a14606aa168b46) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.051688 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 6110b6c4911d4932b4cac4d86598a5c6 (DEFAULT_TABLE table=default.Table_3e911fd952934f79bee759f184cbcce0 [id=0cbd9b74bdc84d9291bd5abee773d650]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.052166 14672 tablet_service.cc:1467] Processing CreateTablet for tablet bf6f61dc685a48bdb367278773bf1d90 (DEFAULT_TABLE table=default.Table_b0ca39935d224741840950333b63bcfe [id=08996d7663c94ed1acb94b675012934c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.051951 14394 tablet_service.cc:1467] Processing CreateTablet for tablet bf6f61dc685a48bdb367278773bf1d90 (DEFAULT_TABLE table=default.Table_b0ca39935d224741840950333b63bcfe [id=08996d7663c94ed1acb94b675012934c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.052366 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 58161a322a3d469c8444d353689bed65 (DEFAULT_TABLE table=default.Table_6df09ae8a979496abf73d384649b81b5 [id=e30f9e0f1aef45b7a1668a6d0f5f42bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.052654 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58161a322a3d469c8444d353689bed65. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:15.052790 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet a1cbe29c08c7477e8944b7e108d46e7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a1cbe29c08c7477e8944b7e108d46e7a already in progress: creating tablet
I20250115 03:06:15.052832 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6110b6c4911d4932b4cac4d86598a5c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.053969 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 341180618b4748eabc568888e2e6a7df (DEFAULT_TABLE table=default.Table_65f0fd5511f64bfeb6539f9019a4873c [id=7191c3890ae642558fe99561455d01e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.054102 14811 tablet_bootstrap.cc:492] T 4a56ae8a3e3e4f57b13e762cef21f753 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.054167 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 341180618b4748eabc568888e2e6a7df. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.054169 14811 ts_tablet_manager.cc:1397] T 4a56ae8a3e3e4f57b13e762cef21f753 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.096s user 0.001s sys 0.000s
I20250115 03:06:15.054551 14811 raft_consensus.cc:357] T 4a56ae8a3e3e4f57b13e762cef21f753 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:15.054586 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 58161a322a3d469c8444d353689bed65 (DEFAULT_TABLE table=default.Table_6df09ae8a979496abf73d384649b81b5 [id=e30f9e0f1aef45b7a1668a6d0f5f42bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.054636 14811 raft_consensus.cc:383] T 4a56ae8a3e3e4f57b13e762cef21f753 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.054688 14811 raft_consensus.cc:738] T 4a56ae8a3e3e4f57b13e762cef21f753 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.054786 14811 consensus_queue.cc:260] T 4a56ae8a3e3e4f57b13e762cef21f753 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:15.054823 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf6f61dc685a48bdb367278773bf1d90. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.055037 14811 ts_tablet_manager.cc:1428] T 4a56ae8a3e3e4f57b13e762cef21f753 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.055169 14811 tablet_bootstrap.cc:492] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.055917 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 341180618b4748eabc568888e2e6a7df (DEFAULT_TABLE table=default.Table_65f0fd5511f64bfeb6539f9019a4873c [id=7191c3890ae642558fe99561455d01e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.056002 14811 tablet_bootstrap.cc:654] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.057171 14660 tablet_service.cc:1467] Processing CreateTablet for tablet d0c3536533e94174bb1e3888a3458395 (DEFAULT_TABLE table=default.Table_1c52ca26880a48289f5916fabdb05ac4 [id=44c03f21942545319762494029cebc02]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.057426 14394 tablet_service.cc:1467] Processing CreateTablet for tablet d0c3536533e94174bb1e3888a3458395 (DEFAULT_TABLE table=default.Table_1c52ca26880a48289f5916fabdb05ac4 [id=44c03f21942545319762494029cebc02]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.057507 14396 tablet_service.cc:1467] Processing CreateTablet for tablet b6175f691fec45bf9853ee90959a1c98 (DEFAULT_TABLE table=default.Table_04eafaab3f094b2998237e6ca1440386 [id=a0cd4e60cd764c0b88b01041241edb3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.058817 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0c3536533e94174bb1e3888a3458395. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.059206 14861 consensus_queue.cc:1035] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.060827 14667 tablet_service.cc:1467] Processing CreateTablet for tablet b6175f691fec45bf9853ee90959a1c98 (DEFAULT_TABLE table=default.Table_04eafaab3f094b2998237e6ca1440386 [id=a0cd4e60cd764c0b88b01041241edb3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.061477 14669 tablet_service.cc:1467] Processing CreateTablet for tablet d895f75911b7456a88e7a5159db7d60b (DEFAULT_TABLE table=default.Table_39e0d37b84fc42c0807a4cbffb7bae0a [id=4b6ce1381c024498a78d91e346f488f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.061682 15153 raft_consensus.cc:491] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.061746 15153 raft_consensus.cc:513] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.062014 15153 leader_election.cc:290] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.062137 14394 tablet_service.cc:1467] Processing CreateTablet for tablet d895f75911b7456a88e7a5159db7d60b (DEFAULT_TABLE table=default.Table_39e0d37b84fc42c0807a4cbffb7bae0a [id=4b6ce1381c024498a78d91e346f488f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.062314 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481705dc9ae841659520a6c909879f9b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:15.062402 14422 raft_consensus.cc:2463] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:15.062584 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d895f75911b7456a88e7a5159db7d60b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.063107 14628 leader_election.cc:304] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.063252 15153 raft_consensus.cc:2798] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:15.063295 15153 raft_consensus.cc:491] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:15.063329 15153 raft_consensus.cc:3054] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.064086 15153 raft_consensus.cc:513] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.064337 15153 leader_election.cc:290] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.064953 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481705dc9ae841659520a6c909879f9b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:15.065042 14422 raft_consensus.cc:3054] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.065505 14809 tablet_bootstrap.cc:492] T 5dd115f83c544c4bba0d08fd82220952 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.065573 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6175f691fec45bf9853ee90959a1c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.065593 14809 ts_tablet_manager.cc:1397] T 5dd115f83c544c4bba0d08fd82220952 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.097s user 0.002s sys 0.000s
I20250115 03:06:15.065824 14422 raft_consensus.cc:2463] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:15.065959 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 38c9e64e1f5e4f2394afa876553357bf (DEFAULT_TABLE table=default.Table_bac546b42308426c9fd8a4ac15b547b3 [id=91731a4895d4478fad68b1bcbdc45d5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.066056 14809 raft_consensus.cc:357] T 5dd115f83c544c4bba0d08fd82220952 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:15.066147 14809 raft_consensus.cc:383] T 5dd115f83c544c4bba0d08fd82220952 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.066176 14809 raft_consensus.cc:738] T 5dd115f83c544c4bba0d08fd82220952 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.066212 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 38c9e64e1f5e4f2394afa876553357bf (DEFAULT_TABLE table=default.Table_bac546b42308426c9fd8a4ac15b547b3 [id=91731a4895d4478fad68b1bcbdc45d5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.066287 14809 consensus_queue.cc:260] T 5dd115f83c544c4bba0d08fd82220952 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:15.066391 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38c9e64e1f5e4f2394afa876553357bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.066555 14809 ts_tablet_manager.cc:1428] T 5dd115f83c544c4bba0d08fd82220952 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:15.066735 14809 tablet_bootstrap.cc:492] T 55f01d704b174b2ea108ea6f1df763df P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.067239 14628 leader_election.cc:304] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.067454 15153 raft_consensus.cc:2798] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:15.067526 15153 raft_consensus.cc:695] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:15.067579 14809 tablet_bootstrap.cc:654] T 55f01d704b174b2ea108ea6f1df763df P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.067631 15153 consensus_queue.cc:237] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.069615 15325 raft_consensus.cc:491] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.069687 15325 raft_consensus.cc:513] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.069953 15325 leader_election.cc:290] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.070284 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ef00eb188a4f628ecbf8fd8906b6b5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:15.070380 14422 raft_consensus.cc:2463] T 99ef00eb188a4f628ecbf8fd8906b6b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:15.070923 14628 leader_election.cc:304] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.071058 15325 raft_consensus.cc:2798] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:15.071097 15325 raft_consensus.cc:491] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:15.071137 15325 raft_consensus.cc:3054] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.071924 15325 raft_consensus.cc:513] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.072198 15325 leader_election.cc:290] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.072546 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ef00eb188a4f628ecbf8fd8906b6b5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:15.072634 14422 raft_consensus.cc:3054] T 99ef00eb188a4f628ecbf8fd8906b6b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.072777 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58161a322a3d469c8444d353689bed65. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.073429 14422 raft_consensus.cc:2463] T 99ef00eb188a4f628ecbf8fd8906b6b5 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:15.074421 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 341180618b4748eabc568888e2e6a7df. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.075021 14823 tablet_replica.cc:331] T 53248628d9584f8b886166c1e8ec5c3c P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.075057 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_fa2e1dcccf5e4b34971533a5d66118d7" } new_table_name: "default.Table_2e68c93eaa1647df9b269879bd4520ab" modify_external_catalogs: true
I20250115 03:06:15.075138 14823 raft_consensus.cc:2238] T 53248628d9584f8b886166c1e8ec5c3c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.075220 14823 raft_consensus.cc:2267] T 53248628d9584f8b886166c1e8ec5c3c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.076068 14628 leader_election.cc:304] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.076181 14823 ts_tablet_manager.cc:1905] T 53248628d9584f8b886166c1e8ec5c3c P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.076196 15325 raft_consensus.cc:2798] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:15.076273 15325 raft_consensus.cc:695] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:15.076375 15325 consensus_queue.cc:237] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.077903 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf6f61dc685a48bdb367278773bf1d90. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.077910 14165 catalog_manager.cc:5526] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:15.078073 14165 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 481705dc9ae841659520a6c909879f9b (table default.Table_cab1959a159a46a6b86daad3647b89b3 [id=fe32f8f8243b4803876ed2b371e9e602]). Expected version 1 got 0
I20250115 03:06:15.079834 14823 ts_tablet_manager.cc:1918] T 53248628d9584f8b886166c1e8ec5c3c P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20250115 03:06:15.079864 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6864f88bf6454ee980f624ec2f2d9fb6 consensus state has no leader
I20250115 03:06:15.079900 14823 log.cc:1198] T 53248628d9584f8b886166c1e8ec5c3c P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/53248628d9584f8b886166c1e8ec5c3c
W20250115 03:06:15.080230 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet a1cbe29c08c7477e8944b7e108d46e7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a1cbe29c08c7477e8944b7e108d46e7a already in progress: creating tablet
I20250115 03:06:15.080868 14527 tablet_service.cc:1467] Processing CreateTablet for tablet bf6f61dc685a48bdb367278773bf1d90 (DEFAULT_TABLE table=default.Table_b0ca39935d224741840950333b63bcfe [id=08996d7663c94ed1acb94b675012934c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.081064 14535 tablet_service.cc:1467] Processing CreateTablet for tablet d0c3536533e94174bb1e3888a3458395 (DEFAULT_TABLE table=default.Table_1c52ca26880a48289f5916fabdb05ac4 [id=44c03f21942545319762494029cebc02]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.081117 14528 tablet_service.cc:1467] Processing CreateTablet for tablet 341180618b4748eabc568888e2e6a7df (DEFAULT_TABLE table=default.Table_65f0fd5511f64bfeb6539f9019a4873c [id=7191c3890ae642558fe99561455d01e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.081125 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf6f61dc685a48bdb367278773bf1d90. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.081533 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0c3536533e94174bb1e3888a3458395. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.083149 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 341180618b4748eabc568888e2e6a7df. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.083714 14529 tablet_service.cc:1467] Processing CreateTablet for tablet d895f75911b7456a88e7a5159db7d60b (DEFAULT_TABLE table=default.Table_39e0d37b84fc42c0807a4cbffb7bae0a [id=4b6ce1381c024498a78d91e346f488f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.083768 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481705dc9ae841659520a6c909879f9b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.083927 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d895f75911b7456a88e7a5159db7d60b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.084049 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481705dc9ae841659520a6c909879f9b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:15.084312 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ef00eb188a4f628ecbf8fd8906b6b5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.084605 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99ef00eb188a4f628ecbf8fd8906b6b5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:15.085597 14535 tablet_service.cc:1467] Processing CreateTablet for tablet b6175f691fec45bf9853ee90959a1c98 (DEFAULT_TABLE table=default.Table_04eafaab3f094b2998237e6ca1440386 [id=a0cd4e60cd764c0b88b01041241edb3b]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:15.085769 14626 leader_election.cc:343] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.085808 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6175f691fec45bf9853ee90959a1c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.085893 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 38c9e64e1f5e4f2394afa876553357bf (DEFAULT_TABLE table=default.Table_bac546b42308426c9fd8a4ac15b547b3 [id=91731a4895d4478fad68b1bcbdc45d5d]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:15.085983 14626 leader_election.cc:343] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.086089 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38c9e64e1f5e4f2394afa876553357bf. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:15.086124 14626 leader_election.cc:343] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:15.086261 14626 leader_election.cc:343] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.086694 14823 ts_tablet_manager.cc:1939] T 53248628d9584f8b886166c1e8ec5c3c P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.087654 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 58161a322a3d469c8444d353689bed65 (DEFAULT_TABLE table=default.Table_6df09ae8a979496abf73d384649b81b5 [id=e30f9e0f1aef45b7a1668a6d0f5f42bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.088057 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58161a322a3d469c8444d353689bed65. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.089851 14861 raft_consensus.cc:491] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.089936 14861 raft_consensus.cc:513] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.090227 14861 leader_election.cc:290] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:15.090534 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d3157859e684da98bb473d5799b24a6" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:15.090641 14689 raft_consensus.cc:2463] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:15.090672 14166 catalog_manager.cc:5526] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:15.090909 14358 leader_election.cc:304] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.091053 14861 raft_consensus.cc:2798] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:15.091089 14861 raft_consensus.cc:491] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:15.091112 14861 raft_consensus.cc:3054] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.091513 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d3157859e684da98bb473d5799b24a6" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.091604 14557 raft_consensus.cc:2463] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:15.091821 14861 raft_consensus.cc:513] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.092106 14861 leader_election.cc:290] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:15.092393 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d3157859e684da98bb473d5799b24a6" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:15.092485 14689 raft_consensus.cc:3054] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.093201 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 858bbbb62fc249d5a915697c12362d0f (DEFAULT_TABLE table=default.Table_79d7aa1c665d4695a2a17501157b3807 [id=5cd92dd0835f4c9d88d917d8ff277bbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.093394 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 858bbbb62fc249d5a915697c12362d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.093793 14689 raft_consensus.cc:2463] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:15.094053 14537 tablet_service.cc:1467] Processing CreateTablet for tablet e63e854c49b34c1ab286f1f97fe443ba (DEFAULT_TABLE table=default.Table_544b91f835734146acd099765f743a9d [id=8460d1b93fbc4c3e987f38acc1d00818]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.094255 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e63e854c49b34c1ab286f1f97fe443ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.094853 14812 tablet_bootstrap.cc:492] T d4210020188d4f21aed9bb85f3b1f2a1 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.094930 14812 ts_tablet_manager.cc:1397] T d4210020188d4f21aed9bb85f3b1f2a1 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.047s user 0.001s sys 0.000s
I20250115 03:06:15.095360 14812 raft_consensus.cc:357] T d4210020188d4f21aed9bb85f3b1f2a1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.095448 14812 raft_consensus.cc:383] T d4210020188d4f21aed9bb85f3b1f2a1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.095479 14812 raft_consensus.cc:738] T d4210020188d4f21aed9bb85f3b1f2a1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.095520 14358 leader_election.cc:304] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.095583 14812 consensus_queue.cc:260] T d4210020188d4f21aed9bb85f3b1f2a1 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.095634 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_35d1dd748a6f4e79adced77fe7b2d08d" } new_table_name: "default.Table_a5cd96954b1c48c68a3bef815c47d8b4" modify_external_catalogs: true
I20250115 03:06:15.095669 14861 raft_consensus.cc:2798] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:15.095734 14861 raft_consensus.cc:695] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:15.095852 14812 ts_tablet_manager.cc:1428] T d4210020188d4f21aed9bb85f3b1f2a1 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.095840 14861 consensus_queue.cc:237] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.096005 14812 tablet_bootstrap.cc:492] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.096865 14812 tablet_bootstrap.cc:654] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.097074 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_68e894a7b2d7416a90c519daa7bf4fb9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.097373 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_68e894a7b2d7416a90c519daa7bf4fb9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.097831 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0c3536533e94174bb1e3888a3458395. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.098232 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6175f691fec45bf9853ee90959a1c98. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.099086 14812 tablet_bootstrap.cc:492] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.099293 14812 ts_tablet_manager.cc:1397] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:15.099855 14812 raft_consensus.cc:357] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.100185 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d895f75911b7456a88e7a5159db7d60b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.100225 14812 raft_consensus.cc:383] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.100271 14812 raft_consensus.cc:738] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.100509 14812 consensus_queue.cc:260] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.100636 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 858bbbb62fc249d5a915697c12362d0f (DEFAULT_TABLE table=default.Table_79d7aa1c665d4695a2a17501157b3807 [id=5cd92dd0835f4c9d88d917d8ff277bbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.100757 14812 ts_tablet_manager.cc:1428] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.100828 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 858bbbb62fc249d5a915697c12362d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.101081 14812 tablet_bootstrap.cc:492] T 47b08f6e7dec4c5a871a49c79421209e P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.101936 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38c9e64e1f5e4f2394afa876553357bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.102658 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 858bbbb62fc249d5a915697c12362d0f (DEFAULT_TABLE table=default.Table_79d7aa1c665d4695a2a17501157b3807 [id=5cd92dd0835f4c9d88d917d8ff277bbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.102887 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 858bbbb62fc249d5a915697c12362d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.103165 14812 tablet_bootstrap.cc:654] T 47b08f6e7dec4c5a871a49c79421209e P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.104663 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 53248628d9584f8b886166c1e8ec5c3c (table default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]) successfully deleted
I20250115 03:06:15.105399 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d3157859e684da98bb473d5799b24a6" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:15.105496 14557 raft_consensus.cc:3054] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.106230 14557 raft_consensus.cc:2463] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:15.107043 14663 tablet_service.cc:1467] Processing CreateTablet for tablet e63e854c49b34c1ab286f1f97fe443ba (DEFAULT_TABLE table=default.Table_544b91f835734146acd099765f743a9d [id=8460d1b93fbc4c3e987f38acc1d00818]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.107237 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e63e854c49b34c1ab286f1f97fe443ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.107362 14170 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5dd115f83c544c4bba0d08fd82220952 (table default.Table_6f23f5e22da844029c46908c62aeeb23 [id=1b7d3a80c5424e5692c2e6c556919342]). Expected version 3 got 0
W20250115 03:06:15.107618 14165 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5b7022db9edc4de8909e16769a3ea136 consensus state has no leader
I20250115 03:06:15.107802 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cc6e937615b4aca8e9625d233452229 with delete_type TABLET_DATA_DELETED (replaced by tablet acb43d614111477abb10def48a1dfb3b) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.109851 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5dd115f83c544c4bba0d08fd82220952 (table default.Table_6f23f5e22da844029c46908c62aeeb23 [id=1b7d3a80c5424e5692c2e6c556919342]). Expected version 3 got 0
I20250115 03:06:15.109964 14165 catalog_manager.cc:5526] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:15.110127 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 2d3157859e684da98bb473d5799b24a6 (table default.Table_02c01532217e4a8d9092ecb9bed7c891 [id=265c248361624c3f9e5ece27144d8e04]). Expected version 1 got 0
I20250115 03:06:15.110917 14823 tablet_replica.cc:331] T 4cc6e937615b4aca8e9625d233452229 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.111068 14823 raft_consensus.cc:2238] T 4cc6e937615b4aca8e9625d233452229 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.111183 14823 raft_consensus.cc:2267] T 4cc6e937615b4aca8e9625d233452229 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.112026 14823 ts_tablet_manager.cc:1905] T 4cc6e937615b4aca8e9625d233452229 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.113448 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_adff4ab8d6594baba1f08a15bf8486eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.113785 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_adff4ab8d6594baba1f08a15bf8486eb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.114622 14823 ts_tablet_manager.cc:1918] T 4cc6e937615b4aca8e9625d233452229 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.114737 14823 log.cc:1198] T 4cc6e937615b4aca8e9625d233452229 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/4cc6e937615b4aca8e9625d233452229
I20250115 03:06:15.115010 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_4f241c86102a4bbfb509a2d5d76953e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:15.115196 14823 ts_tablet_manager.cc:1939] T 4cc6e937615b4aca8e9625d233452229 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.115160 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8350ec7f96f743449d3122b98ad087d7" } new_table_name: "default.Table_098c9884c0494d4592f142b4a197f38c" modify_external_catalogs: true
W20250115 03:06:15.115283 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f241c86102a4bbfb509a2d5d76953e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.115890 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a56ae8a3e3e4f57b13e762cef21f753 with delete_type TABLET_DATA_DELETED (replaced by tablet 5dd115f83c544c4bba0d08fd82220952) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.115896 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_6b937e191b8f4365a3d2ef4a4885db50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:15.116078 14820 tablet_replica.cc:331] T 4a56ae8a3e3e4f57b13e762cef21f753 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
W20250115 03:06:15.116145 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6b937e191b8f4365a3d2ef4a4885db50 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.116186 14820 raft_consensus.cc:2238] T 4a56ae8a3e3e4f57b13e762cef21f753 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.116271 14820 raft_consensus.cc:2267] T 4a56ae8a3e3e4f57b13e762cef21f753 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.116426 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet d4210020188d4f21aed9bb85f3b1f2a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.116693 14823 tablet_replica.cc:331] T d4210020188d4f21aed9bb85f3b1f2a1 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.116715 14820 ts_tablet_manager.cc:1905] T 4a56ae8a3e3e4f57b13e762cef21f753 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.116792 14823 raft_consensus.cc:2238] T d4210020188d4f21aed9bb85f3b1f2a1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.116869 14823 raft_consensus.cc:2267] T d4210020188d4f21aed9bb85f3b1f2a1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.117815 14394 tablet_service.cc:1467] Processing CreateTablet for tablet e63e854c49b34c1ab286f1f97fe443ba (DEFAULT_TABLE table=default.Table_544b91f835734146acd099765f743a9d [id=8460d1b93fbc4c3e987f38acc1d00818]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.118031 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e63e854c49b34c1ab286f1f97fe443ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.118613 14820 ts_tablet_manager.cc:1918] T 4a56ae8a3e3e4f57b13e762cef21f753 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.118674 14820 log.cc:1198] T 4a56ae8a3e3e4f57b13e762cef21f753 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/4a56ae8a3e3e4f57b13e762cef21f753
I20250115 03:06:15.118862 14820 ts_tablet_manager.cc:1939] T 4a56ae8a3e3e4f57b13e762cef21f753 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.119174 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 4a56ae8a3e3e4f57b13e762cef21f753 (table default.Table_6f23f5e22da844029c46908c62aeeb23 [id=1b7d3a80c5424e5692c2e6c556919342]) successfully deleted
I20250115 03:06:15.120724 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_cab1959a159a46a6b86daad3647b89b3" } new_table_name: "default.Table_bae37f38b5e04374ad63b8f34977c09a" modify_external_catalogs: true
W20250115 03:06:15.121979 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet df44c5f5a9ae4d1aa7647dcbe2303a39 consensus state has no leader
I20250115 03:06:15.123068 14809 tablet_bootstrap.cc:492] T 55f01d704b174b2ea108ea6f1df763df P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.123152 14809 ts_tablet_manager.cc:1397] T 55f01d704b174b2ea108ea6f1df763df P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.056s user 0.002s sys 0.000s
I20250115 03:06:15.123165 14811 tablet_bootstrap.cc:492] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.123240 14811 ts_tablet_manager.cc:1397] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.068s user 0.001s sys 0.000s
I20250115 03:06:15.123557 14809 raft_consensus.cc:357] T 55f01d704b174b2ea108ea6f1df763df P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.123646 14809 raft_consensus.cc:383] T 55f01d704b174b2ea108ea6f1df763df P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.123674 14809 raft_consensus.cc:738] T 55f01d704b174b2ea108ea6f1df763df P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.123785 14809 consensus_queue.cc:260] T 55f01d704b174b2ea108ea6f1df763df P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.123785 14811 raft_consensus.cc:357] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.123876 14811 raft_consensus.cc:383] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.123906 14811 raft_consensus.cc:738] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.124009 14811 consensus_queue.cc:260] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.124064 14809 ts_tablet_manager.cc:1428] T 55f01d704b174b2ea108ea6f1df763df P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.124236 14809 tablet_bootstrap.cc:492] T d4210020188d4f21aed9bb85f3b1f2a1 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.124254 14811 ts_tablet_manager.cc:1428] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.124414 14811 tablet_bootstrap.cc:492] T 2dffa87697284dca8f07719f1c861bd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.125494 14811 tablet_bootstrap.cc:654] T 2dffa87697284dca8f07719f1c861bd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.125897 14809 tablet_bootstrap.cc:654] T d4210020188d4f21aed9bb85f3b1f2a1 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.127539 14165 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c (table default.Table_4bfe9fa1f7a143d684bc948b47deb8d8 [id=339b61af27cb486aaaadf8730ab21ce7]). Expected version 1 got 0
I20250115 03:06:15.128567 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 4cc6e937615b4aca8e9625d233452229 (table default.Table_85ea53c1d8b245cc94e6e81e157242aa [id=4d8e724e03dc4e2ab9a6e0281159aca8]) successfully deleted
I20250115 03:06:15.129439 14823 ts_tablet_manager.cc:1905] T d4210020188d4f21aed9bb85f3b1f2a1 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.129554 14812 tablet_bootstrap.cc:492] T 47b08f6e7dec4c5a871a49c79421209e P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.129670 14812 ts_tablet_manager.cc:1397] T 47b08f6e7dec4c5a871a49c79421209e P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.000s
I20250115 03:06:15.130120 14812 raft_consensus.cc:357] T 47b08f6e7dec4c5a871a49c79421209e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.130707 14812 raft_consensus.cc:383] T 47b08f6e7dec4c5a871a49c79421209e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.130803 14812 raft_consensus.cc:738] T 47b08f6e7dec4c5a871a49c79421209e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.130961 14162 master_service.cc:946] ReplaceTablet: received request to replace tablet 53248628d9584f8b886166c1e8ec5c3c from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:15.130955 14812 consensus_queue.cc:260] T 47b08f6e7dec4c5a871a49c79421209e P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.131255 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet 6d10731df02f4e9682b8de6962256635 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:15.132311 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f01d704b174b2ea108ea6f1df763df with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.132452 14819 tablet_replica.cc:331] T 55f01d704b174b2ea108ea6f1df763df P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.132558 14819 raft_consensus.cc:2238] T 55f01d704b174b2ea108ea6f1df763df P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.132637 14819 raft_consensus.cc:2267] T 55f01d704b174b2ea108ea6f1df763df P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.132771 14422 raft_consensus.cc:1270] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 5c0dae7aff4b44c699275b95ba639770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:15.133236 14823 ts_tablet_manager.cc:1918] T d4210020188d4f21aed9bb85f3b1f2a1 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.133347 14819 ts_tablet_manager.cc:1905] T 55f01d704b174b2ea108ea6f1df763df P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.135327 14819 ts_tablet_manager.cc:1918] T 55f01d704b174b2ea108ea6f1df763df P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.135402 14819 log.cc:1198] T 55f01d704b174b2ea108ea6f1df763df P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/55f01d704b174b2ea108ea6f1df763df
I20250115 03:06:15.133294 15153 consensus_queue.cc:1035] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.135592 14819 ts_tablet_manager.cc:1939] T 55f01d704b174b2ea108ea6f1df763df P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.136226 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 55f01d704b174b2ea108ea6f1df763df (table default.Table_64d7a4f0d2694a65a7fb86e55e1255e0 [id=9dcc0154ed914304a15cf96049afaf52]) successfully deleted
I20250115 03:06:15.136469 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 47b08f6e7dec4c5a871a49c79421209e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:15.136466 14626 consensus_peers.cc:487] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:15.137113 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 6a59a9be8fa94ec4bdb42c723ea0ca17 (DEFAULT_TABLE table=default.Table_68e894a7b2d7416a90c519daa7bf4fb9 [id=5b343e28259444528cbad21c3b048db5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.137158 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 6a59a9be8fa94ec4bdb42c723ea0ca17 (DEFAULT_TABLE table=default.Table_68e894a7b2d7416a90c519daa7bf4fb9 [id=5b343e28259444528cbad21c3b048db5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.137302 14823 log.cc:1198] T d4210020188d4f21aed9bb85f3b1f2a1 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/d4210020188d4f21aed9bb85f3b1f2a1
I20250115 03:06:15.137323 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a59a9be8fa94ec4bdb42c723ea0ca17. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.137358 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a59a9be8fa94ec4bdb42c723ea0ca17. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.137495 14823 ts_tablet_manager.cc:1939] T d4210020188d4f21aed9bb85f3b1f2a1 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.138152 14858 raft_consensus.cc:491] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.138219 14858 raft_consensus.cc:513] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.138497 14162 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 481705dc9ae841659520a6c909879f9b (table default.Table_bae37f38b5e04374ad63b8f34977c09a [id=fe32f8f8243b4803876ed2b371e9e602]). Expected version 2 got 0
I20250115 03:06:15.138532 14858 leader_election.cc:290] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:15.138909 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5504a9b8a4ea47e2b4cfd98c92e25d5c" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:15.139024 14689 raft_consensus.cc:2388] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in current term 1: Already voted for candidate e8df7247ee9c43e0aa18216da2661f75 in this term.
I20250115 03:06:15.139670 14812 ts_tablet_manager.cc:1428] T 47b08f6e7dec4c5a871a49c79421209e P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.010s user 0.000s sys 0.001s
I20250115 03:06:15.139644 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5504a9b8a4ea47e2b4cfd98c92e25d5c" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:15.139681 15401 consensus_queue.cc:1035] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.140082 14493 leader_election.cc:304] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:15.140275 14858 raft_consensus.cc:3054] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.141127 14858 raft_consensus.cc:2743] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:15.144122 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet d4210020188d4f21aed9bb85f3b1f2a1 (table default.Table_fb8c07c4a56044248f3fdef9c488f45c [id=eee0481c12b14e31b6b5c5b9c1688f59]) successfully deleted
W20250115 03:06:15.144254 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 47b08f6e7dec4c5a871a49c79421209e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 47b08f6e7dec4c5a871a49c79421209e already in progress: creating tablet
I20250115 03:06:15.144560 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 6a59a9be8fa94ec4bdb42c723ea0ca17 (DEFAULT_TABLE table=default.Table_68e894a7b2d7416a90c519daa7bf4fb9 [id=5b343e28259444528cbad21c3b048db5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.144575 14812 tablet_bootstrap.cc:492] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.144840 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a59a9be8fa94ec4bdb42c723ea0ca17. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.145697 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 6d10731df02f4e9682b8de6962256635
I20250115 03:06:15.145925 14169 catalog_manager.cc:6596] ReplaceTablet: tablet 6d10731df02f4e9682b8de6962256635 deleted and replaced by tablet 5780da67ce984297ba33d3c6901c9e3a
I20250115 03:06:15.146299 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d10731df02f4e9682b8de6962256635 with delete_type TABLET_DATA_DELETED (replaced by tablet 5780da67ce984297ba33d3c6901c9e3a) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:15.146667 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 6d10731df02f4e9682b8de6962256635 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d10731df02f4e9682b8de6962256635 already in progress: creating tablet
I20250115 03:06:15.146733 15274 tablet.cc:1721] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:15.147518 15421 tablet.cc:1721] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:15.148373 14812 tablet_bootstrap.cc:654] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.149349 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d10731df02f4e9682b8de6962256635 with delete_type TABLET_DATA_DELETED (replaced by tablet 5780da67ce984297ba33d3c6901c9e3a) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.149930 14809 tablet_bootstrap.cc:492] T d4210020188d4f21aed9bb85f3b1f2a1 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.149999 14809 ts_tablet_manager.cc:1397] T d4210020188d4f21aed9bb85f3b1f2a1 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.000s
I20250115 03:06:15.149991 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d10731df02f4e9682b8de6962256635 with delete_type TABLET_DATA_DELETED (replaced by tablet 5780da67ce984297ba33d3c6901c9e3a) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:15.150192 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 6d10731df02f4e9682b8de6962256635 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d10731df02f4e9682b8de6962256635 already in progress: creating tablet
I20250115 03:06:15.150375 14809 raft_consensus.cc:357] T d4210020188d4f21aed9bb85f3b1f2a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.150462 14809 raft_consensus.cc:383] T d4210020188d4f21aed9bb85f3b1f2a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.150491 14809 raft_consensus.cc:738] T d4210020188d4f21aed9bb85f3b1f2a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.150589 14809 consensus_queue.cc:260] T d4210020188d4f21aed9bb85f3b1f2a1 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.150763 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 47b08f6e7dec4c5a871a49c79421209e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.150832 14809 ts_tablet_manager.cc:1428] T d4210020188d4f21aed9bb85f3b1f2a1 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.150985 14809 tablet_bootstrap.cc:492] T 53248628d9584f8b886166c1e8ec5c3c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.152089 14809 tablet_bootstrap.cc:654] T 53248628d9584f8b886166c1e8ec5c3c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:15.153959 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 6d10731df02f4e9682b8de6962256635 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6d10731df02f4e9682b8de6962256635 already in progress: creating tablet
I20250115 03:06:15.154702 14162 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_bae37f38b5e04374ad63b8f34977c09a [id=fe32f8f8243b4803876ed2b371e9e602] alter complete (version 2)
I20250115 03:06:15.155315 14823 tablet_replica.cc:331] T 47b08f6e7dec4c5a871a49c79421209e P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.155474 14823 raft_consensus.cc:2238] T 47b08f6e7dec4c5a871a49c79421209e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.155620 14823 raft_consensus.cc:2267] T 47b08f6e7dec4c5a871a49c79421209e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.156726 14812 tablet_bootstrap.cc:492] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.156970 14812 ts_tablet_manager.cc:1397] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.001s
I20250115 03:06:15.158023 15390 consensus_queue.cc:1035] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.158229 14823 ts_tablet_manager.cc:1905] T 47b08f6e7dec4c5a871a49c79421209e P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.160130 14823 ts_tablet_manager.cc:1918] T 47b08f6e7dec4c5a871a49c79421209e P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.160200 14823 log.cc:1198] T 47b08f6e7dec4c5a871a49c79421209e P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/47b08f6e7dec4c5a871a49c79421209e
I20250115 03:06:15.160501 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet d4210020188d4f21aed9bb85f3b1f2a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.161381 14819 tablet_replica.cc:331] T d4210020188d4f21aed9bb85f3b1f2a1 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.161486 14819 raft_consensus.cc:2238] T d4210020188d4f21aed9bb85f3b1f2a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.161296 14823 ts_tablet_manager.cc:1939] T 47b08f6e7dec4c5a871a49c79421209e P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.161567 14819 raft_consensus.cc:2267] T d4210020188d4f21aed9bb85f3b1f2a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.161921 14819 ts_tablet_manager.cc:1905] T d4210020188d4f21aed9bb85f3b1f2a1 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.162056 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 47b08f6e7dec4c5a871a49c79421209e (table default.Table_f4d365d35e3c43e79d3ac20e438e7aa5 [id=835a8f488a7c42af850632b1286f9509]) successfully deleted
I20250115 03:06:15.162662 14812 raft_consensus.cc:357] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.162750 14812 raft_consensus.cc:383] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.162775 14812 raft_consensus.cc:738] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.162864 14812 consensus_queue.cc:260] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.163075 14812 ts_tablet_manager.cc:1428] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.006s user 0.000s sys 0.000s
I20250115 03:06:15.163810 14819 ts_tablet_manager.cc:1918] T d4210020188d4f21aed9bb85f3b1f2a1 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.163863 14819 log.cc:1198] T d4210020188d4f21aed9bb85f3b1f2a1 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/d4210020188d4f21aed9bb85f3b1f2a1
I20250115 03:06:15.164032 14819 ts_tablet_manager.cc:1939] T d4210020188d4f21aed9bb85f3b1f2a1 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.164837 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 93fa227c0de74a3fa86778641fdcda3d (DEFAULT_TABLE table=default.Table_4f241c86102a4bbfb509a2d5d76953e9 [id=03b92c08d0b0473d83090c58e450c84b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.165098 14812 tablet_bootstrap.cc:492] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.165153 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet d4210020188d4f21aed9bb85f3b1f2a1 (table default.Table_fb8c07c4a56044248f3fdef9c488f45c [id=eee0481c12b14e31b6b5c5b9c1688f59]) successfully deleted
I20250115 03:06:15.165294 14396 tablet_service.cc:1467] Processing CreateTablet for tablet edf17b41198b4f8fbe5db22c58aa6bb2 (DEFAULT_TABLE table=default.Table_6b937e191b8f4365a3d2ef4a4885db50 [id=55b469a4aaf44fd5aa149e852cf08a57]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.165491 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edf17b41198b4f8fbe5db22c58aa6bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.165711 14809 tablet_bootstrap.cc:492] T 53248628d9584f8b886166c1e8ec5c3c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.165781 14809 ts_tablet_manager.cc:1397] T 53248628d9584f8b886166c1e8ec5c3c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20250115 03:06:15.165920 14812 tablet_bootstrap.cc:654] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.166184 14809 raft_consensus.cc:357] T 53248628d9584f8b886166c1e8ec5c3c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.166278 14809 raft_consensus.cc:383] T 53248628d9584f8b886166c1e8ec5c3c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.166306 14809 raft_consensus.cc:738] T 53248628d9584f8b886166c1e8ec5c3c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.166402 14809 consensus_queue.cc:260] T 53248628d9584f8b886166c1e8ec5c3c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.166640 14809 ts_tablet_manager.cc:1428] T 53248628d9584f8b886166c1e8ec5c3c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.166791 14809 tablet_bootstrap.cc:492] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.167311 14663 tablet_service.cc:1467] Processing CreateTablet for tablet edf17b41198b4f8fbe5db22c58aa6bb2 (DEFAULT_TABLE table=default.Table_6b937e191b8f4365a3d2ef4a4885db50 [id=55b469a4aaf44fd5aa149e852cf08a57]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.167589 14812 tablet_bootstrap.cc:492] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.167627 14809 tablet_bootstrap.cc:654] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.167660 14812 ts_tablet_manager.cc:1397] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:15.168180 14812 raft_consensus.cc:357] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.168350 14812 raft_consensus.cc:383] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.168552 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ec8ee0d483474271a3b369a2bcde593d" } new_table_name: "default.Table_4e74d34a621f4b1898241f62a20ffc10" modify_external_catalogs: true
I20250115 03:06:15.168597 14858 raft_consensus.cc:491] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.168704 14858 raft_consensus.cc:513] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:15.168910 14811 tablet_bootstrap.cc:492] T 2dffa87697284dca8f07719f1c861bd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.169026 14811 ts_tablet_manager.cc:1397] T 2dffa87697284dca8f07719f1c861bd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.045s user 0.001s sys 0.000s
I20250115 03:06:15.169092 14858 leader_election.cc:290] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:15.169251 14812 raft_consensus.cc:738] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.169440 14811 raft_consensus.cc:357] T 2dffa87697284dca8f07719f1c861bd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.169551 14811 raft_consensus.cc:383] T 2dffa87697284dca8f07719f1c861bd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.169586 14811 raft_consensus.cc:738] T 2dffa87697284dca8f07719f1c861bd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.169625 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c76ab180b26487896f4360efe47b2c7" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:15.169690 14811 consensus_queue.cc:260] T 2dffa87697284dca8f07719f1c861bd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.169546 14812 consensus_queue.cc:260] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.169970 14811 ts_tablet_manager.cc:1428] T 2dffa87697284dca8f07719f1c861bd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:15.170125 14811 tablet_bootstrap.cc:492] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.170182 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93fa227c0de74a3fa86778641fdcda3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.170948 14811 tablet_bootstrap.cc:654] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.171569 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 56c600fe18504e05bab2bcde3c89861d (DEFAULT_TABLE table=default.Table_adff4ab8d6594baba1f08a15bf8486eb [id=3f4310bb3227447ba6fcd011f05d6924]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.172217 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56c600fe18504e05bab2bcde3c89861d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.172593 14812 ts_tablet_manager.cc:1428] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20250115 03:06:15.173327 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c76ab180b26487896f4360efe47b2c7" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:15.173453 14689 raft_consensus.cc:2388] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in current term 1: Already voted for candidate e8df7247ee9c43e0aa18216da2661f75 in this term.
I20250115 03:06:15.174712 14493 leader_election.cc:304] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:15.174934 14858 raft_consensus.cc:3054] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.175043 14557 raft_consensus.cc:1270] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:15.175941 14858 raft_consensus.cc:2743] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:15.176111 14809 tablet_bootstrap.cc:492] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.176182 14809 ts_tablet_manager.cc:1397] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:15.176239 14885 consensus_queue.cc:1035] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.176298 14812 tablet_bootstrap.cc:492] T 89d92f59c7424dfc95427329200455e7 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.176573 14809 raft_consensus.cc:357] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.176666 14809 raft_consensus.cc:383] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.176703 14809 raft_consensus.cc:738] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.176816 14809 consensus_queue.cc:260] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.177197 14809 ts_tablet_manager.cc:1428] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.177408 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edf17b41198b4f8fbe5db22c58aa6bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.177516 14689 raft_consensus.cc:1270] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:15.177788 14809 tablet_bootstrap.cc:492] T b5e7672ebc4442369156d12695261269 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.177920 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_271f46df55f14ffd9b7bb48352620543" } modify_external_catalogs: true
I20250115 03:06:15.178025 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_271f46df55f14ffd9b7bb48352620543" } modify_external_catalogs: true
I20250115 03:06:15.178267 14885 consensus_queue.cc:1035] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.179106 14812 tablet_bootstrap.cc:654] T 89d92f59c7424dfc95427329200455e7 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.179142 14809 tablet_bootstrap.cc:654] T b5e7672ebc4442369156d12695261269 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.179440 14527 tablet_service.cc:1467] Processing CreateTablet for tablet edf17b41198b4f8fbe5db22c58aa6bb2 (DEFAULT_TABLE table=default.Table_6b937e191b8f4365a3d2ef4a4885db50 [id=55b469a4aaf44fd5aa149e852cf08a57]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.179507 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 56c600fe18504e05bab2bcde3c89861d (DEFAULT_TABLE table=default.Table_adff4ab8d6594baba1f08a15bf8486eb [id=3f4310bb3227447ba6fcd011f05d6924]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.179678 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edf17b41198b4f8fbe5db22c58aa6bb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.179811 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 93fa227c0de74a3fa86778641fdcda3d (DEFAULT_TABLE table=default.Table_4f241c86102a4bbfb509a2d5d76953e9 [id=03b92c08d0b0473d83090c58e450c84b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.180017 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93fa227c0de74a3fa86778641fdcda3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.182160 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ff9ccf42b3cc41f0b9b3059fab4c6768"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:15.182214 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_394e29bc18164e5885974223ad73b323"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.182538 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_394e29bc18164e5885974223ad73b323 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:15.182535 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff9ccf42b3cc41f0b9b3059fab4c6768 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.183485 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2773b577ce554a45834f200dca2c2987"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.183806 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2773b577ce554a45834f200dca2c2987 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.184625 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56c600fe18504e05bab2bcde3c89861d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.189271 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 56c600fe18504e05bab2bcde3c89861d (DEFAULT_TABLE table=default.Table_adff4ab8d6594baba1f08a15bf8486eb [id=3f4310bb3227447ba6fcd011f05d6924]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.189503 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56c600fe18504e05bab2bcde3c89861d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.190831 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 53248628d9584f8b886166c1e8ec5c3c with delete_type TABLET_DATA_DELETED (replaced by tablet 7f67cbb929cd4ae589a14606aa168b46) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.190933 14811 tablet_bootstrap.cc:492] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.191006 14811 ts_tablet_manager.cc:1397] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.001s
I20250115 03:06:15.191015 15453 raft_consensus.cc:491] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.191083 15453 raft_consensus.cc:513] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.191421 15453 leader_election.cc:290] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:15.191871 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 3cbc91ad00b54ede804c3629bcfe7fe6
W20250115 03:06:15.192243 14162 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f8b90f854a5e4221a24986f8c71ca300 consensus state has no leader
I20250115 03:06:15.192265 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d4b8d4ed9f44112b83c6ff94aee2206" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:15.192685 14628 leader_election.cc:343] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.192977 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cbc91ad00b54ede804c3629bcfe7fe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.193703 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cbc91ad00b54ede804c3629bcfe7fe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.194222 14166 master_service.cc:946] ReplaceTablet: received request to replace tablet 99ef00eb188a4f628ecbf8fd8906b6b5 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:15.194921 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3e911fd952934f79bee759f184cbcce0" } modify_external_catalogs: true
I20250115 03:06:15.195012 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3e911fd952934f79bee759f184cbcce0" } modify_external_catalogs: true
W20250115 03:06:15.195837 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 3cbc91ad00b54ede804c3629bcfe7fe6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3cbc91ad00b54ede804c3629bcfe7fe6 already in progress: creating tablet
I20250115 03:06:15.196154 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6f23f5e22da844029c46908c62aeeb23" } modify_external_catalogs: true
I20250115 03:06:15.196254 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6f23f5e22da844029c46908c62aeeb23" } modify_external_catalogs: true
I20250115 03:06:15.196959 14858 consensus_queue.cc:1035] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.197451 14811 raft_consensus.cc:357] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.197777 14811 raft_consensus.cc:383] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.197817 14811 raft_consensus.cc:738] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.198519 14811 consensus_queue.cc:260] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.199707 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 99ef00eb188a4f628ecbf8fd8906b6b5
I20250115 03:06:15.199985 14166 catalog_manager.cc:6596] ReplaceTablet: tablet 99ef00eb188a4f628ecbf8fd8906b6b5 deleted and replaced by tablet 6784e353df554bb2aa10b667c8b488da
I20250115 03:06:15.200067 14170 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 2d3157859e684da98bb473d5799b24a6 (table default.Table_02c01532217e4a8d9092ecb9bed7c891 [id=265c248361624c3f9e5ece27144d8e04]). Expected version 1 got 0
I20250115 03:06:15.200171 14170 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet ccf6b58c55d747da89c1c218a70ce09f (table default.Table_cbfbacbaeee4488aba1a8be104138404 [id=e441d6f5acea4837accbfb8e4856f82d]). Expected version 1 got 0
I20250115 03:06:15.200534 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d4b8d4ed9f44112b83c6ff94aee2206" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.200717 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dffa87697284dca8f07719f1c861bd4 with delete_type TABLET_DATA_DELETED (replaced by tablet a1cbe29c08c7477e8944b7e108d46e7a) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.201206 14820 tablet_replica.cc:331] T 2dffa87697284dca8f07719f1c861bd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.201349 14820 raft_consensus.cc:2238] T 2dffa87697284dca8f07719f1c861bd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.201448 14820 raft_consensus.cc:2267] T 2dffa87697284dca8f07719f1c861bd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.201592 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 99ef00eb188a4f628ecbf8fd8906b6b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 6784e353df554bb2aa10b667c8b488da) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.201957 14885 consensus_queue.cc:1035] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.203855 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 6110b6c4911d4932b4cac4d86598a5c6
I20250115 03:06:15.204183 14819 tablet_replica.cc:331] T 53248628d9584f8b886166c1e8ec5c3c P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.204249 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 99ef00eb188a4f628ecbf8fd8906b6b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 6784e353df554bb2aa10b667c8b488da) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.204305 14819 raft_consensus.cc:2238] T 53248628d9584f8b886166c1e8ec5c3c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.204388 14819 raft_consensus.cc:2267] T 53248628d9584f8b886166c1e8ec5c3c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.204598 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 5dd115f83c544c4bba0d08fd82220952
I20250115 03:06:15.205166 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 99ef00eb188a4f628ecbf8fd8906b6b5 with delete_type TABLET_DATA_DELETED (replaced by tablet 6784e353df554bb2aa10b667c8b488da) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.205377 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dd115f83c544c4bba0d08fd82220952 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.205778 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 6110b6c4911d4932b4cac4d86598a5c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.206032 14811 ts_tablet_manager.cc:1428] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.015s user 0.001s sys 0.000s
I20250115 03:06:15.206136 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 6110b6c4911d4932b4cac4d86598a5c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.206151 14164 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 2d3157859e684da98bb473d5799b24a6 (table default.Table_02c01532217e4a8d9092ecb9bed7c891 [id=265c248361624c3f9e5ece27144d8e04]). Expected version 1 got 0
I20250115 03:06:15.206350 14811 tablet_bootstrap.cc:492] T bb809ec664784386aa5f365bc27e0a1a P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.206343 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dd115f83c544c4bba0d08fd82220952 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.206624 14823 tablet_replica.cc:331] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.206756 14823 raft_consensus.cc:2238] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:15.206878 14823 pending_rounds.cc:62] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 1 pending ops.
I20250115 03:06:15.206907 14823 pending_rounds.cc:69] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384896313905152 op_type: NO_OP noop_request { }
I20250115 03:06:15.206950 14823 raft_consensus.cc:2883] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:15.206993 14823 raft_consensus.cc:2267] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.207638 14811 tablet_bootstrap.cc:654] T bb809ec664784386aa5f365bc27e0a1a P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.207669 14823 ts_tablet_manager.cc:1905] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.207857 15390 consensus_queue.cc:1035] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.208241 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_cf7f3fbd11804e94848a7c5357169a86"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.209328 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cf7f3fbd11804e94848a7c5357169a86 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.210048 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 6110b6c4911d4932b4cac4d86598a5c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.210281 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cbc91ad00b54ede804c3629bcfe7fe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.210479 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dd115f83c544c4bba0d08fd82220952 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.210677 14819 ts_tablet_manager.cc:1905] T 53248628d9584f8b886166c1e8ec5c3c P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.212714 14819 ts_tablet_manager.cc:1918] T 53248628d9584f8b886166c1e8ec5c3c P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.212781 14819 log.cc:1198] T 53248628d9584f8b886166c1e8ec5c3c P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/53248628d9584f8b886166c1e8ec5c3c
W20250115 03:06:15.213469 14626 leader_election.cc:343] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.213551 14626 leader_election.cc:304] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:15.213685 15453 raft_consensus.cc:2743] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:15.214017 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cc6e937615b4aca8e9625d233452229 with delete_type TABLET_DATA_DELETED (replaced by tablet acb43d614111477abb10def48a1dfb3b) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.215621 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 93fa227c0de74a3fa86778641fdcda3d (DEFAULT_TABLE table=default.Table_4f241c86102a4bbfb509a2d5d76953e9 [id=03b92c08d0b0473d83090c58e450c84b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.215838 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93fa227c0de74a3fa86778641fdcda3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.217597 14819 ts_tablet_manager.cc:1939] T 53248628d9584f8b886166c1e8ec5c3c P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.218209 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 53248628d9584f8b886166c1e8ec5c3c (table default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]) successfully deleted
W20250115 03:06:15.218658 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 3cbc91ad00b54ede804c3629bcfe7fe6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3cbc91ad00b54ede804c3629bcfe7fe6 already in progress: creating tablet
I20250115 03:06:15.218953 14819 tablet_replica.cc:331] T 99ef00eb188a4f628ecbf8fd8906b6b5 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.219046 14819 raft_consensus.cc:2238] T 99ef00eb188a4f628ecbf8fd8906b6b5 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.218986 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_eec42b37d6de49ccad194b13edd928e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:15.219105 14819 raft_consensus.cc:2267] T 99ef00eb188a4f628ecbf8fd8906b6b5 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:15.219266 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eec42b37d6de49ccad194b13edd928e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.219429 14819 ts_tablet_manager.cc:1905] T 99ef00eb188a4f628ecbf8fd8906b6b5 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.220853 14819 ts_tablet_manager.cc:1918] T 99ef00eb188a4f628ecbf8fd8906b6b5 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.220908 14819 log.cc:1198] T 99ef00eb188a4f628ecbf8fd8906b6b5 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/99ef00eb188a4f628ecbf8fd8906b6b5
I20250115 03:06:15.221632 14823 ts_tablet_manager.cc:1918] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.221747 14823 log.cc:1198] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/99ef00eb188a4f628ecbf8fd8906b6b5
I20250115 03:06:15.222011 14823 ts_tablet_manager.cc:1939] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.222249 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 5780da67ce984297ba33d3c6901c9e3a (DEFAULT_TABLE table=default.Table_ad312d1f941c45b5a0dc5324967d5105 [id=79eb8c9694b74a31a4253b33d064e230]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.222484 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5780da67ce984297ba33d3c6901c9e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.222759 14823 tablet_replica.cc:331] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.222750 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 99ef00eb188a4f628ecbf8fd8906b6b5 (table default.Table_6faeecc09aca49888fb4ccd071352ab7 [id=4d6cb6f5c7354beebee1b3fbe4384d54]) successfully deleted
I20250115 03:06:15.222932 14823 raft_consensus.cc:2238] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
W20250115 03:06:15.222997 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 6110b6c4911d4932b4cac4d86598a5c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6110b6c4911d4932b4cac4d86598a5c6 already in progress: creating tablet
I20250115 03:06:15.223111 14823 raft_consensus.cc:2267] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.223562 14823 ts_tablet_manager.cc:1905] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.223973 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 5780da67ce984297ba33d3c6901c9e3a (DEFAULT_TABLE table=default.Table_ad312d1f941c45b5a0dc5324967d5105 [id=79eb8c9694b74a31a4253b33d064e230]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.224536 14819 ts_tablet_manager.cc:1939] T 99ef00eb188a4f628ecbf8fd8906b6b5 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.224514 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 5780da67ce984297ba33d3c6901c9e3a (DEFAULT_TABLE table=default.Table_ad312d1f941c45b5a0dc5324967d5105 [id=79eb8c9694b74a31a4253b33d064e230]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.224743 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5780da67ce984297ba33d3c6901c9e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.225108 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 99ef00eb188a4f628ecbf8fd8906b6b5 (table default.Table_6faeecc09aca49888fb4ccd071352ab7 [id=4d6cb6f5c7354beebee1b3fbe4384d54]) successfully deleted
I20250115 03:06:15.225282 14819 tablet_replica.cc:331] T 5dd115f83c544c4bba0d08fd82220952 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.225378 14819 raft_consensus.cc:2238] T 5dd115f83c544c4bba0d08fd82220952 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.225375 14823 ts_tablet_manager.cc:1918] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.225435 14823 log.cc:1198] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/5dd115f83c544c4bba0d08fd82220952
I20250115 03:06:15.225463 14819 raft_consensus.cc:2267] T 5dd115f83c544c4bba0d08fd82220952 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.225787 14823 ts_tablet_manager.cc:1939] T 5dd115f83c544c4bba0d08fd82220952 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.225824 14819 ts_tablet_manager.cc:1905] T 5dd115f83c544c4bba0d08fd82220952 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.226323 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 5dd115f83c544c4bba0d08fd82220952 (table default.Table_6f23f5e22da844029c46908c62aeeb23 [id=1b7d3a80c5424e5692c2e6c556919342]) successfully deleted
I20250115 03:06:15.226732 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5780da67ce984297ba33d3c6901c9e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.226869 15390 raft_consensus.cc:491] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.226953 15390 raft_consensus.cc:513] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.227272 15390 leader_election.cc:290] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:15.227805 14358 leader_election.cc:343] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 4cc6e937615b4aca8e9625d233452229
I20250115 03:06:15.228084 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cc6e937615b4aca8e9625d233452229" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.228375 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet 506198162e0b4e24924c408542707c4c from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:15.228866 14820 ts_tablet_manager.cc:1905] T 2dffa87697284dca8f07719f1c861bd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:15.228885 14356 leader_election.cc:343] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.228968 14356 leader_election.cc:304] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:15.230727 14820 ts_tablet_manager.cc:1918] T 2dffa87697284dca8f07719f1c861bd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.230791 14820 log.cc:1198] T 2dffa87697284dca8f07719f1c861bd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/2dffa87697284dca8f07719f1c861bd4
I20250115 03:06:15.230918 15401 raft_consensus.cc:2743] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:15.230960 14820 ts_tablet_manager.cc:1939] T 2dffa87697284dca8f07719f1c861bd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.231389 14811 tablet_bootstrap.cc:492] T bb809ec664784386aa5f365bc27e0a1a P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.231424 14820 tablet_replica.cc:331] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.231465 14811 ts_tablet_manager.cc:1397] T bb809ec664784386aa5f365bc27e0a1a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.000s
I20250115 03:06:15.231530 14820 raft_consensus.cc:2238] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.231519 15421 tablet.cc:1721] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:15.231600 14820 raft_consensus.cc:2267] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.231988 14811 raft_consensus.cc:357] T bb809ec664784386aa5f365bc27e0a1a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.232079 14811 raft_consensus.cc:383] T bb809ec664784386aa5f365bc27e0a1a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.232110 14811 raft_consensus.cc:738] T bb809ec664784386aa5f365bc27e0a1a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.232102 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b175cba7965344979476c3cb6d789eca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:15.232239 14811 consensus_queue.cc:260] T bb809ec664784386aa5f365bc27e0a1a P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:15.232411 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b175cba7965344979476c3cb6d789eca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.232864 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 2dffa87697284dca8f07719f1c861bd4 (table default.Table_7e53fa73287c43e3a9a0e65908b42593 [id=1f983a10a8704e22b8a2b96ef4e06fbc]) successfully deleted
I20250115 03:06:15.233026 15274 tablet.cc:1721] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:15.234040 14820 ts_tablet_manager.cc:1905] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.234656 14811 ts_tablet_manager.cc:1428] T bb809ec664784386aa5f365bc27e0a1a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:15.235193 14811 tablet_bootstrap.cc:492] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.232113 14170 master_service.cc:946] ReplaceTablet: received request to replace tablet bda0e0238da246feb36450f40dd061e5 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:15.238202 14811 tablet_bootstrap.cc:654] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.239933 15390 consensus_queue.cc:1035] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.243577 14820 ts_tablet_manager.cc:1918] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.243650 14820 log.cc:1198] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/99ef00eb188a4f628ecbf8fd8906b6b5
I20250115 03:06:15.243840 14820 ts_tablet_manager.cc:1939] T 99ef00eb188a4f628ecbf8fd8906b6b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.245124 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_75c071e03ba44140bce71e920f31715d" } modify_external_catalogs: true
I20250115 03:06:15.245218 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_75c071e03ba44140bce71e920f31715d" } modify_external_catalogs: true
I20250115 03:06:15.246362 14163 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_02c01532217e4a8d9092ecb9bed7c891 [id=265c248361624c3f9e5ece27144d8e04] alter complete (version 1)
I20250115 03:06:15.246842 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet bda0e0238da246feb36450f40dd061e5
I20250115 03:06:15.247076 14170 catalog_manager.cc:6596] ReplaceTablet: tablet bda0e0238da246feb36450f40dd061e5 deleted and replaced by tablet 30499e99ab7d4138a376dea40b768c32
I20250115 03:06:15.247200 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_77f0e54f01874d84ac9475bf9d819946" } modify_external_catalogs: true
I20250115 03:06:15.247308 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_77f0e54f01874d84ac9475bf9d819946" } modify_external_catalogs: true
I20250115 03:06:15.248302 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet bda0e0238da246feb36450f40dd061e5 with delete_type TABLET_DATA_DELETED (replaced by tablet 30499e99ab7d4138a376dea40b768c32) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.249083 14823 tablet_replica.cc:331] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.249249 14823 raft_consensus.cc:2238] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.250152 14823 raft_consensus.cc:2267] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.250682 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet bda0e0238da246feb36450f40dd061e5 with delete_type TABLET_DATA_DELETED (replaced by tablet 30499e99ab7d4138a376dea40b768c32) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.250905 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 99ef00eb188a4f628ecbf8fd8906b6b5 (table default.Table_6faeecc09aca49888fb4ccd071352ab7 [id=4d6cb6f5c7354beebee1b3fbe4384d54]) successfully deleted
I20250115 03:06:15.251340 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet bda0e0238da246feb36450f40dd061e5 with delete_type TABLET_DATA_DELETED (replaced by tablet 30499e99ab7d4138a376dea40b768c32) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:15.251367 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 6110b6c4911d4932b4cac4d86598a5c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6110b6c4911d4932b4cac4d86598a5c6 already in progress: creating tablet
W20250115 03:06:15.251523 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 3cbc91ad00b54ede804c3629bcfe7fe6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3cbc91ad00b54ede804c3629bcfe7fe6 already in progress: creating tablet
I20250115 03:06:15.252173 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 46e799d700db4803b79ed46273b99df6
I20250115 03:06:15.252722 14809 tablet_bootstrap.cc:492] T b5e7672ebc4442369156d12695261269 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.252816 14809 ts_tablet_manager.cc:1397] T b5e7672ebc4442369156d12695261269 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.075s user 0.002s sys 0.000s
I20250115 03:06:15.253305 14812 tablet_bootstrap.cc:492] T 89d92f59c7424dfc95427329200455e7 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.253374 14812 ts_tablet_manager.cc:1397] T 89d92f59c7424dfc95427329200455e7 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.077s user 0.001s sys 0.000s
I20250115 03:06:15.253613 14809 raft_consensus.cc:357] T b5e7672ebc4442369156d12695261269 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.253705 14809 raft_consensus.cc:383] T b5e7672ebc4442369156d12695261269 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.253746 14809 raft_consensus.cc:738] T b5e7672ebc4442369156d12695261269 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.253809 14812 raft_consensus.cc:357] T 89d92f59c7424dfc95427329200455e7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.253854 14809 consensus_queue.cc:260] T b5e7672ebc4442369156d12695261269 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.253894 14812 raft_consensus.cc:383] T 89d92f59c7424dfc95427329200455e7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.253923 14812 raft_consensus.cc:738] T 89d92f59c7424dfc95427329200455e7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.254022 14812 consensus_queue.cc:260] T 89d92f59c7424dfc95427329200455e7 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.254128 14809 ts_tablet_manager.cc:1428] T b5e7672ebc4442369156d12695261269 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:15.254259 14812 ts_tablet_manager.cc:1428] T 89d92f59c7424dfc95427329200455e7 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.254274 14809 tablet_bootstrap.cc:492] T 914d0f374564460480f93aff40fee17f P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.254407 14812 tablet_bootstrap.cc:492] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.254738 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 46e799d700db4803b79ed46273b99df6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.255159 14809 tablet_bootstrap.cc:654] T 914d0f374564460480f93aff40fee17f P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.255215 14812 tablet_bootstrap.cc:654] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.255908 14820 tablet_replica.cc:331] T bda0e0238da246feb36450f40dd061e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.256016 14820 raft_consensus.cc:2238] T bda0e0238da246feb36450f40dd061e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.256129 14820 raft_consensus.cc:2267] T bda0e0238da246feb36450f40dd061e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.256304 15426 tablet.cc:1721] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:15.257650 14687 raft_consensus.cc:3054] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.258999 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet 46e799d700db4803b79ed46273b99df6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.259110 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 46e799d700db4803b79ed46273b99df6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.259320 15401 consensus_queue.cc:1035] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:15.260236 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 5dd115f83c544c4bba0d08fd82220952 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5dd115f83c544c4bba0d08fd82220952 already in progress: creating tablet
I20250115 03:06:15.263763 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet 89d92f59c7424dfc95427329200455e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.267084 14811 tablet_bootstrap.cc:492] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.267349 14817 consensus_queue.cc:1035] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.268151 14811 ts_tablet_manager.cc:1397] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
I20250115 03:06:15.268625 14811 raft_consensus.cc:357] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.269204 14811 raft_consensus.cc:383] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.269291 14811 raft_consensus.cc:738] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.269440 14811 consensus_queue.cc:260] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.269868 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 58130f2b27b745d08c0002ac96af6d72 (DEFAULT_TABLE table=default.Table_ff9ccf42b3cc41f0b9b3059fab4c6768 [id=6aa86d8861174b9091f29b0cfd39fa55]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.269915 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 6784e353df554bb2aa10b667c8b488da (DEFAULT_TABLE table=default.Table_6faeecc09aca49888fb4ccd071352ab7 [id=4d6cb6f5c7354beebee1b3fbe4384d54]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.270100 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58130f2b27b745d08c0002ac96af6d72. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.270205 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6784e353df554bb2aa10b667c8b488da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.270438 14823 ts_tablet_manager.cc:1905] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.272197 14396 tablet_service.cc:1467] Processing CreateTablet for tablet a1b8b5b7fe03496ea2639dbc9c920974 (DEFAULT_TABLE table=default.Table_2773b577ce554a45834f200dca2c2987 [id=8005446109b64adabae59e8612c35f58]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.272393 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1b8b5b7fe03496ea2639dbc9c920974. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.273499 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 506198162e0b4e24924c408542707c4c
I20250115 03:06:15.272359 14823 ts_tablet_manager.cc:1918] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.273762 14169 catalog_manager.cc:6596] ReplaceTablet: tablet 506198162e0b4e24924c408542707c4c deleted and replaced by tablet a01da82adcca493da401b914fee1bfd0
I20250115 03:06:15.272698 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 8efd181c42eb42628a9f36fad4fa8d64 (DEFAULT_TABLE table=default.Table_394e29bc18164e5885974223ad73b323 [id=65877002fc9047ef85aac66d44a7eb8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.273869 14823 log.cc:1198] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/bda0e0238da246feb36450f40dd061e5
I20250115 03:06:15.274032 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8efd181c42eb42628a9f36fad4fa8d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.274260 14823 ts_tablet_manager.cc:1939] T bda0e0238da246feb36450f40dd061e5 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.274838 14823 tablet_replica.cc:331] T 46e799d700db4803b79ed46273b99df6 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.274947 14823 raft_consensus.cc:2238] T 46e799d700db4803b79ed46273b99df6 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.275031 14823 raft_consensus.cc:2267] T 46e799d700db4803b79ed46273b99df6 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.275826 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 8efd181c42eb42628a9f36fad4fa8d64 (DEFAULT_TABLE table=default.Table_394e29bc18164e5885974223ad73b323 [id=65877002fc9047ef85aac66d44a7eb8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.276021 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8efd181c42eb42628a9f36fad4fa8d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.276031 14823 ts_tablet_manager.cc:1905] T 46e799d700db4803b79ed46273b99df6 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.276381 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet bda0e0238da246feb36450f40dd061e5 (table default.Table_884bdba667784d59bc4fa65b5b437fcb [id=111fc67094164d3f99965c246ae12b24]) successfully deleted
I20250115 03:06:15.272754 14667 tablet_service.cc:1467] Processing CreateTablet for tablet a1b8b5b7fe03496ea2639dbc9c920974 (DEFAULT_TABLE table=default.Table_2773b577ce554a45834f200dca2c2987 [id=8005446109b64adabae59e8612c35f58]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.277102 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1b8b5b7fe03496ea2639dbc9c920974. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.277101 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 506198162e0b4e24924c408542707c4c with delete_type TABLET_DATA_DELETED (replaced by tablet a01da82adcca493da401b914fee1bfd0) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.277172 14537 tablet_service.cc:1467] Processing CreateTablet for tablet a1b8b5b7fe03496ea2639dbc9c920974 (DEFAULT_TABLE table=default.Table_2773b577ce554a45834f200dca2c2987 [id=8005446109b64adabae59e8612c35f58]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.277386 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1b8b5b7fe03496ea2639dbc9c920974. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.277693 14823 ts_tablet_manager.cc:1918] T 46e799d700db4803b79ed46273b99df6 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.277758 14823 log.cc:1198] T 46e799d700db4803b79ed46273b99df6 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/46e799d700db4803b79ed46273b99df6
I20250115 03:06:15.277969 14823 ts_tablet_manager.cc:1939] T 46e799d700db4803b79ed46273b99df6 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.278596 14823 tablet_replica.cc:331] T 89d92f59c7424dfc95427329200455e7 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.278591 14811 ts_tablet_manager.cc:1428] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
I20250115 03:06:15.278692 14823 raft_consensus.cc:2238] T 89d92f59c7424dfc95427329200455e7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.278746 14811 tablet_bootstrap.cc:492] T 382116b1fb6a4f888ab1d55c29917ec8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.278874 14823 raft_consensus.cc:2267] T 89d92f59c7424dfc95427329200455e7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.278963 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 8efd181c42eb42628a9f36fad4fa8d64 (DEFAULT_TABLE table=default.Table_394e29bc18164e5885974223ad73b323 [id=65877002fc9047ef85aac66d44a7eb8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.279167 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8efd181c42eb42628a9f36fad4fa8d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.279261 14823 ts_tablet_manager.cc:1905] T 89d92f59c7424dfc95427329200455e7 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.279455 15401 raft_consensus.cc:491] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.279520 15401 raft_consensus.cc:513] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.279683 14811 tablet_bootstrap.cc:654] T 382116b1fb6a4f888ab1d55c29917ec8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.279829 15401 leader_election.cc:290] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:15.280052 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b22aa85ce34b38b0d6a5e2d38f0a1c" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.280149 14554 raft_consensus.cc:2463] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:15.280226 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b22aa85ce34b38b0d6a5e2d38f0a1c" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:15.280315 14687 raft_consensus.cc:2463] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:15.280406 14356 leader_election.cc:304] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.280872 15390 raft_consensus.cc:2798] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:15.280941 15390 raft_consensus.cc:491] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:15.280984 15390 raft_consensus.cc:3054] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.272867 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 6784e353df554bb2aa10b667c8b488da (DEFAULT_TABLE table=default.Table_6faeecc09aca49888fb4ccd071352ab7 [id=4d6cb6f5c7354beebee1b3fbe4384d54]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.281198 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6784e353df554bb2aa10b667c8b488da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.281219 14823 ts_tablet_manager.cc:1918] T 89d92f59c7424dfc95427329200455e7 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.281276 14823 log.cc:1198] T 89d92f59c7424dfc95427329200455e7 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/89d92f59c7424dfc95427329200455e7
I20250115 03:06:15.281508 14823 ts_tablet_manager.cc:1939] T 89d92f59c7424dfc95427329200455e7 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.281816 15390 raft_consensus.cc:513] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.282150 15390 leader_election.cc:290] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:15.282354 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b22aa85ce34b38b0d6a5e2d38f0a1c" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:15.282469 14811 tablet_bootstrap.cc:492] T 382116b1fb6a4f888ab1d55c29917ec8 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.282454 14554 raft_consensus.cc:3054] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.282531 14811 ts_tablet_manager.cc:1397] T 382116b1fb6a4f888ab1d55c29917ec8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:15.282657 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b22aa85ce34b38b0d6a5e2d38f0a1c" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:15.282754 14687 raft_consensus.cc:3054] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.272892 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 58130f2b27b745d08c0002ac96af6d72 (DEFAULT_TABLE table=default.Table_ff9ccf42b3cc41f0b9b3059fab4c6768 [id=6aa86d8861174b9091f29b0cfd39fa55]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.283293 14554 raft_consensus.cc:2463] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:15.283406 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58130f2b27b745d08c0002ac96af6d72. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.283543 14687 raft_consensus.cc:2463] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:15.283605 14356 leader_election.cc:304] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.284060 14811 raft_consensus.cc:357] T 382116b1fb6a4f888ab1d55c29917ec8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.284145 14811 raft_consensus.cc:383] T 382116b1fb6a4f888ab1d55c29917ec8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.284173 14811 raft_consensus.cc:738] T 382116b1fb6a4f888ab1d55c29917ec8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.284240 15390 raft_consensus.cc:2798] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:15.284269 14811 consensus_queue.cc:260] T 382116b1fb6a4f888ab1d55c29917ec8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.284327 15390 raft_consensus.cc:695] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:15.284510 14811 ts_tablet_manager.cc:1428] T 382116b1fb6a4f888ab1d55c29917ec8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20250115 03:06:15.284466 15390 consensus_queue.cc:237] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.284646 14811 tablet_bootstrap.cc:492] T 37351252b608492e9acbc818b2754731 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.285661 14811 tablet_bootstrap.cc:654] T 37351252b608492e9acbc818b2754731 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.285724 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_884bdba667784d59bc4fa65b5b437fcb" } new_table_name: "default.Table_75cd7b01fc534652975eb593912b88bc" modify_external_catalogs: true
I20250115 03:06:15.286234 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet 506198162e0b4e24924c408542707c4c with delete_type TABLET_DATA_DELETED (replaced by tablet a01da82adcca493da401b914fee1bfd0) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.288286 14162 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet a01da82adcca493da401b914fee1bfd0
I20250115 03:06:15.289635 15390 consensus_queue.cc:1035] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.290654 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 46e799d700db4803b79ed46273b99df6 (table default.Table_77f0e54f01874d84ac9475bf9d819946 [id=f5e54ac99a344e05a0be4c913472dd37]) successfully deleted
W20250115 03:06:15.290692 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 30499e99ab7d4138a376dea40b768c32 has no consensus state
I20250115 03:06:15.290865 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 89d92f59c7424dfc95427329200455e7 (table default.Table_236770d2959743108285eae378066e45 [id=fec89f498f5046839fe491caa3367a36]) successfully deleted
W20250115 03:06:15.292330 14493 consensus_peers.cc:487] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 -> Peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Couldn't send request to peer 5c0dae7aff4b44c699275b95ba639770. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 46e799d700db4803b79ed46273b99df6. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:15.293116 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 58130f2b27b745d08c0002ac96af6d72 (DEFAULT_TABLE table=default.Table_ff9ccf42b3cc41f0b9b3059fab4c6768 [id=6aa86d8861174b9091f29b0cfd39fa55]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.293330 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58130f2b27b745d08c0002ac96af6d72. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.294327 14820 ts_tablet_manager.cc:1905] T bda0e0238da246feb36450f40dd061e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.296022 14169 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c (table default.Table_4bfe9fa1f7a143d684bc948b47deb8d8 [id=339b61af27cb486aaaadf8730ab21ce7]). Expected version 1 got 0
I20250115 03:06:15.296644 14820 ts_tablet_manager.cc:1918] T bda0e0238da246feb36450f40dd061e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.296716 14820 log.cc:1198] T bda0e0238da246feb36450f40dd061e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/bda0e0238da246feb36450f40dd061e5
I20250115 03:06:15.296877 15390 consensus_queue.cc:1035] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.298511 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 506198162e0b4e24924c408542707c4c with delete_type TABLET_DATA_DELETED (replaced by tablet a01da82adcca493da401b914fee1bfd0) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.298770 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 6784e353df554bb2aa10b667c8b488da (DEFAULT_TABLE table=default.Table_6faeecc09aca49888fb4ccd071352ab7 [id=4d6cb6f5c7354beebee1b3fbe4384d54]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.298970 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6784e353df554bb2aa10b667c8b488da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.301610 14170 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c (table default.Table_4bfe9fa1f7a143d684bc948b47deb8d8 [id=339b61af27cb486aaaadf8730ab21ce7]). Expected version 1 got 0
I20250115 03:06:15.305116 14169 catalog_manager.cc:5526] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:15.305337 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c (table default.Table_4bfe9fa1f7a143d684bc948b47deb8d8 [id=339b61af27cb486aaaadf8730ab21ce7]). Expected version 1 got 0
I20250115 03:06:15.306484 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 3007a0506ce14c6c89c600b98a81b168 (DEFAULT_TABLE table=default.Table_cf7f3fbd11804e94848a7c5357169a86 [id=d64515775f264a419768662b8ce0a854]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.306617 14820 ts_tablet_manager.cc:1939] T bda0e0238da246feb36450f40dd061e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.306715 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3007a0506ce14c6c89c600b98a81b168. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.307005 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 3007a0506ce14c6c89c600b98a81b168 (DEFAULT_TABLE table=default.Table_cf7f3fbd11804e94848a7c5357169a86 [id=d64515775f264a419768662b8ce0a854]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.307406 14820 tablet_replica.cc:331] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.307593 14820 raft_consensus.cc:2238] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:15.307684 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3007a0506ce14c6c89c600b98a81b168. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.307803 14820 raft_consensus.cc:2267] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.308108 14820 ts_tablet_manager.cc:1905] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.308586 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_c832613845e141f2b165d1ef2a601af3" } new_table_name: "default.Table_58e0901d0ec94ef9859515f126825621" modify_external_catalogs: true
I20250115 03:06:15.309043 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet bda0e0238da246feb36450f40dd061e5 (table default.Table_75cd7b01fc534652975eb593912b88bc [id=111fc67094164d3f99965c246ae12b24]) successfully deleted
I20250115 03:06:15.309512 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 3007a0506ce14c6c89c600b98a81b168 (DEFAULT_TABLE table=default.Table_cf7f3fbd11804e94848a7c5357169a86 [id=d64515775f264a419768662b8ce0a854]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.309792 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3007a0506ce14c6c89c600b98a81b168. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.309800 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c (table default.Table_4bfe9fa1f7a143d684bc948b47deb8d8 [id=339b61af27cb486aaaadf8730ab21ce7]). Expected version 1 got 0
I20250115 03:06:15.311576 14819 ts_tablet_manager.cc:1918] T 5dd115f83c544c4bba0d08fd82220952 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.311657 14819 log.cc:1198] T 5dd115f83c544c4bba0d08fd82220952 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/5dd115f83c544c4bba0d08fd82220952
I20250115 03:06:15.312278 14666 tablet_service.cc:1467] Processing CreateTablet for tablet aa7754f577af4ef6b6fca41e61e5965c (DEFAULT_TABLE table=default.Table_b175cba7965344979476c3cb6d789eca [id=8ca6a98158114c7bb6bf49940d846cc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.312485 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa7754f577af4ef6b6fca41e61e5965c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.312685 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_1a535f4e7104456598818042ce64476f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.312999 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1a535f4e7104456598818042ce64476f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.313109 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 382116b1fb6a4f888ab1d55c29917ec8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.313249 14537 tablet_service.cc:1467] Processing CreateTablet for tablet aa7754f577af4ef6b6fca41e61e5965c (DEFAULT_TABLE table=default.Table_b175cba7965344979476c3cb6d789eca [id=8ca6a98158114c7bb6bf49940d846cc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.313457 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa7754f577af4ef6b6fca41e61e5965c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.316532 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 5d27310f8433411a85bf88d05a81bcf0 (DEFAULT_TABLE table=default.Table_eec42b37d6de49ccad194b13edd928e3 [id=5d934a52e6704fa68f7c685ae3c72081]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.316756 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d27310f8433411a85bf88d05a81bcf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.317067 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 30499e99ab7d4138a376dea40b768c32 (DEFAULT_TABLE table=default.Table_75cd7b01fc534652975eb593912b88bc [id=111fc67094164d3f99965c246ae12b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.317268 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30499e99ab7d4138a376dea40b768c32. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.317545 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 5d27310f8433411a85bf88d05a81bcf0 (DEFAULT_TABLE table=default.Table_eec42b37d6de49ccad194b13edd928e3 [id=5d934a52e6704fa68f7c685ae3c72081]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.317732 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d27310f8433411a85bf88d05a81bcf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.318759 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 30499e99ab7d4138a376dea40b768c32 (DEFAULT_TABLE table=default.Table_75cd7b01fc534652975eb593912b88bc [id=111fc67094164d3f99965c246ae12b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.318946 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30499e99ab7d4138a376dea40b768c32. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.319871 15274 tablet.cc:1721] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:15.320490 15426 tablet.cc:1721] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:15.323239 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_faa8557664e04a1c88f455c0f098a9d8" } new_table_name: "default.Table_365313892043408c92be8460c31d958e" modify_external_catalogs: true
I20250115 03:06:15.323312 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_4fcf8d20f3984b37910ea7164b26cca8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.323599 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4fcf8d20f3984b37910ea7164b26cca8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.311906 14819 ts_tablet_manager.cc:1939] T 5dd115f83c544c4bba0d08fd82220952 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.326711 14819 tablet_replica.cc:331] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.326835 14819 raft_consensus.cc:2238] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.326925 14819 raft_consensus.cc:2267] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.326916 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_807e7c2704994c25aa03b1267a20f794"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.327190 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_807e7c2704994c25aa03b1267a20f794 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.328054 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_93108a53124a492599eda22f3633780a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.328321 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93108a53124a492599eda22f3633780a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.328330 14820 ts_tablet_manager.cc:1918] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.328390 14820 log.cc:1198] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/46e799d700db4803b79ed46273b99df6
I20250115 03:06:15.328636 14820 ts_tablet_manager.cc:1939] T 46e799d700db4803b79ed46273b99df6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.316856 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 30499e99ab7d4138a376dea40b768c32 (DEFAULT_TABLE table=default.Table_75cd7b01fc534652975eb593912b88bc [id=111fc67094164d3f99965c246ae12b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.329773 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 46e799d700db4803b79ed46273b99df6 (table default.Table_77f0e54f01874d84ac9475bf9d819946 [id=f5e54ac99a344e05a0be4c913472dd37]) successfully deleted
I20250115 03:06:15.329911 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30499e99ab7d4138a376dea40b768c32. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:15.330005 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 745344db22da4102bad39c2f121548a1 consensus state has no leader
I20250115 03:06:15.330655 14819 ts_tablet_manager.cc:1905] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.331079 14809 tablet_bootstrap.cc:492] T 914d0f374564460480f93aff40fee17f P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.331163 14809 ts_tablet_manager.cc:1397] T 914d0f374564460480f93aff40fee17f P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.077s user 0.000s sys 0.002s
I20250115 03:06:15.331673 14809 raft_consensus.cc:357] T 914d0f374564460480f93aff40fee17f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.331768 14809 raft_consensus.cc:383] T 914d0f374564460480f93aff40fee17f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.313666 14396 tablet_service.cc:1467] Processing CreateTablet for tablet aa7754f577af4ef6b6fca41e61e5965c (DEFAULT_TABLE table=default.Table_b175cba7965344979476c3cb6d789eca [id=8ca6a98158114c7bb6bf49940d846cc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.331765 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_53c9a73ae8ef459c987964efac552c96" } new_table_name: "default.Table_3d196756e234426799a1de651bc112f2" modify_external_catalogs: true
I20250115 03:06:15.331800 14809 raft_consensus.cc:738] T 914d0f374564460480f93aff40fee17f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.331943 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa7754f577af4ef6b6fca41e61e5965c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.331907 14809 consensus_queue.cc:260] T 914d0f374564460480f93aff40fee17f P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.332144 14809 ts_tablet_manager.cc:1428] T 914d0f374564460480f93aff40fee17f P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:15.332314 14809 tablet_bootstrap.cc:492] T 47b08f6e7dec4c5a871a49c79421209e P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.333241 14809 tablet_bootstrap.cc:654] T 47b08f6e7dec4c5a871a49c79421209e P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.334445 14819 ts_tablet_manager.cc:1918] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.334511 14819 log.cc:1198] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/4cc6e937615b4aca8e9625d233452229
I20250115 03:06:15.334694 14819 ts_tablet_manager.cc:1939] T 4cc6e937615b4aca8e9625d233452229 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.334940 14169 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_58e0901d0ec94ef9859515f126825621 [id=50c73c26b4ed47a4961348db433b7dd4] alter complete (version 3)
I20250115 03:06:15.335693 14819 tablet_replica.cc:331] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.335872 14819 raft_consensus.cc:2238] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:15.336094 14819 raft_consensus.cc:2267] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.336882 14812 tablet_bootstrap.cc:492] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.336988 14812 ts_tablet_manager.cc:1397] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.083s user 0.001s sys 0.001s
I20250115 03:06:15.337124 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 5dd115f83c544c4bba0d08fd82220952 (table default.Table_6f23f5e22da844029c46908c62aeeb23 [id=1b7d3a80c5424e5692c2e6c556919342]) successfully deleted
W20250115 03:06:15.337253 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 6110b6c4911d4932b4cac4d86598a5c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6110b6c4911d4932b4cac4d86598a5c6 already in progress: creating tablet
I20250115 03:06:15.337405 14812 raft_consensus.cc:357] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.337510 14812 raft_consensus.cc:383] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.337544 14812 raft_consensus.cc:738] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.318288 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 5d27310f8433411a85bf88d05a81bcf0 (DEFAULT_TABLE table=default.Table_eec42b37d6de49ccad194b13edd928e3 [id=5d934a52e6704fa68f7c685ae3c72081]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.337649 14812 consensus_queue.cc:260] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.337738 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d27310f8433411a85bf88d05a81bcf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.337903 14812 ts_tablet_manager.cc:1428] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:15.338083 14812 tablet_bootstrap.cc:492] T e3302d62a34e43d09c11abca292d9110 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.337461 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 4cc6e937615b4aca8e9625d233452229 (table default.Table_85ea53c1d8b245cc94e6e81e157242aa [id=4d8e724e03dc4e2ab9a6e0281159aca8]) successfully deleted
I20250115 03:06:15.338966 14812 tablet_bootstrap.cc:654] T e3302d62a34e43d09c11abca292d9110 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.340045 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 914d0f374564460480f93aff40fee17f with delete_type TABLET_DATA_DELETED (replaced by tablet bf6f61dc685a48bdb367278773bf1d90) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:15.342186 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3a0bb9b5093b454d921bf1150d631565 consensus state has no leader
I20250115 03:06:15.344302 14819 ts_tablet_manager.cc:1905] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.344908 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet bb809ec664784386aa5f365bc27e0a1a from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:15.346187 14819 ts_tablet_manager.cc:1918] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.346252 14819 log.cc:1198] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/bda0e0238da246feb36450f40dd061e5
I20250115 03:06:15.346465 14819 ts_tablet_manager.cc:1939] T bda0e0238da246feb36450f40dd061e5 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.347211 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet bda0e0238da246feb36450f40dd061e5 (table default.Table_75cd7b01fc534652975eb593912b88bc [id=111fc67094164d3f99965c246ae12b24]) successfully deleted
I20250115 03:06:15.350385 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 8b0f0539530643f78207e528c39530af (DEFAULT_TABLE table=default.Table_1a535f4e7104456598818042ce64476f [id=fe357a2a64fc4531870b305eb2480df1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.350628 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b0f0539530643f78207e528c39530af. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.351408 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 8b0f0539530643f78207e528c39530af (DEFAULT_TABLE table=default.Table_1a535f4e7104456598818042ce64476f [id=fe357a2a64fc4531870b305eb2480df1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.351749 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b0f0539530643f78207e528c39530af. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.352031 14819 tablet_replica.cc:331] T 46e799d700db4803b79ed46273b99df6 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.352145 14819 raft_consensus.cc:2238] T 46e799d700db4803b79ed46273b99df6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.352119 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 8b0f0539530643f78207e528c39530af (DEFAULT_TABLE table=default.Table_1a535f4e7104456598818042ce64476f [id=fe357a2a64fc4531870b305eb2480df1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.352236 14819 raft_consensus.cc:2267] T 46e799d700db4803b79ed46273b99df6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.352340 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b0f0539530643f78207e528c39530af. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.352530 14819 ts_tablet_manager.cc:1905] T 46e799d700db4803b79ed46273b99df6 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.353001 15325 raft_consensus.cc:491] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.353078 15325 raft_consensus.cc:513] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.353377 15325 leader_election.cc:290] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:15.353869 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccf6b58c55d747da89c1c218a70ce09f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:15.354096 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet bb809ec664784386aa5f365bc27e0a1a
I20250115 03:06:15.354179 14819 ts_tablet_manager.cc:1918] T 46e799d700db4803b79ed46273b99df6 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.354236 14819 log.cc:1198] T 46e799d700db4803b79ed46273b99df6 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/46e799d700db4803b79ed46273b99df6
I20250115 03:06:15.354363 14163 catalog_manager.cc:6596] ReplaceTablet: tablet bb809ec664784386aa5f365bc27e0a1a deleted and replaced by tablet c84127dd93bc47ada551f93295ead47b
W20250115 03:06:15.354367 14628 leader_election.cc:343] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.354868 14819 ts_tablet_manager.cc:1939] T 46e799d700db4803b79ed46273b99df6 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.355321 14819 tablet_replica.cc:331] T 914d0f374564460480f93aff40fee17f P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.355424 14819 raft_consensus.cc:2238] T 914d0f374564460480f93aff40fee17f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.355510 14819 raft_consensus.cc:2267] T 914d0f374564460480f93aff40fee17f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.355798 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet f5af8b81e7ff4acab5aef13003656988 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:15.356632 14820 tablet_replica.cc:331] T 382116b1fb6a4f888ab1d55c29917ec8 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.357144 14820 raft_consensus.cc:2238] T 382116b1fb6a4f888ab1d55c29917ec8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.357302 14820 raft_consensus.cc:2267] T 382116b1fb6a4f888ab1d55c29917ec8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.357805 14820 ts_tablet_manager.cc:1905] T 382116b1fb6a4f888ab1d55c29917ec8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.357789 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccf6b58c55d747da89c1c218a70ce09f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.358299 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet bb809ec664784386aa5f365bc27e0a1a with delete_type TABLET_DATA_DELETED (replaced by tablet c84127dd93bc47ada551f93295ead47b) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.358433 15421 tablet.cc:1721] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:15.359052 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet bb809ec664784386aa5f365bc27e0a1a with delete_type TABLET_DATA_DELETED (replaced by tablet c84127dd93bc47ada551f93295ead47b) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.359223 14823 tablet_replica.cc:331] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.359226 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 46e799d700db4803b79ed46273b99df6 (table default.Table_77f0e54f01874d84ac9475bf9d819946 [id=f5e54ac99a344e05a0be4c913472dd37]) successfully deleted
I20250115 03:06:15.359361 14823 raft_consensus.cc:2238] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.359457 14823 raft_consensus.cc:2267] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.359781 14823 ts_tablet_manager.cc:1905] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.359961 14809 tablet_bootstrap.cc:492] T 47b08f6e7dec4c5a871a49c79421209e P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.360028 14809 ts_tablet_manager.cc:1397] T 47b08f6e7dec4c5a871a49c79421209e P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.028s user 0.001s sys 0.000s
I20250115 03:06:15.360481 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet f5af8b81e7ff4acab5aef13003656988
I20250115 03:06:15.360450 14809 raft_consensus.cc:357] T 47b08f6e7dec4c5a871a49c79421209e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.360532 14809 raft_consensus.cc:383] T 47b08f6e7dec4c5a871a49c79421209e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.360558 14809 raft_consensus.cc:738] T 47b08f6e7dec4c5a871a49c79421209e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.360656 14809 consensus_queue.cc:260] T 47b08f6e7dec4c5a871a49c79421209e P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.360733 14163 catalog_manager.cc:6596] ReplaceTablet: tablet f5af8b81e7ff4acab5aef13003656988 deleted and replaced by tablet 3efc2fb8427f493ea933347bec34f88f
I20250115 03:06:15.360868 14809 ts_tablet_manager.cc:1428] T 47b08f6e7dec4c5a871a49c79421209e P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.361620 14809 tablet_bootstrap.cc:492] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.361721 14823 ts_tablet_manager.cc:1918] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.361779 14823 log.cc:1198] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/bb809ec664784386aa5f365bc27e0a1a
I20250115 03:06:15.361954 14823 ts_tablet_manager.cc:1939] T bb809ec664784386aa5f365bc27e0a1a P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.362516 14809 tablet_bootstrap.cc:654] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.363489 14811 tablet_bootstrap.cc:492] T 37351252b608492e9acbc818b2754731 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.363569 14811 ts_tablet_manager.cc:1397] T 37351252b608492e9acbc818b2754731 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.079s user 0.001s sys 0.000s
I20250115 03:06:15.363555 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet bb809ec664784386aa5f365bc27e0a1a with delete_type TABLET_DATA_DELETED (replaced by tablet c84127dd93bc47ada551f93295ead47b) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.364032 14811 raft_consensus.cc:357] T 37351252b608492e9acbc818b2754731 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.364126 14811 raft_consensus.cc:383] T 37351252b608492e9acbc818b2754731 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.364158 14811 raft_consensus.cc:738] T 37351252b608492e9acbc818b2754731 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.364151 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet bb809ec664784386aa5f365bc27e0a1a (table default.Table_48970d0e864f42beaa17e0c0ce6d1786 [id=9391b8f0b8a144f592e4aeaef70fbeef]) successfully deleted
I20250115 03:06:15.364271 14811 consensus_queue.cc:260] T 37351252b608492e9acbc818b2754731 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.364518 14811 ts_tablet_manager.cc:1428] T 37351252b608492e9acbc818b2754731 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.364698 14811 tablet_bootstrap.cc:492] T 481705dc9ae841659520a6c909879f9b P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.364742 14812 tablet_bootstrap.cc:492] T e3302d62a34e43d09c11abca292d9110 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.364805 14812 ts_tablet_manager.cc:1397] T e3302d62a34e43d09c11abca292d9110 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.002s
W20250115 03:06:15.365253 14626 leader_election.cc:343] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.365317 14626 leader_election.cc:304] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:15.365482 15325 raft_consensus.cc:2743] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:15.365577 14811 tablet_bootstrap.cc:654] T 481705dc9ae841659520a6c909879f9b P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.365787 14812 raft_consensus.cc:357] T e3302d62a34e43d09c11abca292d9110 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.365876 14812 raft_consensus.cc:383] T e3302d62a34e43d09c11abca292d9110 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.365906 14812 raft_consensus.cc:738] T e3302d62a34e43d09c11abca292d9110 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.366011 14812 consensus_queue.cc:260] T e3302d62a34e43d09c11abca292d9110 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.366103 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet f5af8b81e7ff4acab5aef13003656988 with delete_type TABLET_DATA_DELETED (replaced by tablet 3efc2fb8427f493ea933347bec34f88f) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.366254 14812 ts_tablet_manager.cc:1428] T e3302d62a34e43d09c11abca292d9110 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.366406 14812 tablet_bootstrap.cc:492] T a1cbe29c08c7477e8944b7e108d46e7a P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.366576 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet f5af8b81e7ff4acab5aef13003656988 with delete_type TABLET_DATA_DELETED (replaced by tablet 3efc2fb8427f493ea933347bec34f88f) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.367247 14812 tablet_bootstrap.cc:654] T a1cbe29c08c7477e8944b7e108d46e7a P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.368549 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet f5af8b81e7ff4acab5aef13003656988 with delete_type TABLET_DATA_DELETED (replaced by tablet 3efc2fb8427f493ea933347bec34f88f) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:15.368983 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet f5af8b81e7ff4acab5aef13003656988 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f5af8b81e7ff4acab5aef13003656988 already in progress: creating tablet
I20250115 03:06:15.369843 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 47b08f6e7dec4c5a871a49c79421209e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.370198 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 37351252b608492e9acbc818b2754731 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.370586 14820 ts_tablet_manager.cc:1918] T 382116b1fb6a4f888ab1d55c29917ec8 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.370648 14820 log.cc:1198] T 382116b1fb6a4f888ab1d55c29917ec8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/382116b1fb6a4f888ab1d55c29917ec8
I20250115 03:06:15.371811 14820 ts_tablet_manager.cc:1939] T 382116b1fb6a4f888ab1d55c29917ec8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.372848 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 382116b1fb6a4f888ab1d55c29917ec8 (table default.Table_5e6d1abc27d042dba8506da7b49a6ee3 [id=a5c946319e8a4be5ba4a01a40a9ad0b7]) successfully deleted
I20250115 03:06:15.372910 14819 ts_tablet_manager.cc:1905] T 914d0f374564460480f93aff40fee17f P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.373308 14812 tablet_bootstrap.cc:492] T a1cbe29c08c7477e8944b7e108d46e7a P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.373383 14812 ts_tablet_manager.cc:1397] T a1cbe29c08c7477e8944b7e108d46e7a P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20250115 03:06:15.373801 14812 raft_consensus.cc:357] T a1cbe29c08c7477e8944b7e108d46e7a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.373888 14812 raft_consensus.cc:383] T a1cbe29c08c7477e8944b7e108d46e7a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.373919 14812 raft_consensus.cc:738] T a1cbe29c08c7477e8944b7e108d46e7a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.374020 14812 consensus_queue.cc:260] T a1cbe29c08c7477e8944b7e108d46e7a P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.374118 14820 tablet_replica.cc:331] T bb809ec664784386aa5f365bc27e0a1a P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.374233 14820 raft_consensus.cc:2238] T bb809ec664784386aa5f365bc27e0a1a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.374275 14812 ts_tablet_manager.cc:1428] T a1cbe29c08c7477e8944b7e108d46e7a P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.374320 14820 raft_consensus.cc:2267] T bb809ec664784386aa5f365bc27e0a1a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.374445 14812 tablet_bootstrap.cc:492] T f01c319ebd5148858de9ea1a608ddff3 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.375025 14666 tablet_service.cc:1467] Processing CreateTablet for tablet d2220e7ec26b4708b2c7a2540c053dcf (DEFAULT_TABLE table=default.Table_807e7c2704994c25aa03b1267a20f794 [id=fb7018f8e1ac4d6db5c0c114b0e73a07]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.375223 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2220e7ec26b4708b2c7a2540c053dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.375380 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 7c503b29cb2843338fb1870a89aee443 (DEFAULT_TABLE table=default.Table_4fcf8d20f3984b37910ea7164b26cca8 [id=f0a7068682b9481a82fa34cb54ed528e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.375510 14820 ts_tablet_manager.cc:1905] T bb809ec664784386aa5f365bc27e0a1a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.375588 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c503b29cb2843338fb1870a89aee443. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.376331 14819 ts_tablet_manager.cc:1918] T 914d0f374564460480f93aff40fee17f P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.376390 14819 log.cc:1198] T 914d0f374564460480f93aff40fee17f P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/914d0f374564460480f93aff40fee17f
I20250115 03:06:15.376560 14819 ts_tablet_manager.cc:1939] T 914d0f374564460480f93aff40fee17f P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.377044 14819 tablet_replica.cc:331] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.377149 14819 raft_consensus.cc:2238] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:15.377270 14819 pending_rounds.cc:62] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75: Trying to abort 1 pending ops.
I20250115 03:06:15.377368 14820 ts_tablet_manager.cc:1918] T bb809ec664784386aa5f365bc27e0a1a P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.377893 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 7c503b29cb2843338fb1870a89aee443 (DEFAULT_TABLE table=default.Table_4fcf8d20f3984b37910ea7164b26cca8 [id=f0a7068682b9481a82fa34cb54ed528e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.378086 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c503b29cb2843338fb1870a89aee443. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.378279 14812 tablet_bootstrap.cc:654] T f01c319ebd5148858de9ea1a608ddff3 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.378940 14537 tablet_service.cc:1467] Processing CreateTablet for tablet d2220e7ec26b4708b2c7a2540c053dcf (DEFAULT_TABLE table=default.Table_807e7c2704994c25aa03b1267a20f794 [id=fb7018f8e1ac4d6db5c0c114b0e73a07]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.379134 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2220e7ec26b4708b2c7a2540c053dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.377300 14819 pending_rounds.cc:69] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384896005140480 op_type: NO_OP noop_request { }
I20250115 03:06:15.379835 14819 raft_consensus.cc:2883] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:15.379894 14819 raft_consensus.cc:2267] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.380772 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 7c503b29cb2843338fb1870a89aee443 (DEFAULT_TABLE table=default.Table_4fcf8d20f3984b37910ea7164b26cca8 [id=f0a7068682b9481a82fa34cb54ed528e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.381026 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c503b29cb2843338fb1870a89aee443. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.381271 14819 ts_tablet_manager.cc:1905] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.382917 14819 ts_tablet_manager.cc:1918] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.382982 14819 log.cc:1198] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/bb809ec664784386aa5f365bc27e0a1a
I20250115 03:06:15.383036 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 074fdc09e4e24078ad0bb588b32ff24f (DEFAULT_TABLE table=default.Table_93108a53124a492599eda22f3633780a [id=eaddac40b10648b1933b121cc70d68e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.383199 14819 ts_tablet_manager.cc:1939] T bb809ec664784386aa5f365bc27e0a1a P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.383224 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 074fdc09e4e24078ad0bb588b32ff24f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.383790 14819 tablet_replica.cc:331] T 47b08f6e7dec4c5a871a49c79421209e P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.383895 14819 raft_consensus.cc:2238] T 47b08f6e7dec4c5a871a49c79421209e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.383978 14819 raft_consensus.cc:2267] T 47b08f6e7dec4c5a871a49c79421209e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.384121 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 074fdc09e4e24078ad0bb588b32ff24f (DEFAULT_TABLE table=default.Table_93108a53124a492599eda22f3633780a [id=eaddac40b10648b1933b121cc70d68e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.384338 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 074fdc09e4e24078ad0bb588b32ff24f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.384843 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 914d0f374564460480f93aff40fee17f (table default.Table_b0ca39935d224741840950333b63bcfe [id=08996d7663c94ed1acb94b675012934c]) successfully deleted
I20250115 03:06:15.385037 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet bb809ec664784386aa5f365bc27e0a1a (table default.Table_48970d0e864f42beaa17e0c0ce6d1786 [id=9391b8f0b8a144f592e4aeaef70fbeef]) successfully deleted
W20250115 03:06:15.385146 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet f5af8b81e7ff4acab5aef13003656988 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f5af8b81e7ff4acab5aef13003656988 already in progress: creating tablet
I20250115 03:06:15.385581 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet e3302d62a34e43d09c11abca292d9110 with delete_type TABLET_DATA_DELETED (replaced by tablet a4484651d964411e90ed93c6caab0a13) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.385727 14823 tablet_replica.cc:331] T e3302d62a34e43d09c11abca292d9110 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.385825 14823 raft_consensus.cc:2238] T e3302d62a34e43d09c11abca292d9110 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.385924 14823 raft_consensus.cc:2267] T e3302d62a34e43d09c11abca292d9110 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.386085 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet a1cbe29c08c7477e8944b7e108d46e7a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.386462 14823 ts_tablet_manager.cc:1905] T e3302d62a34e43d09c11abca292d9110 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:15.387603 14626 consensus_peers.cc:487] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:15.388072 14809 tablet_bootstrap.cc:492] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.388150 14809 ts_tablet_manager.cc:1397] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.000s
I20250115 03:06:15.388320 14823 ts_tablet_manager.cc:1918] T e3302d62a34e43d09c11abca292d9110 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.388379 14823 log.cc:1198] T e3302d62a34e43d09c11abca292d9110 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/e3302d62a34e43d09c11abca292d9110
I20250115 03:06:15.388268 14394 tablet_service.cc:1467] Processing CreateTablet for tablet d2220e7ec26b4708b2c7a2540c053dcf (DEFAULT_TABLE table=default.Table_807e7c2704994c25aa03b1267a20f794 [id=fb7018f8e1ac4d6db5c0c114b0e73a07]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.388550 14823 ts_tablet_manager.cc:1939] T e3302d62a34e43d09c11abca292d9110 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.388592 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2220e7ec26b4708b2c7a2540c053dcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.388617 14809 raft_consensus.cc:357] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.388710 14809 raft_consensus.cc:383] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.388736 14809 raft_consensus.cc:738] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.388828 14809 consensus_queue.cc:260] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.389076 14809 ts_tablet_manager.cc:1428] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.389235 14809 tablet_bootstrap.cc:492] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.389195 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 074fdc09e4e24078ad0bb588b32ff24f (DEFAULT_TABLE table=default.Table_93108a53124a492599eda22f3633780a [id=eaddac40b10648b1933b121cc70d68e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.389272 14823 tablet_replica.cc:331] T a1cbe29c08c7477e8944b7e108d46e7a P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.389372 14823 raft_consensus.cc:2238] T a1cbe29c08c7477e8944b7e108d46e7a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.389389 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 074fdc09e4e24078ad0bb588b32ff24f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.389451 14823 raft_consensus.cc:2267] T a1cbe29c08c7477e8944b7e108d46e7a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.389961 14823 ts_tablet_manager.cc:1905] T a1cbe29c08c7477e8944b7e108d46e7a P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.390084 14809 tablet_bootstrap.cc:654] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.390508 14820 log.cc:1198] T bb809ec664784386aa5f365bc27e0a1a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/bb809ec664784386aa5f365bc27e0a1a
I20250115 03:06:15.390722 14820 ts_tablet_manager.cc:1939] T bb809ec664784386aa5f365bc27e0a1a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.391429 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet bb809ec664784386aa5f365bc27e0a1a (table default.Table_48970d0e864f42beaa17e0c0ce6d1786 [id=9391b8f0b8a144f592e4aeaef70fbeef]) successfully deleted
W20250115 03:06:15.391821 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet f5af8b81e7ff4acab5aef13003656988 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f5af8b81e7ff4acab5aef13003656988 already in progress: creating tablet
I20250115 03:06:15.392032 14820 tablet_replica.cc:331] T 37351252b608492e9acbc818b2754731 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.392184 14820 raft_consensus.cc:2238] T 37351252b608492e9acbc818b2754731 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.392307 14820 raft_consensus.cc:2267] T 37351252b608492e9acbc818b2754731 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.392720 14820 ts_tablet_manager.cc:1905] T 37351252b608492e9acbc818b2754731 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.392856 14823 ts_tablet_manager.cc:1918] T a1cbe29c08c7477e8944b7e108d46e7a P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.392999 14823 log.cc:1198] T a1cbe29c08c7477e8944b7e108d46e7a P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/a1cbe29c08c7477e8944b7e108d46e7a
I20250115 03:06:15.393095 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet e3302d62a34e43d09c11abca292d9110 (table default.Table_2db28e1f2ea5450092ed00f593c4939f [id=6fe98239ee3346df8b25712aaec67b82]) successfully deleted
I20250115 03:06:15.393149 14811 tablet_bootstrap.cc:492] T 481705dc9ae841659520a6c909879f9b P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.393215 14811 ts_tablet_manager.cc:1397] T 481705dc9ae841659520a6c909879f9b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.000s
I20250115 03:06:15.393347 14823 ts_tablet_manager.cc:1939] T a1cbe29c08c7477e8944b7e108d46e7a P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.393625 14811 raft_consensus.cc:357] T 481705dc9ae841659520a6c909879f9b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.393747 14811 raft_consensus.cc:383] T 481705dc9ae841659520a6c909879f9b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.393780 14811 raft_consensus.cc:738] T 481705dc9ae841659520a6c909879f9b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.393877 14811 consensus_queue.cc:260] T 481705dc9ae841659520a6c909879f9b P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.394059 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet a1cbe29c08c7477e8944b7e108d46e7a (table default.Table_7e53fa73287c43e3a9a0e65908b42593 [id=1f983a10a8704e22b8a2b96ef4e06fbc]) successfully deleted
I20250115 03:06:15.394109 14811 ts_tablet_manager.cc:1428] T 481705dc9ae841659520a6c909879f9b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:15.394279 14811 tablet_bootstrap.cc:492] T 5dd115f83c544c4bba0d08fd82220952 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.395143 14811 tablet_bootstrap.cc:654] T 5dd115f83c544c4bba0d08fd82220952 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.396258 14820 ts_tablet_manager.cc:1918] T 37351252b608492e9acbc818b2754731 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.396317 14820 log.cc:1198] T 37351252b608492e9acbc818b2754731 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/37351252b608492e9acbc818b2754731
I20250115 03:06:15.396505 14820 ts_tablet_manager.cc:1939] T 37351252b608492e9acbc818b2754731 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.396970 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_717056a529de4818b9c40f37ccb2083b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:15.397166 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d76368b5971a4d08a167c7e7bac8144e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.397293 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_717056a529de4818b9c40f37ccb2083b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:15.397399 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d76368b5971a4d08a167c7e7bac8144e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.398347 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_48970d0e864f42beaa17e0c0ce6d1786" } new_table_name: "default.Table_121a1b9da2674287a03165a67467717f" modify_external_catalogs: true
I20250115 03:06:15.398936 14165 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 481705dc9ae841659520a6c909879f9b (table default.Table_bae37f38b5e04374ad63b8f34977c09a [id=fe32f8f8243b4803876ed2b371e9e602]). Expected version 2 got 0
I20250115 03:06:15.399209 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 37351252b608492e9acbc818b2754731 (table default.Table_e557ad0830f44aec955ce14c80b107e1 [id=0b8e801e3c8f40a6a5e226fbd824379e]) successfully deleted
I20250115 03:06:15.399633 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_a57c628374dc4ba2b91cb5c9a3ce8836"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.399900 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a57c628374dc4ba2b91cb5c9a3ce8836 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.402231 14809 tablet_bootstrap.cc:492] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.402307 14809 ts_tablet_manager.cc:1397] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20250115 03:06:15.402732 14809 raft_consensus.cc:357] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.402818 14809 raft_consensus.cc:383] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.402848 14809 raft_consensus.cc:738] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.402956 14809 consensus_queue.cc:260] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.403201 14809 ts_tablet_manager.cc:1428] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:15.403353 14809 tablet_bootstrap.cc:492] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.404147 14809 tablet_bootstrap.cc:654] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.404182 14795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "147c86b98f604d518e71f2ba3121fc21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:15.404275 14795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [sys.catalog]: This master's current role is: LEADER
I20250115 03:06:15.405596 14809 tablet_bootstrap.cc:492] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.405665 14809 ts_tablet_manager.cc:1397] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:15.405838 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a5cd96954b1c48c68a3bef815c47d8b4" } modify_external_catalogs: true
I20250115 03:06:15.405915 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_bae37f38b5e04374ad63b8f34977c09a" } modify_external_catalogs: true
I20250115 03:06:15.406002 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_bae37f38b5e04374ad63b8f34977c09a" } modify_external_catalogs: true
I20250115 03:06:15.405921 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a5cd96954b1c48c68a3bef815c47d8b4" } modify_external_catalogs: true
I20250115 03:06:15.406289 14809 raft_consensus.cc:357] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.406369 14809 raft_consensus.cc:383] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.406397 14809 raft_consensus.cc:738] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.406502 14809 consensus_queue.cc:260] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.406775 14809 ts_tablet_manager.cc:1428] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:15.406917 14809 tablet_bootstrap.cc:492] T 3e46d4a75e9b4ca8b83a188d06280fcd P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.407536 15390 consensus_queue.cc:1035] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.407877 14812 tablet_bootstrap.cc:492] T f01c319ebd5148858de9ea1a608ddff3 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.407956 14812 ts_tablet_manager.cc:1397] T f01c319ebd5148858de9ea1a608ddff3 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.034s user 0.001s sys 0.000s
I20250115 03:06:15.408347 14812 raft_consensus.cc:357] T f01c319ebd5148858de9ea1a608ddff3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.408458 14812 raft_consensus.cc:383] T f01c319ebd5148858de9ea1a608ddff3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.408486 14812 raft_consensus.cc:738] T f01c319ebd5148858de9ea1a608ddff3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.408586 14812 consensus_queue.cc:260] T f01c319ebd5148858de9ea1a608ddff3 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:15.408772 14157 catalog_manager.cc:4670] Async tablet task AlterTable RPC for tablet 3a0bb9b5093b454d921bf1150d631565 (table default.Table_3d196756e234426799a1de651bc112f2 [id=37d965d8a5eb408ea3f33313bd22c152], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3a0bb9b5093b454d921bf1150d631565 consensus state has no leader
I20250115 03:06:15.408839 14812 ts_tablet_manager.cc:1428] T f01c319ebd5148858de9ea1a608ddff3 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.409536 14812 tablet_bootstrap.cc:492] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.409742 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_68e894a7b2d7416a90c519daa7bf4fb9" } modify_external_catalogs: true
I20250115 03:06:15.409852 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_68e894a7b2d7416a90c519daa7bf4fb9" } modify_external_catalogs: true
I20250115 03:06:15.410404 14812 tablet_bootstrap.cc:654] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.410440 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet d4210020188d4f21aed9bb85f3b1f2a1 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:15.411140 14819 ts_tablet_manager.cc:1905] T 47b08f6e7dec4c5a871a49c79421209e P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.412024 14809 tablet_bootstrap.cc:654] T 3e46d4a75e9b4ca8b83a188d06280fcd P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:15.412063 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c84127dd93bc47ada551f93295ead47b has no consensus state
I20250115 03:06:15.412955 14809 tablet_bootstrap.cc:492] T 3e46d4a75e9b4ca8b83a188d06280fcd P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.413023 14809 ts_tablet_manager.cc:1397] T 3e46d4a75e9b4ca8b83a188d06280fcd P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250115 03:06:15.413764 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet f01c319ebd5148858de9ea1a608ddff3 with delete_type TABLET_DATA_DELETED (replaced by tablet 84ce55f729224f78b420e57977419add) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.413940 14823 tablet_replica.cc:331] T f01c319ebd5148858de9ea1a608ddff3 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.414078 14823 raft_consensus.cc:2238] T f01c319ebd5148858de9ea1a608ddff3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.414213 14823 raft_consensus.cc:2267] T f01c319ebd5148858de9ea1a608ddff3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.415736 14809 raft_consensus.cc:357] T 3e46d4a75e9b4ca8b83a188d06280fcd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.415948 14809 raft_consensus.cc:383] T 3e46d4a75e9b4ca8b83a188d06280fcd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.415977 14823 ts_tablet_manager.cc:1905] T f01c319ebd5148858de9ea1a608ddff3 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.415987 14809 raft_consensus.cc:738] T 3e46d4a75e9b4ca8b83a188d06280fcd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.416112 14809 consensus_queue.cc:260] T 3e46d4a75e9b4ca8b83a188d06280fcd P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.416342 14809 ts_tablet_manager.cc:1428] T 3e46d4a75e9b4ca8b83a188d06280fcd P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:15.417007 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 481705dc9ae841659520a6c909879f9b
I20250115 03:06:15.417213 14809 tablet_bootstrap.cc:492] T e3302d62a34e43d09c11abca292d9110 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.417752 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 481705dc9ae841659520a6c909879f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.417986 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 481705dc9ae841659520a6c909879f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.417976 14823 ts_tablet_manager.cc:1918] T f01c319ebd5148858de9ea1a608ddff3 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.418123 14823 log.cc:1198] T f01c319ebd5148858de9ea1a608ddff3 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/f01c319ebd5148858de9ea1a608ddff3
I20250115 03:06:15.418303 14823 ts_tablet_manager.cc:1939] T f01c319ebd5148858de9ea1a608ddff3 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.418476 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 481705dc9ae841659520a6c909879f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.418602 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 481705dc9ae841659520a6c909879f9b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.419207 14820 tablet_replica.cc:331] T 481705dc9ae841659520a6c909879f9b P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.419486 14820 raft_consensus.cc:2238] T 481705dc9ae841659520a6c909879f9b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.419579 14820 raft_consensus.cc:2267] T 481705dc9ae841659520a6c909879f9b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.420020 14823 tablet_replica.cc:331] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.420123 14823 raft_consensus.cc:2238] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:15.420205 14820 ts_tablet_manager.cc:1905] T 481705dc9ae841659520a6c909879f9b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.420213 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet f01c319ebd5148858de9ea1a608ddff3 (table default.Table_ff21d7bc463d43cfa3330f7288ba7a55 [id=5de066e9f2174fff9303cba23eeec988]) successfully deleted
I20250115 03:06:15.420282 14823 raft_consensus.cc:2267] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.421321 14394 tablet_service.cc:1467] Processing CreateTablet for tablet c84127dd93bc47ada551f93295ead47b (DEFAULT_TABLE table=default.Table_121a1b9da2674287a03165a67467717f [id=9391b8f0b8a144f592e4aeaef70fbeef]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.421402 14527 tablet_service.cc:1467] Processing CreateTablet for tablet c84127dd93bc47ada551f93295ead47b (DEFAULT_TABLE table=default.Table_121a1b9da2674287a03165a67467717f [id=9391b8f0b8a144f592e4aeaef70fbeef]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.421548 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c84127dd93bc47ada551f93295ead47b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.421623 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c84127dd93bc47ada551f93295ead47b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.422120 14820 ts_tablet_manager.cc:1918] T 481705dc9ae841659520a6c909879f9b P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.422173 14820 log.cc:1198] T 481705dc9ae841659520a6c909879f9b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/481705dc9ae841659520a6c909879f9b
W20250115 03:06:15.422248 14156 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.186.252:41855: connect: Connection refused (error 111) [suppressed 7 similar messages]
I20250115 03:06:15.422910 14812 tablet_bootstrap.cc:492] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.422976 14812 ts_tablet_manager.cc:1397] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20250115 03:06:15.423352 14820 ts_tablet_manager.cc:1939] T 481705dc9ae841659520a6c909879f9b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.423362 14812 raft_consensus.cc:357] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.423450 14812 raft_consensus.cc:383] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.423482 14812 raft_consensus.cc:738] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.423588 14812 consensus_queue.cc:260] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.423635 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 3efc2fb8427f493ea933347bec34f88f (DEFAULT_TABLE table=default.Table_f4fc7a15692446fba7fcdb17b6b33d56 [id=364a7905f3f044f684e9c75927f7efda]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.423795 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 3efc2fb8427f493ea933347bec34f88f (DEFAULT_TABLE table=default.Table_f4fc7a15692446fba7fcdb17b6b33d56 [id=364a7905f3f044f684e9c75927f7efda]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.423847 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3efc2fb8427f493ea933347bec34f88f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.423877 14812 ts_tablet_manager.cc:1428] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.424211 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 481705dc9ae841659520a6c909879f9b (table default.Table_bae37f38b5e04374ad63b8f34977c09a [id=fe32f8f8243b4803876ed2b371e9e602]) successfully deleted
W20250115 03:06:15.424338 14156 catalog_manager.cc:4839] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 481705dc9ae841659520a6c909879f9b because the tablet was not found. No further retry: Not found: Tablet not found: 481705dc9ae841659520a6c909879f9b
I20250115 03:06:15.424314 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 3efc2fb8427f493ea933347bec34f88f (DEFAULT_TABLE table=default.Table_f4fc7a15692446fba7fcdb17b6b33d56 [id=364a7905f3f044f684e9c75927f7efda]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.424515 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3efc2fb8427f493ea933347bec34f88f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.424612 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 5b7022db9edc4de8909e16769a3ea136
I20250115 03:06:15.425135 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b7022db9edc4de8909e16769a3ea136 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.425577 14167 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet ccf6b58c55d747da89c1c218a70ce09f (table default.Table_cbfbacbaeee4488aba1a8be104138404 [id=e441d6f5acea4837accbfb8e4856f82d]). Expected version 1 got 0
I20250115 03:06:15.426425 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b7022db9edc4de8909e16769a3ea136 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.426831 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet a1cbe29c08c7477e8944b7e108d46e7a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.426869 14809 tablet_bootstrap.cc:654] T e3302d62a34e43d09c11abca292d9110 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.427031 14812 tablet_bootstrap.cc:492] T 99d9180ed9b24363a4c44d01ea1d6e9b P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.426438 14663 tablet_service.cc:1467] Processing CreateTablet for tablet c84127dd93bc47ada551f93295ead47b (DEFAULT_TABLE table=default.Table_121a1b9da2674287a03165a67467717f [id=9391b8f0b8a144f592e4aeaef70fbeef]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.427312 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c84127dd93bc47ada551f93295ead47b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.427629 14811 tablet_bootstrap.cc:492] T 5dd115f83c544c4bba0d08fd82220952 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.427704 14811 ts_tablet_manager.cc:1397] T 5dd115f83c544c4bba0d08fd82220952 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.033s user 0.000s sys 0.001s
I20250115 03:06:15.427865 14812 tablet_bootstrap.cc:654] T 99d9180ed9b24363a4c44d01ea1d6e9b P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.428081 14811 raft_consensus.cc:357] T 5dd115f83c544c4bba0d08fd82220952 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:15.428174 14811 raft_consensus.cc:383] T 5dd115f83c544c4bba0d08fd82220952 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.428205 14811 raft_consensus.cc:738] T 5dd115f83c544c4bba0d08fd82220952 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.428310 14811 consensus_queue.cc:260] T 5dd115f83c544c4bba0d08fd82220952 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } }
I20250115 03:06:15.428568 14811 ts_tablet_manager.cc:1428] T 5dd115f83c544c4bba0d08fd82220952 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20250115 03:06:15.428742 14811 tablet_bootstrap.cc:492] T f978e85cf659450ca0039368d58b49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.429823 14811 tablet_bootstrap.cc:654] T f978e85cf659450ca0039368d58b49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.430644 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3efc2fb8427f493ea933347bec34f88f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.430771 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b7022db9edc4de8909e16769a3ea136 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:15.431121 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 5b7022db9edc4de8909e16769a3ea136 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5b7022db9edc4de8909e16769a3ea136 already in progress: creating tablet
I20250115 03:06:15.432188 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dd115f83c544c4bba0d08fd82220952 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.432310 14820 tablet_replica.cc:331] T 5dd115f83c544c4bba0d08fd82220952 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.432404 14820 raft_consensus.cc:2238] T 5dd115f83c544c4bba0d08fd82220952 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.432538 14820 raft_consensus.cc:2267] T 5dd115f83c544c4bba0d08fd82220952 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.432972 14820 ts_tablet_manager.cc:1905] T 5dd115f83c544c4bba0d08fd82220952 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.433683 14823 ts_tablet_manager.cc:1905] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.435818 14820 ts_tablet_manager.cc:1918] T 5dd115f83c544c4bba0d08fd82220952 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.435931 14820 log.cc:1198] T 5dd115f83c544c4bba0d08fd82220952 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/5dd115f83c544c4bba0d08fd82220952
I20250115 03:06:15.436154 14820 ts_tablet_manager.cc:1939] T 5dd115f83c544c4bba0d08fd82220952 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.436280 14819 ts_tablet_manager.cc:1918] T 47b08f6e7dec4c5a871a49c79421209e P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.436347 14819 log.cc:1198] T 47b08f6e7dec4c5a871a49c79421209e P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/47b08f6e7dec4c5a871a49c79421209e
I20250115 03:06:15.436546 14819 ts_tablet_manager.cc:1939] T 47b08f6e7dec4c5a871a49c79421209e P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.436733 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 5dd115f83c544c4bba0d08fd82220952 (table default.Table_6f23f5e22da844029c46908c62aeeb23 [id=1b7d3a80c5424e5692c2e6c556919342]) successfully deleted
I20250115 03:06:15.437074 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 47b08f6e7dec4c5a871a49c79421209e (table default.Table_f4d365d35e3c43e79d3ac20e438e7aa5 [id=835a8f488a7c42af850632b1286f9509]) successfully deleted
I20250115 03:06:15.437692 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 6a59a9be8fa94ec4bdb42c723ea0ca17
I20250115 03:06:15.438302 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a59a9be8fa94ec4bdb42c723ea0ca17 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.438490 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a59a9be8fa94ec4bdb42c723ea0ca17 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.438531 14809 tablet_bootstrap.cc:492] T e3302d62a34e43d09c11abca292d9110 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.438606 14809 ts_tablet_manager.cc:1397] T e3302d62a34e43d09c11abca292d9110 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.000s
I20250115 03:06:15.438657 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a59a9be8fa94ec4bdb42c723ea0ca17 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:15.439200 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 6a59a9be8fa94ec4bdb42c723ea0ca17 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a59a9be8fa94ec4bdb42c723ea0ca17 already in progress: creating tablet
I20250115 03:06:15.439159 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_4af064a5c1cc4ad7bc2b1d802e014110"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.439512 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4af064a5c1cc4ad7bc2b1d802e014110 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.441653 14809 raft_consensus.cc:357] T e3302d62a34e43d09c11abca292d9110 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.441833 14809 raft_consensus.cc:383] T e3302d62a34e43d09c11abca292d9110 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.441913 14809 raft_consensus.cc:738] T e3302d62a34e43d09c11abca292d9110 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.442085 14809 consensus_queue.cc:260] T e3302d62a34e43d09c11abca292d9110 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.442469 14809 ts_tablet_manager.cc:1428] T e3302d62a34e43d09c11abca292d9110 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:15.442874 14809 tablet_bootstrap.cc:492] T 89d92f59c7424dfc95427329200455e7 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.443075 15390 raft_consensus.cc:491] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.443143 15390 raft_consensus.cc:513] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.443439 15390 leader_election.cc:290] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:15.443787 14819 tablet_replica.cc:331] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.443897 14819 raft_consensus.cc:2238] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.443991 14819 raft_consensus.cc:2267] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:15.444023 14358 leader_election.cc:343] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: a1cbe29c08c7477e8944b7e108d46e7a
I20250115 03:06:15.444118 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_4a5e8b0dcd8c40de9598eecf00fec54e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:15.444169 14823 ts_tablet_manager.cc:1918] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:15.444227 14823 log.cc:1198] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/481705dc9ae841659520a6c909879f9b
W20250115 03:06:15.444396 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4a5e8b0dcd8c40de9598eecf00fec54e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.444443 14823 ts_tablet_manager.cc:1939] T 481705dc9ae841659520a6c909879f9b P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.444597 14811 tablet_bootstrap.cc:492] T f978e85cf659450ca0039368d58b49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.444700 14811 ts_tablet_manager.cc:1397] T f978e85cf659450ca0039368d58b49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20250115 03:06:15.444845 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1cbe29c08c7477e8944b7e108d46e7a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.445206 14811 raft_consensus.cc:357] T f978e85cf659450ca0039368d58b49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.445287 14811 raft_consensus.cc:383] T f978e85cf659450ca0039368d58b49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.445312 14811 raft_consensus.cc:738] T f978e85cf659450ca0039368d58b49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.445400 14811 consensus_queue.cc:260] T f978e85cf659450ca0039368d58b49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.445619 14811 ts_tablet_manager.cc:1428] T f978e85cf659450ca0039368d58b49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.445772 14811 tablet_bootstrap.cc:492] T 55f01d704b174b2ea108ea6f1df763df P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.445912 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 481705dc9ae841659520a6c909879f9b (table default.Table_bae37f38b5e04374ad63b8f34977c09a [id=fe32f8f8243b4803876ed2b371e9e602]) successfully deleted
I20250115 03:06:15.445950 14809 tablet_bootstrap.cc:654] T 89d92f59c7424dfc95427329200455e7 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.446557 14811 tablet_bootstrap.cc:654] T 55f01d704b174b2ea108ea6f1df763df P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.447202 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet e3302d62a34e43d09c11abca292d9110 with delete_type TABLET_DATA_DELETED (replaced by tablet a4484651d964411e90ed93c6caab0a13) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.450002 14811 tablet_bootstrap.cc:492] T 55f01d704b174b2ea108ea6f1df763df P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
W20250115 03:06:15.450049 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 5b7022db9edc4de8909e16769a3ea136 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5b7022db9edc4de8909e16769a3ea136 already in progress: creating tablet
I20250115 03:06:15.450080 14811 ts_tablet_manager.cc:1397] T 55f01d704b174b2ea108ea6f1df763df P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:15.450421 14811 raft_consensus.cc:357] T 55f01d704b174b2ea108ea6f1df763df P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.450507 14811 raft_consensus.cc:383] T 55f01d704b174b2ea108ea6f1df763df P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.450536 14811 raft_consensus.cc:738] T 55f01d704b174b2ea108ea6f1df763df P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.450629 14811 consensus_queue.cc:260] T 55f01d704b174b2ea108ea6f1df763df P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.450871 14811 ts_tablet_manager.cc:1428] T 55f01d704b174b2ea108ea6f1df763df P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.451021 14811 tablet_bootstrap.cc:492] T b5e7672ebc4442369156d12695261269 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
W20250115 03:06:15.451190 14356 leader_election.cc:343] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.451258 14356 leader_election.cc:304] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:15.451679 14861 raft_consensus.cc:2743] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:15.452025 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 6a59a9be8fa94ec4bdb42c723ea0ca17 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a59a9be8fa94ec4bdb42c723ea0ca17 already in progress: creating tablet
I20250115 03:06:15.452901 14811 tablet_bootstrap.cc:654] T b5e7672ebc4442369156d12695261269 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.453593 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_121a1b9da2674287a03165a67467717f" } modify_external_catalogs: true
I20250115 03:06:15.453684 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_121a1b9da2674287a03165a67467717f" } modify_external_catalogs: true
I20250115 03:06:15.453850 14819 ts_tablet_manager.cc:1905] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.454180 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0ab5bfc343e6431890e7f64f444e1c05"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.454470 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ab5bfc343e6431890e7f64f444e1c05 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.455670 15390 raft_consensus.cc:491] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.455742 15390 raft_consensus.cc:513] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.456030 15390 leader_election.cc:290] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:15.456408 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccf6b58c55d747da89c1c218a70ce09f" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:15.456533 14691 raft_consensus.cc:2463] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:15.456959 14358 leader_election.cc:304] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.458132 15390 raft_consensus.cc:2798] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:15.458186 15390 raft_consensus.cc:491] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:15.458220 15390 raft_consensus.cc:3054] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.458994 15390 raft_consensus.cc:513] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.459242 14819 ts_tablet_manager.cc:1918] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:15.459264 15390 leader_election.cc:290] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:15.459293 14819 log.cc:1198] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/481705dc9ae841659520a6c909879f9b
I20250115 03:06:15.459491 14819 ts_tablet_manager.cc:1939] T 481705dc9ae841659520a6c909879f9b P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.459712 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet 338592cc70ad4e61ba90905a424a3871 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:15.459738 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccf6b58c55d747da89c1c218a70ce09f" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:15.459826 14691 raft_consensus.cc:3054] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.460594 14691 raft_consensus.cc:2463] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:15.461362 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccf6b58c55d747da89c1c218a70ce09f" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:15.461793 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 55f01d704b174b2ea108ea6f1df763df with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.461824 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccf6b58c55d747da89c1c218a70ce09f" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.461926 14820 tablet_replica.cc:331] T 55f01d704b174b2ea108ea6f1df763df P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.462036 14820 raft_consensus.cc:2238] T 55f01d704b174b2ea108ea6f1df763df P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.462105 14820 raft_consensus.cc:2267] T 55f01d704b174b2ea108ea6f1df763df P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.462080 14861 consensus_queue.cc:1035] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.462464 14820 ts_tablet_manager.cc:1905] T 55f01d704b174b2ea108ea6f1df763df P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.462519 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 481705dc9ae841659520a6c909879f9b (table default.Table_bae37f38b5e04374ad63b8f34977c09a [id=fe32f8f8243b4803876ed2b371e9e602]) successfully deleted
I20250115 03:06:15.462867 14358 leader_election.cc:304] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
W20250115 03:06:15.462878 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 5b7022db9edc4de8909e16769a3ea136 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5b7022db9edc4de8909e16769a3ea136 already in progress: creating tablet
I20250115 03:06:15.463004 14861 raft_consensus.cc:2798] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:15.463065 14861 raft_consensus.cc:695] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:15.463162 14861 consensus_queue.cc:237] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.463861 14819 tablet_replica.cc:331] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.464190 14819 raft_consensus.cc:2238] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.464277 14819 raft_consensus.cc:2267] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.464368 14820 ts_tablet_manager.cc:1918] T 55f01d704b174b2ea108ea6f1df763df P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.464426 14820 log.cc:1198] T 55f01d704b174b2ea108ea6f1df763df P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/55f01d704b174b2ea108ea6f1df763df
I20250115 03:06:15.464607 14820 ts_tablet_manager.cc:1939] T 55f01d704b174b2ea108ea6f1df763df P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.464803 14811 tablet_bootstrap.cc:492] T b5e7672ebc4442369156d12695261269 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.464885 14811 ts_tablet_manager.cc:1397] T b5e7672ebc4442369156d12695261269 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.001s
W20250115 03:06:15.464897 14356 leader_election.cc:343] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:15.465076 14356 leader_election.cc:343] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.465296 14811 raft_consensus.cc:357] T b5e7672ebc4442369156d12695261269 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.465378 14811 raft_consensus.cc:383] T b5e7672ebc4442369156d12695261269 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.465404 14811 raft_consensus.cc:738] T b5e7672ebc4442369156d12695261269 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.465502 14811 consensus_queue.cc:260] T b5e7672ebc4442369156d12695261269 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.465726 14811 ts_tablet_manager.cc:1428] T b5e7672ebc4442369156d12695261269 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:15.465853 14811 tablet_bootstrap.cc:492] T 4cc6e937615b4aca8e9625d233452229 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.466782 14811 tablet_bootstrap.cc:654] T 4cc6e937615b4aca8e9625d233452229 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.468353 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 55f01d704b174b2ea108ea6f1df763df (table default.Table_64d7a4f0d2694a65a7fb86e55e1255e0 [id=9dcc0154ed914304a15cf96049afaf52]) successfully deleted
I20250115 03:06:15.469344 14811 tablet_bootstrap.cc:492] T 4cc6e937615b4aca8e9625d233452229 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.469411 14811 ts_tablet_manager.cc:1397] T 4cc6e937615b4aca8e9625d233452229 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20250115 03:06:15.471117 14811 raft_consensus.cc:357] T 4cc6e937615b4aca8e9625d233452229 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.471174 14812 tablet_bootstrap.cc:492] T 99d9180ed9b24363a4c44d01ea1d6e9b P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.471211 14811 raft_consensus.cc:383] T 4cc6e937615b4aca8e9625d233452229 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.471238 14811 raft_consensus.cc:738] T 4cc6e937615b4aca8e9625d233452229 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.471251 14812 ts_tablet_manager.cc:1397] T 99d9180ed9b24363a4c44d01ea1d6e9b P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.044s user 0.001s sys 0.000s
I20250115 03:06:15.471447 14811 consensus_queue.cc:260] T 4cc6e937615b4aca8e9625d233452229 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.471735 14812 raft_consensus.cc:357] T 99d9180ed9b24363a4c44d01ea1d6e9b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.471858 14812 raft_consensus.cc:383] T 99d9180ed9b24363a4c44d01ea1d6e9b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.471892 14812 raft_consensus.cc:738] T 99d9180ed9b24363a4c44d01ea1d6e9b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.471998 14812 consensus_queue.cc:260] T 99d9180ed9b24363a4c44d01ea1d6e9b P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.472249 14812 ts_tablet_manager.cc:1428] T 99d9180ed9b24363a4c44d01ea1d6e9b P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.472440 14812 tablet_bootstrap.cc:492] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.473044 14811 ts_tablet_manager.cc:1428] T 4cc6e937615b4aca8e9625d233452229 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:15.473366 14812 tablet_bootstrap.cc:654] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.474172 14811 tablet_bootstrap.cc:492] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.475135 14811 tablet_bootstrap.cc:654] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.475144 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 338592cc70ad4e61ba90905a424a3871
I20250115 03:06:15.475417 14164 catalog_manager.cc:6596] ReplaceTablet: tablet 338592cc70ad4e61ba90905a424a3871 deleted and replaced by tablet 0f3bf849316a4ebd841c3345ca17087e
I20250115 03:06:15.475857 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 338592cc70ad4e61ba90905a424a3871 with delete_type TABLET_DATA_DELETED (replaced by tablet 0f3bf849316a4ebd841c3345ca17087e) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.476194 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 338592cc70ad4e61ba90905a424a3871 with delete_type TABLET_DATA_DELETED (replaced by tablet 0f3bf849316a4ebd841c3345ca17087e) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.476975 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 99d9180ed9b24363a4c44d01ea1d6e9b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.477015 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 338592cc70ad4e61ba90905a424a3871 with delete_type TABLET_DATA_DELETED (replaced by tablet 0f3bf849316a4ebd841c3345ca17087e) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.477169 14823 tablet_replica.cc:331] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.477224 14819 ts_tablet_manager.cc:1905] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.477283 14823 raft_consensus.cc:2238] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.477378 14823 raft_consensus.cc:2267] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.478245 14812 tablet_bootstrap.cc:492] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.478286 14820 tablet_replica.cc:331] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.478333 14812 ts_tablet_manager.cc:1397] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20250115 03:06:15.478387 14820 raft_consensus.cc:2238] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.478478 14820 raft_consensus.cc:2267] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.478741 14812 raft_consensus.cc:357] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.478825 14812 raft_consensus.cc:383] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.478854 14812 raft_consensus.cc:738] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.478955 14812 consensus_queue.cc:260] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.479202 14812 ts_tablet_manager.cc:1428] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:15.479286 14820 ts_tablet_manager.cc:1905] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.479359 14812 tablet_bootstrap.cc:492] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.480048 14812 tablet_bootstrap.cc:654] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.481068 14820 ts_tablet_manager.cc:1918] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20250115 03:06:15.481128 14820 log.cc:1198] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/338592cc70ad4e61ba90905a424a3871
I20250115 03:06:15.481338 14820 ts_tablet_manager.cc:1939] T 338592cc70ad4e61ba90905a424a3871 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.478957 14823 ts_tablet_manager.cc:1905] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.481899 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 338592cc70ad4e61ba90905a424a3871 (table default.Table_96c8d5f733034ee3a6dd8a61dabfd288 [id=ff4681cab8ce44ceb45ed76c70c1132e]) successfully deleted
I20250115 03:06:15.482463 14819 ts_tablet_manager.cc:1918] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.482519 14819 log.cc:1198] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/a1cbe29c08c7477e8944b7e108d46e7a
I20250115 03:06:15.482533 14812 tablet_bootstrap.cc:492] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.482599 14812 ts_tablet_manager.cc:1397] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20250115 03:06:15.482707 14819 ts_tablet_manager.cc:1939] T a1cbe29c08c7477e8944b7e108d46e7a P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.482998 14812 raft_consensus.cc:357] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.483081 14812 raft_consensus.cc:383] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.483109 14812 raft_consensus.cc:738] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.483215 14812 consensus_queue.cc:260] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.483268 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet a1cbe29c08c7477e8944b7e108d46e7a (table default.Table_7e53fa73287c43e3a9a0e65908b42593 [id=1f983a10a8704e22b8a2b96ef4e06fbc]) successfully deleted
I20250115 03:06:15.483489 14812 ts_tablet_manager.cc:1428] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20250115 03:06:15.483650 14812 tablet_bootstrap.cc:492] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:15.483636 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 6a59a9be8fa94ec4bdb42c723ea0ca17 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a59a9be8fa94ec4bdb42c723ea0ca17 already in progress: creating tablet
I20250115 03:06:15.483816 14819 tablet_replica.cc:331] T e3302d62a34e43d09c11abca292d9110 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.483494 14823 ts_tablet_manager.cc:1918] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20250115 03:06:15.483938 14819 raft_consensus.cc:2238] T e3302d62a34e43d09c11abca292d9110 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.483974 14823 log.cc:1198] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/338592cc70ad4e61ba90905a424a3871
I20250115 03:06:15.484022 14819 raft_consensus.cc:2267] T e3302d62a34e43d09c11abca292d9110 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.484189 14823 ts_tablet_manager.cc:1939] T 338592cc70ad4e61ba90905a424a3871 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.484642 14823 tablet_replica.cc:331] T 99d9180ed9b24363a4c44d01ea1d6e9b P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.484685 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 338592cc70ad4e61ba90905a424a3871 (table default.Table_96c8d5f733034ee3a6dd8a61dabfd288 [id=ff4681cab8ce44ceb45ed76c70c1132e]) successfully deleted
I20250115 03:06:15.484735 14823 raft_consensus.cc:2238] T 99d9180ed9b24363a4c44d01ea1d6e9b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.484812 14823 raft_consensus.cc:2267] T 99d9180ed9b24363a4c44d01ea1d6e9b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.484900 14812 tablet_bootstrap.cc:654] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.485145 14819 ts_tablet_manager.cc:1905] T e3302d62a34e43d09c11abca292d9110 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.485272 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 22cc0833bfde458ebe3985c05d29b8e6 (table default.Table_3db38338e5b04298954aded6d5482169 [id=f56a0de6e56441bfa620902f175feb4a]). Expected version 2 got 0
W20250115 03:06:15.485515 14167 catalog_manager.cc:5718] Failed to send AlterTable RPC for tablet 22cc0833bfde458ebe3985c05d29b8e6 (table default.Table_3db38338e5b04298954aded6d5482169 [id=f56a0de6e56441bfa620902f175feb4a], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 22cc0833bfde458ebe3985c05d29b8e6 consensus state has no leader
I20250115 03:06:15.486811 14819 ts_tablet_manager.cc:1918] T e3302d62a34e43d09c11abca292d9110 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.486876 14819 log.cc:1198] T e3302d62a34e43d09c11abca292d9110 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/e3302d62a34e43d09c11abca292d9110
I20250115 03:06:15.487046 14819 ts_tablet_manager.cc:1939] T e3302d62a34e43d09c11abca292d9110 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.487474 14819 tablet_replica.cc:331] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.487691 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet e3302d62a34e43d09c11abca292d9110 (table default.Table_2db28e1f2ea5450092ed00f593c4939f [id=6fe98239ee3346df8b25712aaec67b82]) successfully deleted
I20250115 03:06:15.490906 14811 tablet_bootstrap.cc:492] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.490967 14811 ts_tablet_manager.cc:1397] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20250115 03:06:15.491283 14811 raft_consensus.cc:357] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.491381 14811 raft_consensus.cc:383] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.491406 14811 raft_consensus.cc:738] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.491500 14811 consensus_queue.cc:260] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.491559 14166 catalog_manager.cc:5526] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:15.491715 14811 ts_tablet_manager.cc:1428] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.491780 14166 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet ccf6b58c55d747da89c1c218a70ce09f (table default.Table_cbfbacbaeee4488aba1a8be104138404 [id=e441d6f5acea4837accbfb8e4856f82d]). Expected version 1 got 0
I20250115 03:06:15.491847 14811 tablet_bootstrap.cc:492] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.492269 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 31446d60b85d4a878e9afe142011aeb1 (DEFAULT_TABLE table=default.Table_717056a529de4818b9c40f37ccb2083b [id=fa31aef04ed04e58888ba86469303e0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.492486 14811 tablet_bootstrap.cc:654] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.492501 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31446d60b85d4a878e9afe142011aeb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.493575 14823 ts_tablet_manager.cc:1905] T 99d9180ed9b24363a4c44d01ea1d6e9b P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.493764 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 01433c76a398462bb6708a02997047fa (DEFAULT_TABLE table=default.Table_a57c628374dc4ba2b91cb5c9a3ce8836 [id=960713d5552d4e339aa8c94edbe9c7b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.493988 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01433c76a398462bb6708a02997047fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.495187 14823 ts_tablet_manager.cc:1918] T 99d9180ed9b24363a4c44d01ea1d6e9b P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.495246 14823 log.cc:1198] T 99d9180ed9b24363a4c44d01ea1d6e9b P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/99d9180ed9b24363a4c44d01ea1d6e9b
I20250115 03:06:15.495411 14823 ts_tablet_manager.cc:1939] T 99d9180ed9b24363a4c44d01ea1d6e9b P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.495862 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4bfe9fa1f7a143d684bc948b47deb8d8" } modify_external_catalogs: true
I20250115 03:06:15.495952 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4bfe9fa1f7a143d684bc948b47deb8d8" } modify_external_catalogs: true
I20250115 03:06:15.496142 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 31446d60b85d4a878e9afe142011aeb1 (DEFAULT_TABLE table=default.Table_717056a529de4818b9c40f37ccb2083b [id=fa31aef04ed04e58888ba86469303e0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.496249 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 69533c30b57e45dab0b00239dfd90f13 (DEFAULT_TABLE table=default.Table_d76368b5971a4d08a167c7e7bac8144e [id=3968ef424a944c9c83aacaea3ccc7c0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.496304 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 69533c30b57e45dab0b00239dfd90f13 (DEFAULT_TABLE table=default.Table_d76368b5971a4d08a167c7e7bac8144e [id=3968ef424a944c9c83aacaea3ccc7c0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.496348 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31446d60b85d4a878e9afe142011aeb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.496456 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69533c30b57e45dab0b00239dfd90f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.496481 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69533c30b57e45dab0b00239dfd90f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.496668 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 01433c76a398462bb6708a02997047fa (DEFAULT_TABLE table=default.Table_a57c628374dc4ba2b91cb5c9a3ce8836 [id=960713d5552d4e339aa8c94edbe9c7b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.496914 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01433c76a398462bb6708a02997047fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.498610 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 31446d60b85d4a878e9afe142011aeb1 (DEFAULT_TABLE table=default.Table_717056a529de4818b9c40f37ccb2083b [id=fa31aef04ed04e58888ba86469303e0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.498678 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 01433c76a398462bb6708a02997047fa (DEFAULT_TABLE table=default.Table_a57c628374dc4ba2b91cb5c9a3ce8836 [id=960713d5552d4e339aa8c94edbe9c7b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.498828 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31446d60b85d4a878e9afe142011aeb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.498664 14819 raft_consensus.cc:2238] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:15.498966 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01433c76a398462bb6708a02997047fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.501368 14819 raft_consensus.cc:2267] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.501849 14819 ts_tablet_manager.cc:1905] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.503037 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet c84127dd93bc47ada551f93295ead47b
I20250115 03:06:15.503687 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet c84127dd93bc47ada551f93295ead47b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.503669 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 99d9180ed9b24363a4c44d01ea1d6e9b (table default.Table_6bb3a06c1ae749e593cdc3f2b3992eaf [id=89578bd1e7e64c0fa550f20f592ccc6f]) successfully deleted
I20250115 03:06:15.503707 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 69533c30b57e45dab0b00239dfd90f13 (DEFAULT_TABLE table=default.Table_d76368b5971a4d08a167c7e7bac8144e [id=3968ef424a944c9c83aacaea3ccc7c0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.503888 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 69533c30b57e45dab0b00239dfd90f13. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.504254 14812 tablet_bootstrap.cc:492] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.504328 14812 ts_tablet_manager.cc:1397] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.001s
I20250115 03:06:15.504483 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_863155db5aee457a9ada887faf6deb85" } new_table_name: "default.Table_375bb4e2095c4675b734247fabacbec7" modify_external_catalogs: true
I20250115 03:06:15.505139 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet 55dcc7ac82ac44ab9d85055720760d3d from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:15.505749 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet c84127dd93bc47ada551f93295ead47b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.506225 14812 raft_consensus.cc:357] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.506340 14812 raft_consensus.cc:383] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.506395 14812 raft_consensus.cc:738] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.506523 14812 consensus_queue.cc:260] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.507531 14812 ts_tablet_manager.cc:1428] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20250115 03:06:15.507709 14812 tablet_bootstrap.cc:492] T 2379bae3690c44949e658dde84c1712e P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.508184 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet c84127dd93bc47ada551f93295ead47b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.508553 14812 tablet_bootstrap.cc:654] T 2379bae3690c44949e658dde84c1712e P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.509464 14819 ts_tablet_manager.cc:1918] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20250115 03:06:15.509502 14812 tablet_bootstrap.cc:492] T 2379bae3690c44949e658dde84c1712e P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.509531 14819 log.cc:1198] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/338592cc70ad4e61ba90905a424a3871
I20250115 03:06:15.509568 14812 ts_tablet_manager.cc:1397] T 2379bae3690c44949e658dde84c1712e P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20250115 03:06:15.509932 14812 raft_consensus.cc:357] T 2379bae3690c44949e658dde84c1712e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.510015 14812 raft_consensus.cc:383] T 2379bae3690c44949e658dde84c1712e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.510044 14812 raft_consensus.cc:738] T 2379bae3690c44949e658dde84c1712e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.510129 14812 consensus_queue.cc:260] T 2379bae3690c44949e658dde84c1712e P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.510217 14811 tablet_bootstrap.cc:492] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.510282 14811 ts_tablet_manager.cc:1397] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20250115 03:06:15.510537 14812 ts_tablet_manager.cc:1428] T 2379bae3690c44949e658dde84c1712e P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:15.510710 14812 tablet_bootstrap.cc:492] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.510740 14811 raft_consensus.cc:357] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.510835 14811 raft_consensus.cc:383] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.510869 14811 raft_consensus.cc:738] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.510965 14811 consensus_queue.cc:260] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.511179 14811 ts_tablet_manager.cc:1428] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.511335 14811 tablet_bootstrap.cc:492] T 914d0f374564460480f93aff40fee17f P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.512110 14811 tablet_bootstrap.cc:654] T 914d0f374564460480f93aff40fee17f P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:15.512441 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet c84127dd93bc47ada551f93295ead47b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c84127dd93bc47ada551f93295ead47b already in progress: creating tablet
I20250115 03:06:15.512478 14812 tablet_bootstrap.cc:654] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.513607 14812 tablet_bootstrap.cc:492] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.513679 14812 ts_tablet_manager.cc:1397] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:15.514082 14812 raft_consensus.cc:357] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.514164 14812 raft_consensus.cc:383] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.514196 14812 raft_consensus.cc:738] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.514256 14819 ts_tablet_manager.cc:1939] T 338592cc70ad4e61ba90905a424a3871 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.514283 14812 consensus_queue.cc:260] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.514525 14812 ts_tablet_manager.cc:1428] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.514668 14812 tablet_bootstrap.cc:492] T 82c2f41769d84bd48bfc1bc7f504be4b P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.515542 14812 tablet_bootstrap.cc:654] T 82c2f41769d84bd48bfc1bc7f504be4b P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.516130 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c
W20250115 03:06:15.516557 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dcdb5282e41c43dbb235ae78465d19f9 consensus state has no leader
I20250115 03:06:15.516863 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 55dcc7ac82ac44ab9d85055720760d3d
I20250115 03:06:15.517359 14811 tablet_bootstrap.cc:492] T 914d0f374564460480f93aff40fee17f P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.517436 14811 ts_tablet_manager.cc:1397] T 914d0f374564460480f93aff40fee17f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:15.517686 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 338592cc70ad4e61ba90905a424a3871 (table default.Table_96c8d5f733034ee3a6dd8a61dabfd288 [id=ff4681cab8ce44ceb45ed76c70c1132e]) successfully deleted
I20250115 03:06:15.517797 14171 catalog_manager.cc:6596] ReplaceTablet: tablet 55dcc7ac82ac44ab9d85055720760d3d deleted and replaced by tablet 907301ca08e0467f9f29c6dcb36bdd5e
I20250115 03:06:15.517778 14811 raft_consensus.cc:357] T 914d0f374564460480f93aff40fee17f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:15.517818 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet c84127dd93bc47ada551f93295ead47b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c84127dd93bc47ada551f93295ead47b already in progress: creating tablet
I20250115 03:06:15.517861 14811 raft_consensus.cc:383] T 914d0f374564460480f93aff40fee17f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.517889 14811 raft_consensus.cc:738] T 914d0f374564460480f93aff40fee17f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.517987 14811 consensus_queue.cc:260] T 914d0f374564460480f93aff40fee17f P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.518193 14811 ts_tablet_manager.cc:1428] T 914d0f374564460480f93aff40fee17f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.518225 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 55dcc7ac82ac44ab9d85055720760d3d with delete_type TABLET_DATA_DELETED (replaced by tablet 907301ca08e0467f9f29c6dcb36bdd5e) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.518342 14811 tablet_bootstrap.cc:492] T 47b08f6e7dec4c5a871a49c79421209e P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.518731 14812 tablet_bootstrap.cc:492] T 82c2f41769d84bd48bfc1bc7f504be4b P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.518801 14812 ts_tablet_manager.cc:1397] T 82c2f41769d84bd48bfc1bc7f504be4b P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250115 03:06:15.519192 14811 tablet_bootstrap.cc:654] T 47b08f6e7dec4c5a871a49c79421209e P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.519202 14812 raft_consensus.cc:357] T 82c2f41769d84bd48bfc1bc7f504be4b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.519286 14812 raft_consensus.cc:383] T 82c2f41769d84bd48bfc1bc7f504be4b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.519313 14812 raft_consensus.cc:738] T 82c2f41769d84bd48bfc1bc7f504be4b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.519414 14819 tablet_replica.cc:331] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.519405 14812 consensus_queue.cc:260] T 82c2f41769d84bd48bfc1bc7f504be4b P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.519523 14819 raft_consensus.cc:2238] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:15.519645 14812 ts_tablet_manager.cc:1428] T 82c2f41769d84bd48bfc1bc7f504be4b P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.519698 14819 raft_consensus.cc:2267] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.519785 14812 tablet_bootstrap.cc:492] T b1644ba8616349dc9d10a790b6089571 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.519938 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.520061 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 55dcc7ac82ac44ab9d85055720760d3d with delete_type TABLET_DATA_DELETED (replaced by tablet 907301ca08e0467f9f29c6dcb36bdd5e) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.520166 14823 tablet_replica.cc:331] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.520260 14823 raft_consensus.cc:2238] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.520349 14823 raft_consensus.cc:2267] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.520604 14812 tablet_bootstrap.cc:654] T b1644ba8616349dc9d10a790b6089571 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.521288 14823 ts_tablet_manager.cc:1905] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:15.522828 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet c84127dd93bc47ada551f93295ead47b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c84127dd93bc47ada551f93295ead47b already in progress: creating tablet
I20250115 03:06:15.523015 14823 ts_tablet_manager.cc:1918] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.523065 14823 log.cc:1198] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/5504a9b8a4ea47e2b4cfd98c92e25d5c
I20250115 03:06:15.523250 14823 ts_tablet_manager.cc:1939] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.523299 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0e5754dbfde84156b74d86bba62a0046"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:15.523464 14809 tablet_bootstrap.cc:492] T 89d92f59c7424dfc95427329200455e7 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.523533 14809 ts_tablet_manager.cc:1397] T 89d92f59c7424dfc95427329200455e7 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.081s user 0.001s sys 0.000s
W20250115 03:06:15.523612 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0e5754dbfde84156b74d86bba62a0046 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.523809 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cc6e937615b4aca8e9625d233452229 with delete_type TABLET_DATA_DELETED (replaced by tablet acb43d614111477abb10def48a1dfb3b) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.523928 14820 tablet_replica.cc:331] T 4cc6e937615b4aca8e9625d233452229 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.524006 14820 raft_consensus.cc:2238] T 4cc6e937615b4aca8e9625d233452229 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.523957 14809 raft_consensus.cc:357] T 89d92f59c7424dfc95427329200455e7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.524045 14809 raft_consensus.cc:383] T 89d92f59c7424dfc95427329200455e7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.524075 14820 raft_consensus.cc:2267] T 4cc6e937615b4aca8e9625d233452229 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.524076 14809 raft_consensus.cc:738] T 89d92f59c7424dfc95427329200455e7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.524186 14809 consensus_queue.cc:260] T 89d92f59c7424dfc95427329200455e7 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.524425 14809 ts_tablet_manager.cc:1428] T 89d92f59c7424dfc95427329200455e7 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:15.524582 14809 tablet_bootstrap.cc:492] T f01c319ebd5148858de9ea1a608ddff3 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.525427 14809 tablet_bootstrap.cc:654] T f01c319ebd5148858de9ea1a608ddff3 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.525807 14823 tablet_replica.cc:331] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.525898 14823 raft_consensus.cc:2238] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.525897 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.525965 14823 raft_consensus.cc:2267] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.526007 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 55dcc7ac82ac44ab9d85055720760d3d with delete_type TABLET_DATA_DELETED (replaced by tablet 907301ca08e0467f9f29c6dcb36bdd5e) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.526086 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet d4210020188d4f21aed9bb85f3b1f2a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:13 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.526605 14820 ts_tablet_manager.cc:1905] T 4cc6e937615b4aca8e9625d233452229 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.527642 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c (table default.Table_4bfe9fa1f7a143d684bc948b47deb8d8 [id=339b61af27cb486aaaadf8730ab21ce7]) successfully deleted
I20250115 03:06:15.528473 14820 ts_tablet_manager.cc:1918] T 4cc6e937615b4aca8e9625d233452229 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.528538 14820 log.cc:1198] T 4cc6e937615b4aca8e9625d233452229 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/4cc6e937615b4aca8e9625d233452229
I20250115 03:06:15.528708 14820 ts_tablet_manager.cc:1939] T 4cc6e937615b4aca8e9625d233452229 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.528944 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a75300c14c1f40d68ecd81adc691f38a" } new_table_name: "default.Table_ef19dd8e16aa4bf9a97897ebc1c8eef5" modify_external_catalogs: true
I20250115 03:06:15.529536 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 914d0f374564460480f93aff40fee17f with delete_type TABLET_DATA_DELETED (replaced by tablet bf6f61dc685a48bdb367278773bf1d90) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.529634 14820 tablet_replica.cc:331] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.529656 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 53248628d9584f8b886166c1e8ec5c3c with delete_type TABLET_DATA_DELETED (replaced by tablet 7f67cbb929cd4ae589a14606aa168b46) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.529740 14820 raft_consensus.cc:2238] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.529810 14820 raft_consensus.cc:2267] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.529886 14817 raft_consensus.cc:491] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.529942 14817 raft_consensus.cc:513] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.530232 14817 leader_election.cc:290] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:15.530915 14493 leader_election.cc:343] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: d4210020188d4f21aed9bb85f3b1f2a1
I20250115 03:06:15.530972 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 4cc6e937615b4aca8e9625d233452229 (table default.Table_85ea53c1d8b245cc94e6e81e157242aa [id=4d8e724e03dc4e2ab9a6e0281159aca8]) successfully deleted
I20250115 03:06:15.531394 14823 ts_tablet_manager.cc:1905] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.531735 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8350c61b90924563b015ad45382885e3" } new_table_name: "default.Table_6fadef5b204546918bd04347978ae2c2" modify_external_catalogs: true
W20250115 03:06:15.532756 14493 leader_election.cc:343] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: d4210020188d4f21aed9bb85f3b1f2a1
I20250115 03:06:15.532829 14493 leader_election.cc:304] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:15.533012 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.533211 14817 raft_consensus.cc:2743] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:15.533329 14820 ts_tablet_manager.cc:1905] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.533377 14819 ts_tablet_manager.cc:1905] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.533784 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_652f8e06e005491381404a11759aefb1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:15.533883 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 89d92f59c7424dfc95427329200455e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:15.534066 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_652f8e06e005491381404a11759aefb1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.535190 14819 ts_tablet_manager.cc:1918] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.535915 14820 ts_tablet_manager.cc:1918] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.532954 14823 ts_tablet_manager.cc:1918] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.536082 14823 log.cc:1198] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/55dcc7ac82ac44ab9d85055720760d3d
I20250115 03:06:15.536317 14823 ts_tablet_manager.cc:1939] T 55dcc7ac82ac44ab9d85055720760d3d P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.536827 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 55dcc7ac82ac44ab9d85055720760d3d (table default.Table_46f14a41d55b4e4783139238fbd0ba49 [id=c9645f75dd5e4011974d87019ff11435]) successfully deleted
I20250115 03:06:15.537110 14820 log.cc:1198] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/5504a9b8a4ea47e2b4cfd98c92e25d5c
I20250115 03:06:15.537329 14820 ts_tablet_manager.cc:1939] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.537814 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c (table default.Table_4bfe9fa1f7a143d684bc948b47deb8d8 [id=339b61af27cb486aaaadf8730ab21ce7]) successfully deleted
I20250115 03:06:15.537971 14820 tablet_replica.cc:331] T 55dcc7ac82ac44ab9d85055720760d3d P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.538065 14820 raft_consensus.cc:2238] T 55dcc7ac82ac44ab9d85055720760d3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.538138 14820 raft_consensus.cc:2267] T 55dcc7ac82ac44ab9d85055720760d3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.538204 14811 tablet_bootstrap.cc:492] T 47b08f6e7dec4c5a871a49c79421209e P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.538276 14811 ts_tablet_manager.cc:1397] T 47b08f6e7dec4c5a871a49c79421209e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.000s
I20250115 03:06:15.537865 14809 tablet_bootstrap.cc:492] T f01c319ebd5148858de9ea1a608ddff3 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.538496 14809 ts_tablet_manager.cc:1397] T f01c319ebd5148858de9ea1a608ddff3 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.002s
I20250115 03:06:15.538676 14811 raft_consensus.cc:357] T 47b08f6e7dec4c5a871a49c79421209e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.538771 14811 raft_consensus.cc:383] T 47b08f6e7dec4c5a871a49c79421209e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.538803 14811 raft_consensus.cc:738] T 47b08f6e7dec4c5a871a49c79421209e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.538863 14809 raft_consensus.cc:357] T f01c319ebd5148858de9ea1a608ddff3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.538906 14811 consensus_queue.cc:260] T 47b08f6e7dec4c5a871a49c79421209e P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.538947 14809 raft_consensus.cc:383] T f01c319ebd5148858de9ea1a608ddff3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.538973 14809 raft_consensus.cc:738] T f01c319ebd5148858de9ea1a608ddff3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.539081 14809 consensus_queue.cc:260] T f01c319ebd5148858de9ea1a608ddff3 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.539137 14811 ts_tablet_manager.cc:1428] T 47b08f6e7dec4c5a871a49c79421209e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.539304 14811 tablet_bootstrap.cc:492] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.540180 14811 tablet_bootstrap.cc:654] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.541388 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 82bf1d7ac9fe4a2eb7b57cf1ab627c3d (DEFAULT_TABLE table=default.Table_0ab5bfc343e6431890e7f64f444e1c05 [id=2f9b0aa8377744488a89ba41fb20d5c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.541592 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82bf1d7ac9fe4a2eb7b57cf1ab627c3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.541726 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 51b412277e634ea39bb6d2b90a43b875 (DEFAULT_TABLE table=default.Table_4af064a5c1cc4ad7bc2b1d802e014110 [id=752d9e6368564d1ca34397e8e3eea7e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.541927 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51b412277e634ea39bb6d2b90a43b875. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.543515 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 5cb34cb2006f4716b87bef67f3a6dd8e (DEFAULT_TABLE table=default.Table_4a5e8b0dcd8c40de9598eecf00fec54e [id=ed28aa2aaacd4ba58dcb3216a7156a91]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.543720 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cb34cb2006f4716b87bef67f3a6dd8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.544202 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3bf849316a4ebd841c3345ca17087e (DEFAULT_TABLE table=default.Table_96c8d5f733034ee3a6dd8a61dabfd288 [id=ff4681cab8ce44ceb45ed76c70c1132e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.544404 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f3bf849316a4ebd841c3345ca17087e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.544476 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 51b412277e634ea39bb6d2b90a43b875 (DEFAULT_TABLE table=default.Table_4af064a5c1cc4ad7bc2b1d802e014110 [id=752d9e6368564d1ca34397e8e3eea7e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.544699 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51b412277e634ea39bb6d2b90a43b875. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.545097 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 82bf1d7ac9fe4a2eb7b57cf1ab627c3d (DEFAULT_TABLE table=default.Table_0ab5bfc343e6431890e7f64f444e1c05 [id=2f9b0aa8377744488a89ba41fb20d5c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.545287 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82bf1d7ac9fe4a2eb7b57cf1ab627c3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.545573 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3bf849316a4ebd841c3345ca17087e (DEFAULT_TABLE table=default.Table_96c8d5f733034ee3a6dd8a61dabfd288 [id=ff4681cab8ce44ceb45ed76c70c1132e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.545758 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f3bf849316a4ebd841c3345ca17087e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.547048 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 5cb34cb2006f4716b87bef67f3a6dd8e (DEFAULT_TABLE table=default.Table_4a5e8b0dcd8c40de9598eecf00fec54e [id=ed28aa2aaacd4ba58dcb3216a7156a91]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.547226 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cb34cb2006f4716b87bef67f3a6dd8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.547490 14853 raft_consensus.cc:491] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.547562 14853 raft_consensus.cc:513] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.547881 14853 leader_election.cc:290] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:15.548484 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 51b412277e634ea39bb6d2b90a43b875 (DEFAULT_TABLE table=default.Table_4af064a5c1cc4ad7bc2b1d802e014110 [id=752d9e6368564d1ca34397e8e3eea7e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.548709 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51b412277e634ea39bb6d2b90a43b875. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.548745 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3bf849316a4ebd841c3345ca17087e (DEFAULT_TABLE table=default.Table_96c8d5f733034ee3a6dd8a61dabfd288 [id=ff4681cab8ce44ceb45ed76c70c1132e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.549233 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 5cb34cb2006f4716b87bef67f3a6dd8e (DEFAULT_TABLE table=default.Table_4a5e8b0dcd8c40de9598eecf00fec54e [id=ed28aa2aaacd4ba58dcb3216a7156a91]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.549857 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 82bf1d7ac9fe4a2eb7b57cf1ab627c3d (DEFAULT_TABLE table=default.Table_0ab5bfc343e6431890e7f64f444e1c05 [id=2f9b0aa8377744488a89ba41fb20d5c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.550503 14812 tablet_bootstrap.cc:492] T b1644ba8616349dc9d10a790b6089571 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.550581 14812 ts_tablet_manager.cc:1397] T b1644ba8616349dc9d10a790b6089571 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.031s user 0.001s sys 0.000s
I20250115 03:06:15.551056 14812 raft_consensus.cc:357] T b1644ba8616349dc9d10a790b6089571 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.551142 14812 raft_consensus.cc:383] T b1644ba8616349dc9d10a790b6089571 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.551170 14812 raft_consensus.cc:738] T b1644ba8616349dc9d10a790b6089571 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.551255 14812 consensus_queue.cc:260] T b1644ba8616349dc9d10a790b6089571 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.551529 14812 ts_tablet_manager.cc:1428] T b1644ba8616349dc9d10a790b6089571 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:15.551672 14812 tablet_bootstrap.cc:492] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.552443 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f3bf849316a4ebd841c3345ca17087e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.552611 14812 tablet_bootstrap.cc:654] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.553807 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d9b79701501644caa5b8be7be1d354df" } new_table_name: "default.Table_6b1f81ee12e04acda4bcf7d7f289a67a" modify_external_catalogs: true
I20250115 03:06:15.554337 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cb34cb2006f4716b87bef67f3a6dd8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.554484 14809 ts_tablet_manager.cc:1428] T f01c319ebd5148858de9ea1a608ddff3 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.016s user 0.000s sys 0.001s
I20250115 03:06:15.554630 14809 tablet_bootstrap.cc:492] T e368f87556944c46a6d8f160ead17ac9 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.554826 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_67f7434767174808bef25bc2c55394fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.555128 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_67f7434767174808bef25bc2c55394fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.555735 14809 tablet_bootstrap.cc:654] T e368f87556944c46a6d8f160ead17ac9 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.556571 14820 ts_tablet_manager.cc:1905] T 55dcc7ac82ac44ab9d85055720760d3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.557041 14811 tablet_bootstrap.cc:492] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.557117 14811 ts_tablet_manager.cc:1397] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20250115 03:06:15.557155 15274 tablet.cc:1721] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:15.557469 14811 raft_consensus.cc:357] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.557552 14811 raft_consensus.cc:383] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.557579 14811 raft_consensus.cc:738] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.557673 14811 consensus_queue.cc:260] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.557907 14811 ts_tablet_manager.cc:1428] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.558068 14811 tablet_bootstrap.cc:492] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.559022 14811 tablet_bootstrap.cc:654] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.559329 14820 ts_tablet_manager.cc:1918] T 55dcc7ac82ac44ab9d85055720760d3d P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.559444 14820 log.cc:1198] T 55dcc7ac82ac44ab9d85055720760d3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/55dcc7ac82ac44ab9d85055720760d3d
I20250115 03:06:15.559710 14820 ts_tablet_manager.cc:1939] T 55dcc7ac82ac44ab9d85055720760d3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.560161 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 47b08f6e7dec4c5a871a49c79421209e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.560339 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 55dcc7ac82ac44ab9d85055720760d3d (table default.Table_46f14a41d55b4e4783139238fbd0ba49 [id=c9645f75dd5e4011974d87019ff11435]) successfully deleted
I20250115 03:06:15.560310 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_1a2ec424c5cb40fbb3c93f327531e36f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.560593 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1a2ec424c5cb40fbb3c93f327531e36f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.561259 15426 tablet.cc:1721] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:15.561786 14171 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet ccf6b58c55d747da89c1c218a70ce09f (table default.Table_cbfbacbaeee4488aba1a8be104138404 [id=e441d6f5acea4837accbfb8e4856f82d]). Expected version 1 got 0
I20250115 03:06:15.562193 14812 tablet_bootstrap.cc:492] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.562271 14812 ts_tablet_manager.cc:1397] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20250115 03:06:15.562489 15453 raft_consensus.cc:491] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.562556 15453 raft_consensus.cc:513] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.562690 14812 raft_consensus.cc:357] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.562763 14812 raft_consensus.cc:383] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.562791 14812 raft_consensus.cc:738] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.562858 15453 leader_election.cc:290] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:15.562876 14812 consensus_queue.cc:260] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.563126 14812 ts_tablet_manager.cc:1428] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:15.563266 14812 tablet_bootstrap.cc:492] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.564121 14812 tablet_bootstrap.cc:654] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.564522 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5e7672ebc4442369156d12695261269" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.564631 14554 raft_consensus.cc:2463] T b5e7672ebc4442369156d12695261269 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
W20250115 03:06:15.564882 14493 leader_election.cc:343] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 53248628d9584f8b886166c1e8ec5c3c
I20250115 03:06:15.564879 14626 leader_election.cc:304] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:15.565073 15325 raft_consensus.cc:2798] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:15.565112 15325 raft_consensus.cc:491] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:15.565145 15325 raft_consensus.cc:3054] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.565117 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5e7672ebc4442369156d12695261269" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:15.565215 14420 raft_consensus.cc:2463] T b5e7672ebc4442369156d12695261269 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
W20250115 03:06:15.565798 14493 leader_election.cc:343] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 53248628d9584f8b886166c1e8ec5c3c
I20250115 03:06:15.565863 14493 leader_election.cc:304] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:15.565932 15325 raft_consensus.cc:513] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.566068 14817 raft_consensus.cc:2743] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:15.566207 15325 leader_election.cc:290] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:15.566771 14811 tablet_bootstrap.cc:492] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.566843 14811 ts_tablet_manager.cc:1397] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20250115 03:06:15.566861 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5e7672ebc4442369156d12695261269" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:15.566951 14554 raft_consensus.cc:3054] T b5e7672ebc4442369156d12695261269 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.567211 14811 raft_consensus.cc:357] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.567296 14811 raft_consensus.cc:383] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.567327 14811 raft_consensus.cc:738] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.567422 14811 consensus_queue.cc:260] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.567695 14811 ts_tablet_manager.cc:1428] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.567746 14554 raft_consensus.cc:2463] T b5e7672ebc4442369156d12695261269 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:15.567844 14811 tablet_bootstrap.cc:492] T e3302d62a34e43d09c11abca292d9110 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.568233 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82bf1d7ac9fe4a2eb7b57cf1ab627c3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.568686 14811 tablet_bootstrap.cc:654] T e3302d62a34e43d09c11abca292d9110 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.569231 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5e7672ebc4442369156d12695261269" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:15.569324 14420 raft_consensus.cc:3054] T b5e7672ebc4442369156d12695261269 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.569669 14626 leader_election.cc:304] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:15.569834 15325 raft_consensus.cc:2798] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:15.569914 15325 raft_consensus.cc:695] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:15.570026 15325 consensus_queue.cc:237] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.570124 14420 raft_consensus.cc:2463] T b5e7672ebc4442369156d12695261269 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:15.570781 15426 tablet.cc:1721] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:15.571092 15421 tablet.cc:1721] T fc056b9dbc374d26bf3feaf524f4d834 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:15.535264 14819 log.cc:1198] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/55dcc7ac82ac44ab9d85055720760d3d
I20250115 03:06:15.573201 14819 ts_tablet_manager.cc:1939] T 55dcc7ac82ac44ab9d85055720760d3d P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.573655 14819 tablet_replica.cc:331] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.573727 14820 tablet_replica.cc:331] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.573827 14820 raft_consensus.cc:2238] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.573899 14820 raft_consensus.cc:2267] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.574314 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 55dcc7ac82ac44ab9d85055720760d3d (table default.Table_46f14a41d55b4e4783139238fbd0ba49 [id=c9645f75dd5e4011974d87019ff11435]) successfully deleted
I20250115 03:06:15.574867 14819 raft_consensus.cc:2238] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:15.575024 14819 raft_consensus.cc:2267] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.575435 14819 ts_tablet_manager.cc:1905] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.575584 14171 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet ccf6b58c55d747da89c1c218a70ce09f (table default.Table_cbfbacbaeee4488aba1a8be104138404 [id=e441d6f5acea4837accbfb8e4856f82d]). Expected version 1 got 0
I20250115 03:06:15.575733 15421 tablet.cc:1721] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:15.577230 14819 ts_tablet_manager.cc:1918] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.577230 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet f01c319ebd5148858de9ea1a608ddff3 with delete_type TABLET_DATA_DELETED (replaced by tablet 84ce55f729224f78b420e57977419add) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.577318 14819 log.cc:1198] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/5504a9b8a4ea47e2b4cfd98c92e25d5c
I20250115 03:06:15.577612 14820 ts_tablet_manager.cc:1905] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.578171 14811 tablet_bootstrap.cc:492] T e3302d62a34e43d09c11abca292d9110 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.578254 14811 ts_tablet_manager.cc:1397] T e3302d62a34e43d09c11abca292d9110 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20250115 03:06:15.578258 14819 ts_tablet_manager.cc:1939] T 5504a9b8a4ea47e2b4cfd98c92e25d5c P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.578649 14811 raft_consensus.cc:357] T e3302d62a34e43d09c11abca292d9110 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.578748 14811 raft_consensus.cc:383] T e3302d62a34e43d09c11abca292d9110 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.578783 14811 raft_consensus.cc:738] T e3302d62a34e43d09c11abca292d9110 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.578838 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 5504a9b8a4ea47e2b4cfd98c92e25d5c (table default.Table_4bfe9fa1f7a143d684bc948b47deb8d8 [id=339b61af27cb486aaaadf8730ab21ce7]) successfully deleted
I20250115 03:06:15.578889 14811 consensus_queue.cc:260] T e3302d62a34e43d09c11abca292d9110 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.579142 14811 ts_tablet_manager.cc:1428] T e3302d62a34e43d09c11abca292d9110 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:15.580276 14819 tablet_replica.cc:331] T 89d92f59c7424dfc95427329200455e7 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.580391 14819 raft_consensus.cc:2238] T 89d92f59c7424dfc95427329200455e7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.580479 14819 raft_consensus.cc:2267] T 89d92f59c7424dfc95427329200455e7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.580919 14811 tablet_bootstrap.cc:492] T a1cbe29c08c7477e8944b7e108d46e7a P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.581043 14819 ts_tablet_manager.cc:1905] T 89d92f59c7424dfc95427329200455e7 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.581753 14809 tablet_bootstrap.cc:492] T e368f87556944c46a6d8f160ead17ac9 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.581820 14811 tablet_bootstrap.cc:654] T a1cbe29c08c7477e8944b7e108d46e7a P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.581840 14809 ts_tablet_manager.cc:1397] T e368f87556944c46a6d8f160ead17ac9 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.001s
I20250115 03:06:15.582226 14809 raft_consensus.cc:357] T e368f87556944c46a6d8f160ead17ac9 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.582320 14809 raft_consensus.cc:383] T e368f87556944c46a6d8f160ead17ac9 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.582355 14809 raft_consensus.cc:738] T e368f87556944c46a6d8f160ead17ac9 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.582465 14809 consensus_queue.cc:260] T e368f87556944c46a6d8f160ead17ac9 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.582666 14820 ts_tablet_manager.cc:1918] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.583316 14809 ts_tablet_manager.cc:1428] T e368f87556944c46a6d8f160ead17ac9 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.583307 15453 raft_consensus.cc:491] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.583379 15453 raft_consensus.cc:513] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.583690 15453 leader_election.cc:290] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.584053 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4484651d964411e90ed93c6caab0a13" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:15.584347 14628 leader_election.cc:343] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.584380 14809 tablet_bootstrap.cc:492] T 2379bae3690c44949e658dde84c1712e P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.583988 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4484651d964411e90ed93c6caab0a13" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.585199 14820 log.cc:1198] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/d4210020188d4f21aed9bb85f3b1f2a1
W20250115 03:06:15.585299 14626 leader_election.cc:343] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.585354 14626 leader_election.cc:304] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:15.585393 14820 ts_tablet_manager.cc:1939] T d4210020188d4f21aed9bb85f3b1f2a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.585526 15453 raft_consensus.cc:2743] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:15.585853 14820 tablet_replica.cc:331] T 914d0f374564460480f93aff40fee17f P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.585963 14820 raft_consensus.cc:2238] T 914d0f374564460480f93aff40fee17f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.585958 14165 master_service.cc:946] ReplaceTablet: received request to replace tablet 82c2f41769d84bd48bfc1bc7f504be4b from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:15.586046 14820 raft_consensus.cc:2267] T 914d0f374564460480f93aff40fee17f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.586028 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet d4210020188d4f21aed9bb85f3b1f2a1 (table default.Table_fb8c07c4a56044248f3fdef9c488f45c [id=eee0481c12b14e31b6b5c5b9c1688f59]) successfully deleted
I20250115 03:06:15.586228 14809 tablet_bootstrap.cc:654] T 2379bae3690c44949e658dde84c1712e P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.587098 14809 tablet_bootstrap.cc:492] T 2379bae3690c44949e658dde84c1712e P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.587229 14809 ts_tablet_manager.cc:1397] T 2379bae3690c44949e658dde84c1712e P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20250115 03:06:15.587680 14809 raft_consensus.cc:357] T 2379bae3690c44949e658dde84c1712e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.587822 14809 raft_consensus.cc:383] T 2379bae3690c44949e658dde84c1712e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.587893 14809 raft_consensus.cc:738] T 2379bae3690c44949e658dde84c1712e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.588064 14809 consensus_queue.cc:260] T 2379bae3690c44949e658dde84c1712e P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.588531 14809 ts_tablet_manager.cc:1428] T 2379bae3690c44949e658dde84c1712e P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:15.588725 14809 tablet_bootstrap.cc:492] T 99d9180ed9b24363a4c44d01ea1d6e9b P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.589277 14820 ts_tablet_manager.cc:1905] T 914d0f374564460480f93aff40fee17f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.588829 14819 ts_tablet_manager.cc:1918] T 89d92f59c7424dfc95427329200455e7 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.589357 14819 log.cc:1198] T 89d92f59c7424dfc95427329200455e7 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/89d92f59c7424dfc95427329200455e7
I20250115 03:06:15.589555 14819 ts_tablet_manager.cc:1939] T 89d92f59c7424dfc95427329200455e7 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.589891 14809 tablet_bootstrap.cc:654] T 99d9180ed9b24363a4c44d01ea1d6e9b P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.590083 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 89d92f59c7424dfc95427329200455e7 (table default.Table_236770d2959743108285eae378066e45 [id=fec89f498f5046839fe491caa3367a36]) successfully deleted
I20250115 03:06:15.590269 14819 tablet_replica.cc:331] T f01c319ebd5148858de9ea1a608ddff3 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.590376 14819 raft_consensus.cc:2238] T f01c319ebd5148858de9ea1a608ddff3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.590884 14819 raft_consensus.cc:2267] T f01c319ebd5148858de9ea1a608ddff3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.591491 14819 ts_tablet_manager.cc:1905] T f01c319ebd5148858de9ea1a608ddff3 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.592561 14820 ts_tablet_manager.cc:1918] T 914d0f374564460480f93aff40fee17f P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.592630 14820 log.cc:1198] T 914d0f374564460480f93aff40fee17f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/914d0f374564460480f93aff40fee17f
I20250115 03:06:15.593189 14819 ts_tablet_manager.cc:1918] T f01c319ebd5148858de9ea1a608ddff3 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.593256 14819 log.cc:1198] T f01c319ebd5148858de9ea1a608ddff3 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/f01c319ebd5148858de9ea1a608ddff3
I20250115 03:06:15.593438 14819 ts_tablet_manager.cc:1939] T f01c319ebd5148858de9ea1a608ddff3 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.593922 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet f01c319ebd5148858de9ea1a608ddff3 (table default.Table_ff21d7bc463d43cfa3330f7288ba7a55 [id=5de066e9f2174fff9303cba23eeec988]) successfully deleted
I20250115 03:06:15.596802 14812 tablet_bootstrap.cc:492] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.596885 14812 ts_tablet_manager.cc:1397] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.034s user 0.000s sys 0.002s
I20250115 03:06:15.597326 14812 raft_consensus.cc:357] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.597378 14809 tablet_bootstrap.cc:492] T 99d9180ed9b24363a4c44d01ea1d6e9b P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.597421 14812 raft_consensus.cc:383] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.597455 14812 raft_consensus.cc:738] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.597450 14809 ts_tablet_manager.cc:1397] T 99d9180ed9b24363a4c44d01ea1d6e9b P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:15.597535 15274 tablet.cc:1721] T fc056b9dbc374d26bf3feaf524f4d834 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:15.597558 14812 consensus_queue.cc:260] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.597815 14812 ts_tablet_manager.cc:1428] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.597839 14809 raft_consensus.cc:357] T 99d9180ed9b24363a4c44d01ea1d6e9b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.597924 14809 raft_consensus.cc:383] T 99d9180ed9b24363a4c44d01ea1d6e9b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.597954 14809 raft_consensus.cc:738] T 99d9180ed9b24363a4c44d01ea1d6e9b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.597970 14812 tablet_bootstrap.cc:492] T 2d78a7793f84424bad42a1aba89eab92 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.598052 14809 consensus_queue.cc:260] T 99d9180ed9b24363a4c44d01ea1d6e9b P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.598290 14809 ts_tablet_manager.cc:1428] T 99d9180ed9b24363a4c44d01ea1d6e9b P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.598438 14809 tablet_bootstrap.cc:492] T f3ac7f1e1bfb4383895dc2c2123a743b P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.598708 14812 tablet_bootstrap.cc:654] T 2d78a7793f84424bad42a1aba89eab92 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.599280 14809 tablet_bootstrap.cc:654] T f3ac7f1e1bfb4383895dc2c2123a743b P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.596865 14811 tablet_bootstrap.cc:492] T a1cbe29c08c7477e8944b7e108d46e7a P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.601032 14811 ts_tablet_manager.cc:1397] T a1cbe29c08c7477e8944b7e108d46e7a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.001s
I20250115 03:06:15.601547 14811 raft_consensus.cc:357] T a1cbe29c08c7477e8944b7e108d46e7a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.601629 14811 raft_consensus.cc:383] T a1cbe29c08c7477e8944b7e108d46e7a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.601656 14811 raft_consensus.cc:738] T a1cbe29c08c7477e8944b7e108d46e7a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.601761 14811 consensus_queue.cc:260] T a1cbe29c08c7477e8944b7e108d46e7a P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.602007 14811 ts_tablet_manager.cc:1428] T a1cbe29c08c7477e8944b7e108d46e7a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.602124 14537 tablet_service.cc:1467] Processing CreateTablet for tablet d12d1f964d2b486bb703ed98cb2e7949 (DEFAULT_TABLE table=default.Table_0e5754dbfde84156b74d86bba62a0046 [id=739ec7394ea24ccb849d2486e451496b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.602182 14811 tablet_bootstrap.cc:492] T 89d92f59c7424dfc95427329200455e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.602347 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d12d1f964d2b486bb703ed98cb2e7949. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.602892 14811 tablet_bootstrap.cc:654] T 89d92f59c7424dfc95427329200455e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.603514 14820 ts_tablet_manager.cc:1939] T 914d0f374564460480f93aff40fee17f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.603693 14811 tablet_bootstrap.cc:492] T 89d92f59c7424dfc95427329200455e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.603785 14811 ts_tablet_manager.cc:1397] T 89d92f59c7424dfc95427329200455e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:15.603971 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 914d0f374564460480f93aff40fee17f (table default.Table_b0ca39935d224741840950333b63bcfe [id=08996d7663c94ed1acb94b675012934c]) successfully deleted
I20250115 03:06:15.604178 14811 raft_consensus.cc:357] T 89d92f59c7424dfc95427329200455e7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.604517 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 907301ca08e0467f9f29c6dcb36bdd5e (DEFAULT_TABLE table=default.Table_46f14a41d55b4e4783139238fbd0ba49 [id=c9645f75dd5e4011974d87019ff11435]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.604735 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 907301ca08e0467f9f29c6dcb36bdd5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.604784 14393 tablet_service.cc:1467] Processing CreateTablet for tablet d12d1f964d2b486bb703ed98cb2e7949 (DEFAULT_TABLE table=default.Table_0e5754dbfde84156b74d86bba62a0046 [id=739ec7394ea24ccb849d2486e451496b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.605015 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d12d1f964d2b486bb703ed98cb2e7949. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.605136 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 907301ca08e0467f9f29c6dcb36bdd5e (DEFAULT_TABLE table=default.Table_46f14a41d55b4e4783139238fbd0ba49 [id=c9645f75dd5e4011974d87019ff11435]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.605311 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 907301ca08e0467f9f29c6dcb36bdd5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.606581 14663 tablet_service.cc:1467] Processing CreateTablet for tablet d12d1f964d2b486bb703ed98cb2e7949 (DEFAULT_TABLE table=default.Table_0e5754dbfde84156b74d86bba62a0046 [id=739ec7394ea24ccb849d2486e451496b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.604559 14811 raft_consensus.cc:383] T 89d92f59c7424dfc95427329200455e7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.606882 14811 raft_consensus.cc:738] T 89d92f59c7424dfc95427329200455e7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.607062 14811 consensus_queue.cc:260] T 89d92f59c7424dfc95427329200455e7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.607100 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 907301ca08e0467f9f29c6dcb36bdd5e (DEFAULT_TABLE table=default.Table_46f14a41d55b4e4783139238fbd0ba49 [id=c9645f75dd5e4011974d87019ff11435]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.607314 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 907301ca08e0467f9f29c6dcb36bdd5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.607573 14811 ts_tablet_manager.cc:1428] T 89d92f59c7424dfc95427329200455e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:15.607970 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d12d1f964d2b486bb703ed98cb2e7949. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.604180 14820 tablet_replica.cc:331] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.609555 14820 raft_consensus.cc:2238] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.609644 14820 raft_consensus.cc:2267] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.610038 14820 ts_tablet_manager.cc:1905] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.610112 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 82c2f41769d84bd48bfc1bc7f504be4b
I20250115 03:06:15.610380 14165 catalog_manager.cc:6596] ReplaceTablet: tablet 82c2f41769d84bd48bfc1bc7f504be4b deleted and replaced by tablet 80aa570e2c154172ad6da29d268bede7
I20250115 03:06:15.611195 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 82c2f41769d84bd48bfc1bc7f504be4b with delete_type TABLET_DATA_DELETED (replaced by tablet 80aa570e2c154172ad6da29d268bede7) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.612048 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 82c2f41769d84bd48bfc1bc7f504be4b with delete_type TABLET_DATA_DELETED (replaced by tablet 80aa570e2c154172ad6da29d268bede7) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.612206 14823 tablet_replica.cc:331] T 82c2f41769d84bd48bfc1bc7f504be4b P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.612320 14823 raft_consensus.cc:2238] T 82c2f41769d84bd48bfc1bc7f504be4b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.612413 14811 tablet_bootstrap.cc:492] T 3e46d4a75e9b4ca8b83a188d06280fcd P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.612421 14823 raft_consensus.cc:2267] T 82c2f41769d84bd48bfc1bc7f504be4b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.612695 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 82c2f41769d84bd48bfc1bc7f504be4b with delete_type TABLET_DATA_DELETED (replaced by tablet 80aa570e2c154172ad6da29d268bede7) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:15.613181 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 82c2f41769d84bd48bfc1bc7f504be4b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82c2f41769d84bd48bfc1bc7f504be4b already in progress: creating tablet
I20250115 03:06:15.613209 14823 ts_tablet_manager.cc:1905] T 82c2f41769d84bd48bfc1bc7f504be4b P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.613924 14811 tablet_bootstrap.cc:654] T 3e46d4a75e9b4ca8b83a188d06280fcd P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.613963 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f960fb76bf0d4e3288988b6a3abb2e8c" } new_table_name: "default.Table_e72d1ee69400410c9da9082d4bd8bc9a" modify_external_catalogs: true
I20250115 03:06:15.614639 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_968db1bba3cf48dbba6f4456bcfeb41a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.614944 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_968db1bba3cf48dbba6f4456bcfeb41a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.615164 14823 ts_tablet_manager.cc:1918] T 82c2f41769d84bd48bfc1bc7f504be4b P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.615227 14823 log.cc:1198] T 82c2f41769d84bd48bfc1bc7f504be4b P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/82c2f41769d84bd48bfc1bc7f504be4b
I20250115 03:06:15.615406 14823 ts_tablet_manager.cc:1939] T 82c2f41769d84bd48bfc1bc7f504be4b P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.615998 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_561cca4d810b4e73934fcee99a8994c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.616267 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_561cca4d810b4e73934fcee99a8994c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.619428 14812 tablet_bootstrap.cc:492] T 2d78a7793f84424bad42a1aba89eab92 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.619434 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 82c2f41769d84bd48bfc1bc7f504be4b (table default.Table_24cbe6b962b248a9b340506b63c41b20 [id=9866275e65ab4dfcbfee6bf209c3654e]) successfully deleted
I20250115 03:06:15.619503 14812 ts_tablet_manager.cc:1397] T 2d78a7793f84424bad42a1aba89eab92 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s
I20250115 03:06:15.619405 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_3fab7f59cc324dcf811b852f62ba1f9e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:15.619582 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_1c52ca26880a48289f5916fabdb05ac4" } modify_external_catalogs: true
I20250115 03:06:15.619657 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_1c52ca26880a48289f5916fabdb05ac4" } modify_external_catalogs: true
W20250115 03:06:15.619742 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3fab7f59cc324dcf811b852f62ba1f9e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.619889 14812 raft_consensus.cc:357] T 2d78a7793f84424bad42a1aba89eab92 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.619971 14812 raft_consensus.cc:383] T 2d78a7793f84424bad42a1aba89eab92 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.620002 14812 raft_consensus.cc:738] T 2d78a7793f84424bad42a1aba89eab92 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.620069 15453 consensus_queue.cc:1035] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.620096 14812 consensus_queue.cc:260] T 2d78a7793f84424bad42a1aba89eab92 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.620329 14812 ts_tablet_manager.cc:1428] T 2d78a7793f84424bad42a1aba89eab92 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.620476 14812 tablet_bootstrap.cc:492] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.621376 14812 tablet_bootstrap.cc:654] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.622507 14164 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet ccf6b58c55d747da89c1c218a70ce09f (table default.Table_cbfbacbaeee4488aba1a8be104138404 [id=e441d6f5acea4837accbfb8e4856f82d]). Expected version 1 got 0
I20250115 03:06:15.622897 14820 ts_tablet_manager.cc:1918] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.622959 14820 log.cc:1198] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/53248628d9584f8b886166c1e8ec5c3c
I20250115 03:06:15.623100 14171 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_6fadef5b204546918bd04347978ae2c2 [id=3bcfb6a13f2b4d56b801fcf210764fea] alter complete (version 2)
I20250115 03:06:15.624099 14820 ts_tablet_manager.cc:1939] T 53248628d9584f8b886166c1e8ec5c3c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.625151 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 53248628d9584f8b886166c1e8ec5c3c (table default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]) successfully deleted
I20250115 03:06:15.625594 14820 tablet_replica.cc:331] T 47b08f6e7dec4c5a871a49c79421209e P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.625818 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_632fc9d76fb3482c8a5e87feda0ceeed" } modify_external_catalogs: true
I20250115 03:06:15.625918 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_632fc9d76fb3482c8a5e87feda0ceeed" } modify_external_catalogs: true
I20250115 03:06:15.627441 14820 raft_consensus.cc:2238] T 47b08f6e7dec4c5a871a49c79421209e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.627548 14820 raft_consensus.cc:2267] T 47b08f6e7dec4c5a871a49c79421209e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:15.628082 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a2fc04950cfe4c608a4712afd257ca76 consensus state has no leader
I20250115 03:06:15.628963 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 82c2f41769d84bd48bfc1bc7f504be4b with delete_type TABLET_DATA_DELETED (replaced by tablet 80aa570e2c154172ad6da29d268bede7) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:15.630219 14162 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5986558350244e7aa4ddcd5cccb7d676 consensus state has no leader
W20250115 03:06:15.630656 14158 catalog_manager.cc:4839] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 82c2f41769d84bd48bfc1bc7f504be4b because the tablet was not found. No further retry: Not found: Tablet not found: 82c2f41769d84bd48bfc1bc7f504be4b
I20250115 03:06:15.630896 14691 raft_consensus.cc:1270] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:15.631239 15390 consensus_queue.cc:1035] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.631693 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 7d894471f7b94aec8a19df043906d7e7 (DEFAULT_TABLE table=default.Table_1a2ec424c5cb40fbb3c93f327531e36f [id=50a51914c37b462aa45a71fb260c809b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.633436 14554 raft_consensus.cc:3054] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.634269 14554 raft_consensus.cc:1270] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:15.634508 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 7d894471f7b94aec8a19df043906d7e7 (DEFAULT_TABLE table=default.Table_1a2ec424c5cb40fbb3c93f327531e36f [id=50a51914c37b462aa45a71fb260c809b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.634720 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d894471f7b94aec8a19df043906d7e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.634910 15617 tablet.cc:1721] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:15.636376 14393 tablet_service.cc:1467] Processing CreateTablet for tablet cff95c69fb9045cf9ec0ebb128c9ef6b (DEFAULT_TABLE table=default.Table_67f7434767174808bef25bc2c55394fd [id=8a02676ab3334b7c81ce22069b29551f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.636794 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet 33380937a3b7428995b06e477a896c02 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:15.636961 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 7d894471f7b94aec8a19df043906d7e7 (DEFAULT_TABLE table=default.Table_1a2ec424c5cb40fbb3c93f327531e36f [id=50a51914c37b462aa45a71fb260c809b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.637187 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d894471f7b94aec8a19df043906d7e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.637560 14667 tablet_service.cc:1467] Processing CreateTablet for tablet cff95c69fb9045cf9ec0ebb128c9ef6b (DEFAULT_TABLE table=default.Table_67f7434767174808bef25bc2c55394fd [id=8a02676ab3334b7c81ce22069b29551f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.637661 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d894471f7b94aec8a19df043906d7e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.637758 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cff95c69fb9045cf9ec0ebb128c9ef6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.639595 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cff95c69fb9045cf9ec0ebb128c9ef6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.640250 14861 consensus_queue.cc:1035] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.642117 14166 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b386f9bd2bc4416792a7480d1539e13d" } modify_external_catalogs: true
I20250115 03:06:15.642218 14166 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b386f9bd2bc4416792a7480d1539e13d" } modify_external_catalogs: true
I20250115 03:06:15.642424 15426 tablet.cc:1721] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:15.642481 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 2379bae3690c44949e658dde84c1712e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.642975 14823 tablet_replica.cc:331] T 2379bae3690c44949e658dde84c1712e P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.643108 14823 raft_consensus.cc:2238] T 2379bae3690c44949e658dde84c1712e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.643189 14823 raft_consensus.cc:2267] T 2379bae3690c44949e658dde84c1712e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.643149 14162 catalog_manager.cc:5526] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:15.643370 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 4ff51d514fb84809aa53fa9fc1010681 (table default.Table_9cc1022af4f14e7a8a8e11cc1803ca87 [id=98f48acd89a6461bb4511141cab4d88b]). Expected version 2 got 0
I20250115 03:06:15.643698 14823 ts_tablet_manager.cc:1905] T 2379bae3690c44949e658dde84c1712e P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.643728 14164 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_ef19dd8e16aa4bf9a97897ebc1c8eef5 [id=38bfb0375c334d16b97c7251d6ddd76b] alter complete (version 3)
I20250115 03:06:15.644109 15274 tablet.cc:1721] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:15.644507 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet b1644ba8616349dc9d10a790b6089571
I20250115 03:06:15.645673 14823 ts_tablet_manager.cc:1918] T 2379bae3690c44949e658dde84c1712e P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.645730 14823 log.cc:1198] T 2379bae3690c44949e658dde84c1712e P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/2379bae3690c44949e658dde84c1712e
I20250115 03:06:15.645871 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet b1644ba8616349dc9d10a790b6089571 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.645911 14823 ts_tablet_manager.cc:1939] T 2379bae3690c44949e658dde84c1712e P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.637194 14537 tablet_service.cc:1467] Processing CreateTablet for tablet cff95c69fb9045cf9ec0ebb128c9ef6b (DEFAULT_TABLE table=default.Table_67f7434767174808bef25bc2c55394fd [id=8a02676ab3334b7c81ce22069b29551f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.646231 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cff95c69fb9045cf9ec0ebb128c9ef6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.646925 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet b1644ba8616349dc9d10a790b6089571 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.647552 15153 raft_consensus.cc:491] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.647622 15153 raft_consensus.cc:513] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.647954 15153 leader_election.cc:290] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.648268 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2fc04950cfe4c608a4712afd257ca76" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:15.648439 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2fc04950cfe4c608a4712afd257ca76" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.648490 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet b1644ba8616349dc9d10a790b6089571 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:15.648555 14628 leader_election.cc:343] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:15.648741 14626 leader_election.cc:343] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.648795 14626 leader_election.cc:304] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:15.648954 15153 raft_consensus.cc:2743] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:15.648979 14823 tablet_replica.cc:331] T b1644ba8616349dc9d10a790b6089571 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.649082 14823 raft_consensus.cc:2238] T b1644ba8616349dc9d10a790b6089571 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
W20250115 03:06:15.649087 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet b1644ba8616349dc9d10a790b6089571 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b1644ba8616349dc9d10a790b6089571 already in progress: creating tablet
I20250115 03:06:15.649164 14823 raft_consensus.cc:2267] T b1644ba8616349dc9d10a790b6089571 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.649220 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 2379bae3690c44949e658dde84c1712e (table default.Table_ebca4429c164491c974b3ee244c3b1de [id=01a92c882c3640dcbc950b3e76f97c4b]) successfully deleted
I20250115 03:06:15.649394 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 1f2dc6928e2941ef9b5b5a7891376500 (DEFAULT_TABLE table=default.Table_652f8e06e005491381404a11759aefb1 [id=6fb6e311a0af4f51b190a880f84373d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.649580 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f2dc6928e2941ef9b5b5a7891376500. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.650004 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 1f2dc6928e2941ef9b5b5a7891376500 (DEFAULT_TABLE table=default.Table_652f8e06e005491381404a11759aefb1 [id=6fb6e311a0af4f51b190a880f84373d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.650197 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f2dc6928e2941ef9b5b5a7891376500. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.652347 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 1f2dc6928e2941ef9b5b5a7891376500 (DEFAULT_TABLE table=default.Table_652f8e06e005491381404a11759aefb1 [id=6fb6e311a0af4f51b190a880f84373d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.652522 14823 ts_tablet_manager.cc:1905] T b1644ba8616349dc9d10a790b6089571 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.652566 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f2dc6928e2941ef9b5b5a7891376500. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.653914 14812 tablet_bootstrap.cc:492] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.653992 14812 ts_tablet_manager.cc:1397] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.000s
I20250115 03:06:15.654387 14812 raft_consensus.cc:357] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.654474 14812 raft_consensus.cc:383] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.654505 14812 raft_consensus.cc:738] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.654603 14812 consensus_queue.cc:260] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.654856 14812 ts_tablet_manager.cc:1428] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.655004 14812 tablet_bootstrap.cc:492] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.655885 14812 tablet_bootstrap.cc:654] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.658087 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet d0c3536533e94174bb1e3888a3458395
I20250115 03:06:15.658922 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet d0c3536533e94174bb1e3888a3458395 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.659284 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet d0c3536533e94174bb1e3888a3458395 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.659693 14809 tablet_bootstrap.cc:492] T f3ac7f1e1bfb4383895dc2c2123a743b P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.659871 14809 ts_tablet_manager.cc:1397] T f3ac7f1e1bfb4383895dc2c2123a743b P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.061s user 0.002s sys 0.000s
W20250115 03:06:15.660061 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet d0c3536533e94174bb1e3888a3458395 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0c3536533e94174bb1e3888a3458395 already in progress: creating tablet
I20250115 03:06:15.660435 14809 raft_consensus.cc:357] T f3ac7f1e1bfb4383895dc2c2123a743b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.660511 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 72f9b43975074ac5a7012b132ef74662
I20250115 03:06:15.660599 14809 raft_consensus.cc:383] T f3ac7f1e1bfb4383895dc2c2123a743b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.660631 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet d0c3536533e94174bb1e3888a3458395 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.660672 14809 raft_consensus.cc:738] T f3ac7f1e1bfb4383895dc2c2123a743b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.660812 14809 consensus_queue.cc:260] T f3ac7f1e1bfb4383895dc2c2123a743b P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.660952 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 33380937a3b7428995b06e477a896c02
I20250115 03:06:15.661181 14168 catalog_manager.cc:6596] ReplaceTablet: tablet 33380937a3b7428995b06e477a896c02 deleted and replaced by tablet c251960c7c9a44429102a294f35f925c
I20250115 03:06:15.661397 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 33380937a3b7428995b06e477a896c02 with delete_type TABLET_DATA_DELETED (replaced by tablet c251960c7c9a44429102a294f35f925c) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.661514 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 72f9b43975074ac5a7012b132ef74662 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.661731 14809 ts_tablet_manager.cc:1428] T f3ac7f1e1bfb4383895dc2c2123a743b P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:15.661916 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 33380937a3b7428995b06e477a896c02 with delete_type TABLET_DATA_DELETED (replaced by tablet c251960c7c9a44429102a294f35f925c) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.662025 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 72f9b43975074ac5a7012b132ef74662 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.662146 14819 tablet_replica.cc:331] T 72f9b43975074ac5a7012b132ef74662 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.662251 14819 raft_consensus.cc:2238] T 72f9b43975074ac5a7012b132ef74662 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.662330 14819 raft_consensus.cc:2267] T 72f9b43975074ac5a7012b132ef74662 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.663213 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 72f9b43975074ac5a7012b132ef74662 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.663313 14809 tablet_bootstrap.cc:492] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.663401 14163 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_cbfbacbaeee4488aba1a8be104138404 [id=e441d6f5acea4837accbfb8e4856f82d] alter complete (version 1)
I20250115 03:06:15.663563 14819 ts_tablet_manager.cc:1905] T 72f9b43975074ac5a7012b132ef74662 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.659493 14811 tablet_bootstrap.cc:492] T 3e46d4a75e9b4ca8b83a188d06280fcd P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.664223 14811 ts_tablet_manager.cc:1397] T 3e46d4a75e9b4ca8b83a188d06280fcd P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.052s user 0.001s sys 0.000s
I20250115 03:06:15.664238 14809 tablet_bootstrap.cc:654] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.664665 14811 raft_consensus.cc:357] T 3e46d4a75e9b4ca8b83a188d06280fcd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.664734 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 33380937a3b7428995b06e477a896c02 with delete_type TABLET_DATA_DELETED (replaced by tablet c251960c7c9a44429102a294f35f925c) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.664767 14811 raft_consensus.cc:383] T 3e46d4a75e9b4ca8b83a188d06280fcd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.664795 14811 raft_consensus.cc:738] T 3e46d4a75e9b4ca8b83a188d06280fcd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.664906 14811 consensus_queue.cc:260] T 3e46d4a75e9b4ca8b83a188d06280fcd P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.665192 14811 ts_tablet_manager.cc:1428] T 3e46d4a75e9b4ca8b83a188d06280fcd P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.665349 14811 tablet_bootstrap.cc:492] T f01c319ebd5148858de9ea1a608ddff3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.665613 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet a1cbe29c08c7477e8944b7e108d46e7a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.666329 14811 tablet_bootstrap.cc:654] T f01c319ebd5148858de9ea1a608ddff3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.666563 15153 raft_consensus.cc:491] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.666635 15153 raft_consensus.cc:513] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.666970 15153 leader_election.cc:290] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.667285 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 89d92f59c7424dfc95427329200455e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.667399 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet e3302d62a34e43d09c11abca292d9110 with delete_type TABLET_DATA_DELETED (replaced by tablet a4484651d964411e90ed93c6caab0a13) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:15.668479 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 33380937a3b7428995b06e477a896c02 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33380937a3b7428995b06e477a896c02 already in progress: creating tablet
I20250115 03:06:15.668536 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 99d9180ed9b24363a4c44d01ea1d6e9b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.668726 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3ac7f1e1bfb4383895dc2c2123a743b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:15.668800 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 2379bae3690c44949e658dde84c1712e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.668839 14421 raft_consensus.cc:2463] T f3ac7f1e1bfb4383895dc2c2123a743b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:15.669050 14820 ts_tablet_manager.cc:1905] T 47b08f6e7dec4c5a871a49c79421209e P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.669481 14823 ts_tablet_manager.cc:1918] T b1644ba8616349dc9d10a790b6089571 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.669546 14823 log.cc:1198] T b1644ba8616349dc9d10a790b6089571 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/b1644ba8616349dc9d10a790b6089571
I20250115 03:06:15.669734 14823 ts_tablet_manager.cc:1939] T b1644ba8616349dc9d10a790b6089571 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.670086 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3ac7f1e1bfb4383895dc2c2123a743b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.670121 14628 leader_election.cc:304] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.670267 14823 tablet_replica.cc:331] T 72f9b43975074ac5a7012b132ef74662 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.671803 14819 ts_tablet_manager.cc:1918] T 72f9b43975074ac5a7012b132ef74662 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.671876 14819 log.cc:1198] T 72f9b43975074ac5a7012b132ef74662 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/72f9b43975074ac5a7012b132ef74662
I20250115 03:06:15.672103 14819 ts_tablet_manager.cc:1939] T 72f9b43975074ac5a7012b132ef74662 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.670608 15153 raft_consensus.cc:2798] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:15.672219 15153 raft_consensus.cc:491] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:15.672286 15153 raft_consensus.cc:3054] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.672600 14819 tablet_replica.cc:331] T 99d9180ed9b24363a4c44d01ea1d6e9b P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.672701 14819 raft_consensus.cc:2238] T 99d9180ed9b24363a4c44d01ea1d6e9b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.672787 14819 raft_consensus.cc:2267] T 99d9180ed9b24363a4c44d01ea1d6e9b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:15.671021 14626 leader_election.cc:343] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.673184 15153 raft_consensus.cc:513] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.673496 15153 leader_election.cc:290] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.673712 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_5a68d6c8c53b4746a7123860185fe0cc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.674041 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a68d6c8c53b4746a7123860185fe0cc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.674017 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_7f271037abae4e2a979c2358897fc66f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:15.674088 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d78a7793f84424bad42a1aba89eab92 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:15.674309 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7f271037abae4e2a979c2358897fc66f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.674554 14819 ts_tablet_manager.cc:1905] T 99d9180ed9b24363a4c44d01ea1d6e9b P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.674592 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3ac7f1e1bfb4383895dc2c2123a743b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:15.674929 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3ac7f1e1bfb4383895dc2c2123a743b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:15.675027 14421 raft_consensus.cc:3054] T f3ac7f1e1bfb4383895dc2c2123a743b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:15.675411 14626 leader_election.cc:343] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.673115 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet b1644ba8616349dc9d10a790b6089571 (table default.Table_632fc9d76fb3482c8a5e87feda0ceeed [id=dd1bb6f7163f46748711bc33cc8a3f85]) successfully deleted
W20250115 03:06:15.675698 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet d0c3536533e94174bb1e3888a3458395 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0c3536533e94174bb1e3888a3458395 already in progress: creating tablet
I20250115 03:06:15.675827 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 72f9b43975074ac5a7012b132ef74662 (table default.Table_b386f9bd2bc4416792a7480d1539e13d [id=4ae12906873e4ed59c179ec32ae13fa4]) successfully deleted
I20250115 03:06:15.675846 14421 raft_consensus.cc:2463] T f3ac7f1e1bfb4383895dc2c2123a743b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:15.676101 14628 leader_election.cc:304] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:15.676296 15153 raft_consensus.cc:2798] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:15.676367 15153 raft_consensus.cc:695] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:15.676502 15153 consensus_queue.cc:237] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.677937 14819 ts_tablet_manager.cc:1918] T 99d9180ed9b24363a4c44d01ea1d6e9b P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.677991 14812 tablet_bootstrap.cc:492] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.678002 14819 log.cc:1198] T 99d9180ed9b24363a4c44d01ea1d6e9b P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/99d9180ed9b24363a4c44d01ea1d6e9b
I20250115 03:06:15.678066 14812 ts_tablet_manager.cc:1397] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s
I20250115 03:06:15.678185 14819 ts_tablet_manager.cc:1939] T 99d9180ed9b24363a4c44d01ea1d6e9b P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.678485 14812 raft_consensus.cc:357] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.678578 14812 raft_consensus.cc:383] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.678609 14812 raft_consensus.cc:738] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.678710 14812 consensus_queue.cc:260] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.678910 14812 ts_tablet_manager.cc:1428] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:15.679375 14820 ts_tablet_manager.cc:1918] T 47b08f6e7dec4c5a871a49c79421209e P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.679435 14820 log.cc:1198] T 47b08f6e7dec4c5a871a49c79421209e P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/47b08f6e7dec4c5a871a49c79421209e
I20250115 03:06:15.679622 14820 ts_tablet_manager.cc:1939] T 47b08f6e7dec4c5a871a49c79421209e P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.679987 15453 raft_consensus.cc:491] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.680056 15453 raft_consensus.cc:513] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.680126 14823 raft_consensus.cc:2238] T 72f9b43975074ac5a7012b132ef74662 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.680222 14823 raft_consensus.cc:2267] T 72f9b43975074ac5a7012b132ef74662 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.680323 14811 tablet_bootstrap.cc:492] T f01c319ebd5148858de9ea1a608ddff3 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.680347 15453 leader_election.cc:290] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.680390 14811 ts_tablet_manager.cc:1397] T f01c319ebd5148858de9ea1a608ddff3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20250115 03:06:15.680791 14811 raft_consensus.cc:357] T f01c319ebd5148858de9ea1a608ddff3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.680897 14811 raft_consensus.cc:383] T f01c319ebd5148858de9ea1a608ddff3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.680951 14811 raft_consensus.cc:738] T f01c319ebd5148858de9ea1a608ddff3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.680958 14823 ts_tablet_manager.cc:1905] T 72f9b43975074ac5a7012b132ef74662 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.680960 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5af8b81e7ff4acab5aef13003656988" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:15.681260 14628 leader_election.cc:343] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.681798 14812 tablet_bootstrap.cc:492] T 68974a1815cb482d8b28d54f42634608 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.682687 14812 tablet_bootstrap.cc:654] T 68974a1815cb482d8b28d54f42634608 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.683271 14820 tablet_replica.cc:331] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.683753 14809 tablet_bootstrap.cc:492] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.683874 14809 ts_tablet_manager.cc:1397] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20250115 03:06:15.684301 14809 raft_consensus.cc:357] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.684468 14809 raft_consensus.cc:383] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.684501 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 47b08f6e7dec4c5a871a49c79421209e (table default.Table_f4d365d35e3c43e79d3ac20e438e7aa5 [id=835a8f488a7c42af850632b1286f9509]) successfully deleted
I20250115 03:06:15.684543 14809 raft_consensus.cc:738] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
W20250115 03:06:15.684649 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 82c2f41769d84bd48bfc1bc7f504be4b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82c2f41769d84bd48bfc1bc7f504be4b already in progress: creating tablet
W20250115 03:06:15.684770 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet b1644ba8616349dc9d10a790b6089571 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b1644ba8616349dc9d10a790b6089571 already in progress: creating tablet
I20250115 03:06:15.684804 14809 consensus_queue.cc:260] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:15.684887 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet d0c3536533e94174bb1e3888a3458395 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d0c3536533e94174bb1e3888a3458395 already in progress: creating tablet
W20250115 03:06:15.685040 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 33380937a3b7428995b06e477a896c02 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33380937a3b7428995b06e477a896c02 already in progress: creating tablet
I20250115 03:06:15.685999 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 99d9180ed9b24363a4c44d01ea1d6e9b (table default.Table_6bb3a06c1ae749e593cdc3f2b3992eaf [id=89578bd1e7e64c0fa550f20f592ccc6f]) successfully deleted
I20250115 03:06:15.686167 14819 tablet_replica.cc:331] T 2379bae3690c44949e658dde84c1712e P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.686267 14819 raft_consensus.cc:2238] T 2379bae3690c44949e658dde84c1712e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.686338 14819 raft_consensus.cc:2267] T 2379bae3690c44949e658dde84c1712e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.686749 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_95ad0daf75e040a28fa088bf49371f95"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:15.686839 14809 ts_tablet_manager.cc:1428] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
W20250115 03:06:15.687075 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_95ad0daf75e040a28fa088bf49371f95 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.687532 14809 tablet_bootstrap.cc:492] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.687664 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet a2f815e6fe4e4936823d51fcd16b453b from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:15.688532 14809 tablet_bootstrap.cc:654] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.689424 14809 tablet_bootstrap.cc:492] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.689543 14809 ts_tablet_manager.cc:1397] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20250115 03:06:15.689934 14819 ts_tablet_manager.cc:1905] T 2379bae3690c44949e658dde84c1712e P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.690723 14809 raft_consensus.cc:357] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.690955 14809 raft_consensus.cc:383] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.691021 14809 raft_consensus.cc:738] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.691180 14809 consensus_queue.cc:260] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.691640 14809 ts_tablet_manager.cc:1428] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:15.691843 14809 tablet_bootstrap.cc:492] T 17b5e362481c49d2a3a13b1550ceef23 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.692862 14809 tablet_bootstrap.cc:654] T 17b5e362481c49d2a3a13b1550ceef23 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.693475 14819 ts_tablet_manager.cc:1918] T 2379bae3690c44949e658dde84c1712e P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.693539 14819 log.cc:1198] T 2379bae3690c44949e658dde84c1712e P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/2379bae3690c44949e658dde84c1712e
I20250115 03:06:15.693717 14819 ts_tablet_manager.cc:1939] T 2379bae3690c44949e658dde84c1712e P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.694240 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 2379bae3690c44949e658dde84c1712e (table default.Table_ebca4429c164491c974b3ee244c3b1de [id=01a92c882c3640dcbc950b3e76f97c4b]) successfully deleted
I20250115 03:06:15.684548 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5af8b81e7ff4acab5aef13003656988" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.690079 14811 consensus_queue.cc:260] T f01c319ebd5148858de9ea1a608ddff3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:15.694758 14626 leader_election.cc:343] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.694830 14626 leader_election.cc:304] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:15.695236 15153 raft_consensus.cc:2743] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:15.695384 14811 ts_tablet_manager.cc:1428] T f01c319ebd5148858de9ea1a608ddff3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.015s user 0.001s sys 0.000s
I20250115 03:06:15.695992 14823 ts_tablet_manager.cc:1918] T 72f9b43975074ac5a7012b132ef74662 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.696060 14823 log.cc:1198] T 72f9b43975074ac5a7012b132ef74662 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/72f9b43975074ac5a7012b132ef74662
I20250115 03:06:15.696264 14823 ts_tablet_manager.cc:1939] T 72f9b43975074ac5a7012b132ef74662 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.696821 14823 tablet_replica.cc:331] T 2d78a7793f84424bad42a1aba89eab92 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.696869 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 72f9b43975074ac5a7012b132ef74662 (table default.Table_b386f9bd2bc4416792a7480d1539e13d [id=4ae12906873e4ed59c179ec32ae13fa4]) successfully deleted
W20250115 03:06:15.697050 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 33380937a3b7428995b06e477a896c02 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 33380937a3b7428995b06e477a896c02 already in progress: creating tablet
I20250115 03:06:15.698207 14811 tablet_bootstrap.cc:492] T e368f87556944c46a6d8f160ead17ac9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.698361 15390 raft_consensus.cc:491] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.698439 15390 raft_consensus.cc:513] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.698740 15390 leader_election.cc:290] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:15.699105 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d4b8d4ed9f44112b83c6ff94aee2206" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.699211 14554 raft_consensus.cc:2463] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:15.700012 14356 leader_election.cc:304] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.700140 14811 tablet_bootstrap.cc:654] T e368f87556944c46a6d8f160ead17ac9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.700153 15390 raft_consensus.cc:2798] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:15.700201 15390 raft_consensus.cc:491] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:15.700232 15390 raft_consensus.cc:3054] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.700677 14820 raft_consensus.cc:2238] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:15.700682 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet f978e85cf659450ca0039368d58b49e5 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:15.700863 14820 raft_consensus.cc:2267] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.701017 15390 raft_consensus.cc:513] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.701346 15390 leader_election.cc:290] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:15.701242 14820 ts_tablet_manager.cc:1905] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.701548 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d4b8d4ed9f44112b83c6ff94aee2206" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:15.701637 14554 raft_consensus.cc:3054] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.702406 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet f01c319ebd5148858de9ea1a608ddff3 with delete_type TABLET_DATA_DELETED (replaced by tablet 84ce55f729224f78b420e57977419add) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.702414 14554 raft_consensus.cc:2463] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:15.702805 14356 leader_election.cc:304] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.702978 15390 raft_consensus.cc:2798] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:15.703063 15390 raft_consensus.cc:695] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:15.703158 15390 consensus_queue.cc:237] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.699051 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d4b8d4ed9f44112b83c6ff94aee2206" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:15.701689 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d4b8d4ed9f44112b83c6ff94aee2206" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:15.704442 14691 raft_consensus.cc:2463] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:15.704977 14823 raft_consensus.cc:2238] T 2d78a7793f84424bad42a1aba89eab92 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.705231 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_eec42b37d6de49ccad194b13edd928e3" } modify_external_catalogs: true
I20250115 03:06:15.705336 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_eec42b37d6de49ccad194b13edd928e3" } modify_external_catalogs: true
I20250115 03:06:15.706332 14820 ts_tablet_manager.cc:1918] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.706398 14820 log.cc:1198] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/72f9b43975074ac5a7012b132ef74662
I20250115 03:06:15.706626 14820 ts_tablet_manager.cc:1939] T 72f9b43975074ac5a7012b132ef74662 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.706833 14823 raft_consensus.cc:2267] T 2d78a7793f84424bad42a1aba89eab92 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.707262 14823 ts_tablet_manager.cc:1905] T 2d78a7793f84424bad42a1aba89eab92 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.707753 14820 tablet_replica.cc:331] T a1cbe29c08c7477e8944b7e108d46e7a P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.707834 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 72f9b43975074ac5a7012b132ef74662 (table default.Table_b386f9bd2bc4416792a7480d1539e13d [id=4ae12906873e4ed59c179ec32ae13fa4]) successfully deleted
I20250115 03:06:15.707952 14820 raft_consensus.cc:2238] T a1cbe29c08c7477e8944b7e108d46e7a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.708077 14820 raft_consensus.cc:2267] T a1cbe29c08c7477e8944b7e108d46e7a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.708510 14820 ts_tablet_manager.cc:1905] T a1cbe29c08c7477e8944b7e108d46e7a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.708712 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet a2f815e6fe4e4936823d51fcd16b453b
I20250115 03:06:15.709158 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet a2f815e6fe4e4936823d51fcd16b453b with delete_type TABLET_DATA_DELETED (replaced by tablet e2c6db404348466ea5211ffa69c8f01c) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.709563 14169 catalog_manager.cc:6596] ReplaceTablet: tablet a2f815e6fe4e4936823d51fcd16b453b deleted and replaced by tablet e2c6db404348466ea5211ffa69c8f01c
I20250115 03:06:15.709692 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet a2f815e6fe4e4936823d51fcd16b453b with delete_type TABLET_DATA_DELETED (replaced by tablet e2c6db404348466ea5211ffa69c8f01c) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.709376 14823 ts_tablet_manager.cc:1918] T 2d78a7793f84424bad42a1aba89eab92 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.710023 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet a2f815e6fe4e4936823d51fcd16b453b with delete_type TABLET_DATA_DELETED (replaced by tablet e2c6db404348466ea5211ffa69c8f01c) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.710050 14823 log.cc:1198] T 2d78a7793f84424bad42a1aba89eab92 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/2d78a7793f84424bad42a1aba89eab92
I20250115 03:06:15.709935 14819 tablet_replica.cc:331] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.710237 14823 ts_tablet_manager.cc:1939] T 2d78a7793f84424bad42a1aba89eab92 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.710228 14819 raft_consensus.cc:2238] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.710330 14819 raft_consensus.cc:2267] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.710775 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 2d78a7793f84424bad42a1aba89eab92 (table default.Table_638e39bcf2aa4131ac57afbe63ac9b73 [id=50928432524e42d79855e9143c537b61]) successfully deleted
I20250115 03:06:15.710812 14819 ts_tablet_manager.cc:1905] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.710978 14823 tablet_replica.cc:331] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.711099 14823 raft_consensus.cc:2238] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:15.711256 14823 raft_consensus.cc:2267] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.711758 14823 ts_tablet_manager.cc:1905] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.712597 14819 ts_tablet_manager.cc:1918] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20250115 03:06:15.712670 14819 log.cc:1198] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/a2f815e6fe4e4936823d51fcd16b453b
I20250115 03:06:15.712896 14819 ts_tablet_manager.cc:1939] T a2f815e6fe4e4936823d51fcd16b453b P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.713487 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet a2f815e6fe4e4936823d51fcd16b453b (table default.Table_ef19dd8e16aa4bf9a97897ebc1c8eef5 [id=38bfb0375c334d16b97c7251d6ddd76b]) successfully deleted
I20250115 03:06:15.713770 14823 ts_tablet_manager.cc:1918] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20250115 03:06:15.713873 14823 log.cc:1198] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/a2f815e6fe4e4936823d51fcd16b453b
I20250115 03:06:15.714147 14823 ts_tablet_manager.cc:1939] T a2f815e6fe4e4936823d51fcd16b453b P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.714674 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet a2f815e6fe4e4936823d51fcd16b453b (table default.Table_ef19dd8e16aa4bf9a97897ebc1c8eef5 [id=38bfb0375c334d16b97c7251d6ddd76b]) successfully deleted
I20250115 03:06:15.715056 14820 ts_tablet_manager.cc:1918] T a1cbe29c08c7477e8944b7e108d46e7a P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.715128 14820 log.cc:1198] T a1cbe29c08c7477e8944b7e108d46e7a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/a1cbe29c08c7477e8944b7e108d46e7a
I20250115 03:06:15.715350 14820 ts_tablet_manager.cc:1939] T a1cbe29c08c7477e8944b7e108d46e7a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.715893 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet a1cbe29c08c7477e8944b7e108d46e7a (table default.Table_7e53fa73287c43e3a9a0e65908b42593 [id=1f983a10a8704e22b8a2b96ef4e06fbc]) successfully deleted
I20250115 03:06:15.715934 14820 tablet_replica.cc:331] T 89d92f59c7424dfc95427329200455e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.716109 14820 raft_consensus.cc:2238] T 89d92f59c7424dfc95427329200455e7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.716255 14820 raft_consensus.cc:2267] T 89d92f59c7424dfc95427329200455e7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.716341 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet f978e85cf659450ca0039368d58b49e5
I20250115 03:06:15.716627 14171 catalog_manager.cc:6596] ReplaceTablet: tablet f978e85cf659450ca0039368d58b49e5 deleted and replaced by tablet da95daa40c5b4615a0219f6d8e3770f6
I20250115 03:06:15.716856 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet f978e85cf659450ca0039368d58b49e5 with delete_type TABLET_DATA_DELETED (replaced by tablet da95daa40c5b4615a0219f6d8e3770f6) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.717198 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet f978e85cf659450ca0039368d58b49e5 with delete_type TABLET_DATA_DELETED (replaced by tablet da95daa40c5b4615a0219f6d8e3770f6) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.717370 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet f978e85cf659450ca0039368d58b49e5 with delete_type TABLET_DATA_DELETED (replaced by tablet da95daa40c5b4615a0219f6d8e3770f6) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.717696 14819 tablet_replica.cc:331] T f978e85cf659450ca0039368d58b49e5 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.717810 14819 raft_consensus.cc:2238] T f978e85cf659450ca0039368d58b49e5 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.717905 14819 raft_consensus.cc:2267] T f978e85cf659450ca0039368d58b49e5 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.718344 14819 ts_tablet_manager.cc:1905] T f978e85cf659450ca0039368d58b49e5 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.718366 14812 tablet_bootstrap.cc:492] T 68974a1815cb482d8b28d54f42634608 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.718431 14812 ts_tablet_manager.cc:1397] T 68974a1815cb482d8b28d54f42634608 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.037s user 0.001s sys 0.000s
I20250115 03:06:15.718830 14812 raft_consensus.cc:357] T 68974a1815cb482d8b28d54f42634608 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.718928 14812 raft_consensus.cc:383] T 68974a1815cb482d8b28d54f42634608 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.718958 14812 raft_consensus.cc:738] T 68974a1815cb482d8b28d54f42634608 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.719058 14812 consensus_queue.cc:260] T 68974a1815cb482d8b28d54f42634608 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.719286 14812 ts_tablet_manager.cc:1428] T 68974a1815cb482d8b28d54f42634608 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.719458 14812 tablet_bootstrap.cc:492] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.720315 14819 ts_tablet_manager.cc:1918] T f978e85cf659450ca0039368d58b49e5 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.720348 14809 tablet_bootstrap.cc:492] T 17b5e362481c49d2a3a13b1550ceef23 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.720371 14819 log.cc:1198] T f978e85cf659450ca0039368d58b49e5 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/f978e85cf659450ca0039368d58b49e5
I20250115 03:06:15.720405 14809 ts_tablet_manager.cc:1397] T 17b5e362481c49d2a3a13b1550ceef23 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.000s
I20250115 03:06:15.720438 14812 tablet_bootstrap.cc:654] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.720587 14819 ts_tablet_manager.cc:1939] T f978e85cf659450ca0039368d58b49e5 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.720724 14809 raft_consensus.cc:357] T 17b5e362481c49d2a3a13b1550ceef23 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.720794 14809 raft_consensus.cc:383] T 17b5e362481c49d2a3a13b1550ceef23 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.720813 14809 raft_consensus.cc:738] T 17b5e362481c49d2a3a13b1550ceef23 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.721056 14823 tablet_replica.cc:331] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.721134 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet f978e85cf659450ca0039368d58b49e5 (table default.Table_4bde77af131940a099eabda16fa2ccb2 [id=4f952e57c86746f6bb1df72e13589832]) successfully deleted
I20250115 03:06:15.721158 14823 raft_consensus.cc:2238] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:15.720944 14809 consensus_queue.cc:260] T 17b5e362481c49d2a3a13b1550ceef23 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.721314 14823 raft_consensus.cc:2267] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.721360 14812 tablet_bootstrap.cc:492] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.721429 14812 ts_tablet_manager.cc:1397] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:15.721823 14812 raft_consensus.cc:357] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.721910 14812 raft_consensus.cc:383] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.721936 14812 raft_consensus.cc:738] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.721961 14820 ts_tablet_manager.cc:1905] T 89d92f59c7424dfc95427329200455e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.722834 14809 ts_tablet_manager.cc:1428] T 17b5e362481c49d2a3a13b1550ceef23 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:15.722062 14812 consensus_queue.cc:260] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.723114 14809 tablet_bootstrap.cc:492] T 2d78a7793f84424bad42a1aba89eab92 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.723435 14812 ts_tablet_manager.cc:1428] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:15.723592 14812 tablet_bootstrap.cc:492] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.724002 14809 tablet_bootstrap.cc:654] T 2d78a7793f84424bad42a1aba89eab92 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.724095 14820 ts_tablet_manager.cc:1918] T 89d92f59c7424dfc95427329200455e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.724153 14820 log.cc:1198] T 89d92f59c7424dfc95427329200455e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/89d92f59c7424dfc95427329200455e7
I20250115 03:06:15.724326 14820 ts_tablet_manager.cc:1939] T 89d92f59c7424dfc95427329200455e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.724424 14812 tablet_bootstrap.cc:654] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.724880 14820 tablet_replica.cc:331] T e3302d62a34e43d09c11abca292d9110 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.725068 14820 raft_consensus.cc:2238] T e3302d62a34e43d09c11abca292d9110 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.725468 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 89d92f59c7424dfc95427329200455e7 (table default.Table_236770d2959743108285eae378066e45 [id=fec89f498f5046839fe491caa3367a36]) successfully deleted
I20250115 03:06:15.725595 14823 ts_tablet_manager.cc:1905] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.725811 14166 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 22cc0833bfde458ebe3985c05d29b8e6 (table default.Table_3db38338e5b04298954aded6d5482169 [id=f56a0de6e56441bfa620902f175feb4a]). Expected version 2 got 0
I20250115 03:06:15.725200 14820 raft_consensus.cc:2267] T e3302d62a34e43d09c11abca292d9110 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.727780 14820 ts_tablet_manager.cc:1905] T e3302d62a34e43d09c11abca292d9110 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.728142 14823 ts_tablet_manager.cc:1918] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.728214 14823 log.cc:1198] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/f978e85cf659450ca0039368d58b49e5
I20250115 03:06:15.728444 14823 ts_tablet_manager.cc:1939] T f978e85cf659450ca0039368d58b49e5 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.729048 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_490cb72d634f46eaa4130e320f581469" } modify_external_catalogs: true
I20250115 03:06:15.729143 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_490cb72d634f46eaa4130e320f581469" } modify_external_catalogs: true
I20250115 03:06:15.729522 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8ecc7de5c6b3497fb2d931c10ec6b373" } new_table_name: "default.Table_590b26746c6a4006bb430ba9dcbacafc" modify_external_catalogs: true
I20250115 03:06:15.729933 15390 consensus_queue.cc:1035] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.730777 14820 ts_tablet_manager.cc:1918] T e3302d62a34e43d09c11abca292d9110 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.731362 14820 log.cc:1198] T e3302d62a34e43d09c11abca292d9110 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/e3302d62a34e43d09c11abca292d9110
I20250115 03:06:15.731884 14820 ts_tablet_manager.cc:1939] T e3302d62a34e43d09c11abca292d9110 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.732465 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet e3302d62a34e43d09c11abca292d9110 (table default.Table_2db28e1f2ea5450092ed00f593c4939f [id=6fe98239ee3346df8b25712aaec67b82]) successfully deleted
I20250115 03:06:15.732686 14820 tablet_replica.cc:331] T f01c319ebd5148858de9ea1a608ddff3 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.732833 14820 raft_consensus.cc:2238] T f01c319ebd5148858de9ea1a608ddff3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
W20250115 03:06:15.733055 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet eeff2cd0825c4419aacfc73ebe267a8e consensus state has no leader
I20250115 03:06:15.733047 14820 raft_consensus.cc:2267] T f01c319ebd5148858de9ea1a608ddff3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.733304 15153 raft_consensus.cc:491] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.733378 15153 raft_consensus.cc:513] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.733664 15153 leader_election.cc:290] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.733690 14820 ts_tablet_manager.cc:1905] T f01c319ebd5148858de9ea1a608ddff3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.734004 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e594ea5cff81475886cdceb8400645a6" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.734241 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e594ea5cff81475886cdceb8400645a6" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:15.734318 14626 leader_election.cc:343] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.734515 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet f978e85cf659450ca0039368d58b49e5 (table default.Table_4bde77af131940a099eabda16fa2ccb2 [id=4f952e57c86746f6bb1df72e13589832]) successfully deleted
W20250115 03:06:15.735026 14628 leader_election.cc:343] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.735088 14628 leader_election.cc:304] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:15.735419 15153 raft_consensus.cc:2743] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:15.735972 14820 ts_tablet_manager.cc:1918] T f01c319ebd5148858de9ea1a608ddff3 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.736085 14820 log.cc:1198] T f01c319ebd5148858de9ea1a608ddff3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/f01c319ebd5148858de9ea1a608ddff3
I20250115 03:06:15.736320 14820 ts_tablet_manager.cc:1939] T f01c319ebd5148858de9ea1a608ddff3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.736346 14809 tablet_bootstrap.cc:492] T 2d78a7793f84424bad42a1aba89eab92 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.736408 14809 ts_tablet_manager.cc:1397] T 2d78a7793f84424bad42a1aba89eab92 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20250115 03:06:15.736543 14166 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 22cc0833bfde458ebe3985c05d29b8e6 (table default.Table_3db38338e5b04298954aded6d5482169 [id=f56a0de6e56441bfa620902f175feb4a]). Expected version 2 got 0
I20250115 03:06:15.736651 14166 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet a2fc04950cfe4c608a4712afd257ca76 (table default.Table_6b1f81ee12e04acda4bcf7d7f289a67a [id=8f811bda53fc4d76a88a3dcf69aa3dc6]). Expected version 1 got 0
I20250115 03:06:15.736752 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet f01c319ebd5148858de9ea1a608ddff3 (table default.Table_ff21d7bc463d43cfa3330f7288ba7a55 [id=5de066e9f2174fff9303cba23eeec988]) successfully deleted
I20250115 03:06:15.736748 14166 catalog_manager.cc:5526] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:15.736789 14809 raft_consensus.cc:357] T 2d78a7793f84424bad42a1aba89eab92 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.736884 14809 raft_consensus.cc:383] T 2d78a7793f84424bad42a1aba89eab92 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.736910 14809 raft_consensus.cc:738] T 2d78a7793f84424bad42a1aba89eab92 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.736948 14820 tablet_replica.cc:331] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.737046 14820 raft_consensus.cc:2238] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.737025 14809 consensus_queue.cc:260] T 2d78a7793f84424bad42a1aba89eab92 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.737134 14820 raft_consensus.cc:2267] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.737244 14809 ts_tablet_manager.cc:1428] T 2d78a7793f84424bad42a1aba89eab92 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:15.737401 14809 tablet_bootstrap.cc:492] T b1644ba8616349dc9d10a790b6089571 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.737625 14820 ts_tablet_manager.cc:1905] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.738441 14809 tablet_bootstrap.cc:654] T b1644ba8616349dc9d10a790b6089571 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.739782 14812 tablet_bootstrap.cc:492] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.739858 14812 ts_tablet_manager.cc:1397] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20250115 03:06:15.740267 14812 raft_consensus.cc:357] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.740367 14812 raft_consensus.cc:383] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.740402 14812 raft_consensus.cc:738] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.740507 14812 consensus_queue.cc:260] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.740760 14812 ts_tablet_manager.cc:1428] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:15.740938 14812 tablet_bootstrap.cc:492] T 9808c7f96a3147848da5f78948895699 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.741044 14820 ts_tablet_manager.cc:1918] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20250115 03:06:15.741582 14820 log.cc:1198] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/a2f815e6fe4e4936823d51fcd16b453b
I20250115 03:06:15.741762 14812 tablet_bootstrap.cc:654] T 9808c7f96a3147848da5f78948895699 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.741982 14820 ts_tablet_manager.cc:1939] T a2f815e6fe4e4936823d51fcd16b453b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.742368 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet a2f815e6fe4e4936823d51fcd16b453b (table default.Table_ef19dd8e16aa4bf9a97897ebc1c8eef5 [id=38bfb0375c334d16b97c7251d6ddd76b]) successfully deleted
I20250115 03:06:15.742714 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 01aa12e3d8fa463cbaee1287e7b74dd4
I20250115 03:06:15.743237 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 01aa12e3d8fa463cbaee1287e7b74dd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.743284 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 01aa12e3d8fa463cbaee1287e7b74dd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.743377 14819 tablet_replica.cc:331] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.743496 14819 raft_consensus.cc:2238] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:15.743520 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_98f4dad994f5494daf0f44483bb15e33"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:15.743649 14819 raft_consensus.cc:2267] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.743789 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 01aa12e3d8fa463cbaee1287e7b74dd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:15.743827 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_98f4dad994f5494daf0f44483bb15e33 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.744557 14819 ts_tablet_manager.cc:1905] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.745116 14168 catalog_manager.cc:5526] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:15.746431 14819 ts_tablet_manager.cc:1918] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.746501 14819 log.cc:1198] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/01aa12e3d8fa463cbaee1287e7b74dd4
I20250115 03:06:15.746711 14819 ts_tablet_manager.cc:1939] T 01aa12e3d8fa463cbaee1287e7b74dd4 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.747319 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 01aa12e3d8fa463cbaee1287e7b74dd4 (table default.Table_490cb72d634f46eaa4130e320f581469 [id=fe70549b0af7478ea3ddd2b658fb42fc]) successfully deleted
I20250115 03:06:15.747356 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 3b9c09b6315345039cdea9eeee3f181a (DEFAULT_TABLE table=default.Table_561cca4d810b4e73934fcee99a8994c7 [id=270d1081a9a04f09b76df0f45cfbbff6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.747574 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b9c09b6315345039cdea9eeee3f181a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.749882 14393 tablet_service.cc:1467] Processing CreateTablet for tablet d5655935aabd48a19d0ef03739e089a4 (DEFAULT_TABLE table=default.Table_968db1bba3cf48dbba6f4456bcfeb41a [id=ed8713b68f234c46bc2ef1dae0059a2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.750087 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5655935aabd48a19d0ef03739e089a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.751780 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 5d27310f8433411a85bf88d05a81bcf0
I20250115 03:06:15.752607 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d27310f8433411a85bf88d05a81bcf0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.752777 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 3b9c09b6315345039cdea9eeee3f181a (DEFAULT_TABLE table=default.Table_561cca4d810b4e73934fcee99a8994c7 [id=270d1081a9a04f09b76df0f45cfbbff6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.753007 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b9c09b6315345039cdea9eeee3f181a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.755306 15153 raft_consensus.cc:491] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.755375 15153 raft_consensus.cc:513] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.755667 15153 leader_election.cc:290] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.756090 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_1b5e80210299494eabf041f3be441b9e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:15.756186 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e46d4a75e9b4ca8b83a188d06280fcd" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:15.756279 14421 raft_consensus.cc:2463] T 3e46d4a75e9b4ca8b83a188d06280fcd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
W20250115 03:06:15.756381 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1b5e80210299494eabf041f3be441b9e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.757097 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 3b9c09b6315345039cdea9eeee3f181a (DEFAULT_TABLE table=default.Table_561cca4d810b4e73934fcee99a8994c7 [id=270d1081a9a04f09b76df0f45cfbbff6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.757097 14535 tablet_service.cc:1467] Processing CreateTablet for tablet d5655935aabd48a19d0ef03739e089a4 (DEFAULT_TABLE table=default.Table_968db1bba3cf48dbba6f4456bcfeb41a [id=ed8713b68f234c46bc2ef1dae0059a2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.757366 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5655935aabd48a19d0ef03739e089a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.757416 14628 leader_election.cc:304] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.757440 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d27310f8433411a85bf88d05a81bcf0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.757916 15325 raft_consensus.cc:2798] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:15.757956 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d27310f8433411a85bf88d05a81bcf0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.757964 15325 raft_consensus.cc:491] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:15.758019 15325 raft_consensus.cc:3054] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.758229 14663 tablet_service.cc:1467] Processing CreateTablet for tablet d5655935aabd48a19d0ef03739e089a4 (DEFAULT_TABLE table=default.Table_968db1bba3cf48dbba6f4456bcfeb41a [id=ed8713b68f234c46bc2ef1dae0059a2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.758445 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5655935aabd48a19d0ef03739e089a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.758832 15325 raft_consensus.cc:513] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.759142 15325 leader_election.cc:290] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.757337 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e46d4a75e9b4ca8b83a188d06280fcd" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.759407 14556 raft_consensus.cc:2463] T 3e46d4a75e9b4ca8b83a188d06280fcd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:15.759611 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_95ad0daf75e040a28fa088bf49371f95" } new_table_name: "default.Table_0aecd3e601c04f0497cefafda025e6e1" modify_external_catalogs: true
I20250115 03:06:15.759691 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b9c09b6315345039cdea9eeee3f181a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.760466 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e46d4a75e9b4ca8b83a188d06280fcd" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:15.760553 14421 raft_consensus.cc:3054] T 3e46d4a75e9b4ca8b83a188d06280fcd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:15.761212 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 5d27310f8433411a85bf88d05a81bcf0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d27310f8433411a85bf88d05a81bcf0 already in progress: creating tablet
I20250115 03:06:15.761389 14421 raft_consensus.cc:2463] T 3e46d4a75e9b4ca8b83a188d06280fcd P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:15.761351 14171 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_849d2ef8d13549b7bb6bfb03314d998b" } new_table_name: "default.Table_92b7f10774fe4db0ac9872d33fceb7a7" modify_external_catalogs: true
I20250115 03:06:15.761770 14628 leader_election.cc:304] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.761790 14059 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu
/tmp/dist-test-taskE2XgYg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.186.252:41855
--webserver_interface=127.13.186.252
--webserver_port=39163
--builtin_ntp_servers=127.13.186.212:37407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.186.254:39901,127.13.186.253:35411,127.13.186.252:41855
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20250115 03:06:15.761919 15453 raft_consensus.cc:2798] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:15.761942 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d78a7793f84424bad42a1aba89eab92 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.761996 15453 raft_consensus.cc:695] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:15.762071 14819 tablet_replica.cc:331] T 2d78a7793f84424bad42a1aba89eab92 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.762090 15453 consensus_queue.cc:237] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.762176 14819 raft_consensus.cc:2238] T 2d78a7793f84424bad42a1aba89eab92 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.762254 14819 raft_consensus.cc:2267] T 2d78a7793f84424bad42a1aba89eab92 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.762394 14823 tablet_replica.cc:331] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.762499 14823 raft_consensus.cc:2238] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.762586 14823 raft_consensus.cc:2267] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.762722 14820 tablet_replica.cc:331] T f978e85cf659450ca0039368d58b49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.763311 14820 raft_consensus.cc:2238] T f978e85cf659450ca0039368d58b49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.763404 14820 raft_consensus.cc:2267] T f978e85cf659450ca0039368d58b49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.763680 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e46d4a75e9b4ca8b83a188d06280fcd" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:15.763772 14556 raft_consensus.cc:3054] T 3e46d4a75e9b4ca8b83a188d06280fcd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.763789 14823 ts_tablet_manager.cc:1905] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.764573 14556 raft_consensus.cc:2463] T 3e46d4a75e9b4ca8b83a188d06280fcd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:15.768103 14819 ts_tablet_manager.cc:1905] T 2d78a7793f84424bad42a1aba89eab92 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.769588 14527 tablet_service.cc:1467] Processing CreateTablet for tablet cc67fb818b8146629d92f9ac1944dba6 (DEFAULT_TABLE table=default.Table_3fab7f59cc324dcf811b852f62ba1f9e [id=0a2e80b35ce148ff85d040843c7a1a2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.769726 14393 tablet_service.cc:1467] Processing CreateTablet for tablet cc67fb818b8146629d92f9ac1944dba6 (DEFAULT_TABLE table=default.Table_3fab7f59cc324dcf811b852f62ba1f9e [id=0a2e80b35ce148ff85d040843c7a1a2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.769814 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc67fb818b8146629d92f9ac1944dba6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.769940 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc67fb818b8146629d92f9ac1944dba6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.772305 14819 ts_tablet_manager.cc:1918] T 2d78a7793f84424bad42a1aba89eab92 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.772367 14819 log.cc:1198] T 2d78a7793f84424bad42a1aba89eab92 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/2d78a7793f84424bad42a1aba89eab92
I20250115 03:06:15.772778 14819 ts_tablet_manager.cc:1939] T 2d78a7793f84424bad42a1aba89eab92 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.773389 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 2d78a7793f84424bad42a1aba89eab92 (table default.Table_638e39bcf2aa4131ac57afbe63ac9b73 [id=50928432524e42d79855e9143c537b61]) successfully deleted
I20250115 03:06:15.773502 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 80aa570e2c154172ad6da29d268bede7 (DEFAULT_TABLE table=default.Table_24cbe6b962b248a9b340506b63c41b20 [id=9866275e65ab4dfcbfee6bf209c3654e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.773698 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80aa570e2c154172ad6da29d268bede7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.774060 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 80aa570e2c154172ad6da29d268bede7 (DEFAULT_TABLE table=default.Table_24cbe6b962b248a9b340506b63c41b20 [id=9866275e65ab4dfcbfee6bf209c3654e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.774063 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 80aa570e2c154172ad6da29d268bede7 (DEFAULT_TABLE table=default.Table_24cbe6b962b248a9b340506b63c41b20 [id=9866275e65ab4dfcbfee6bf209c3654e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.774257 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80aa570e2c154172ad6da29d268bede7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.774761 14663 tablet_service.cc:1467] Processing CreateTablet for tablet cc67fb818b8146629d92f9ac1944dba6 (DEFAULT_TABLE table=default.Table_3fab7f59cc324dcf811b852f62ba1f9e [id=0a2e80b35ce148ff85d040843c7a1a2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.774964 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc67fb818b8146629d92f9ac1944dba6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.775723 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet 33380937a3b7428995b06e477a896c02 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:15.775885 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80aa570e2c154172ad6da29d268bede7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.778053 14162 catalog_manager.cc:5526] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:15.781364 14823 ts_tablet_manager.cc:1918] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.781484 14823 log.cc:1198] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/01aa12e3d8fa463cbaee1287e7b74dd4
I20250115 03:06:15.782805 15401 consensus_queue.cc:1035] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:15.784418 14823 ts_tablet_manager.cc:1939] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.786099 14820 ts_tablet_manager.cc:1905] T f978e85cf659450ca0039368d58b49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.787112 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 01aa12e3d8fa463cbaee1287e7b74dd4 (table default.Table_490cb72d634f46eaa4130e320f581469 [id=fe70549b0af7478ea3ddd2b658fb42fc]) successfully deleted
W20250115 03:06:15.787261 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 5d27310f8433411a85bf88d05a81bcf0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d27310f8433411a85bf88d05a81bcf0 already in progress: creating tablet
I20250115 03:06:15.788179 14820 ts_tablet_manager.cc:1918] T f978e85cf659450ca0039368d58b49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.788288 14820 log.cc:1198] T f978e85cf659450ca0039368d58b49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/f978e85cf659450ca0039368d58b49e5
I20250115 03:06:15.788537 14820 ts_tablet_manager.cc:1939] T f978e85cf659450ca0039368d58b49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.789363 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet f978e85cf659450ca0039368d58b49e5 (table default.Table_4bde77af131940a099eabda16fa2ccb2 [id=4f952e57c86746f6bb1df72e13589832]) successfully deleted
I20250115 03:06:15.789590 14820 tablet_replica.cc:331] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.789741 14820 raft_consensus.cc:2238] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.789885 14820 raft_consensus.cc:2267] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:15.790376 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ca2093980c8e480e99ae0cd79606f318 has no consensus state
I20250115 03:06:15.790441 14812 tablet_bootstrap.cc:492] T 9808c7f96a3147848da5f78948895699 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.790505 14812 ts_tablet_manager.cc:1397] T 9808c7f96a3147848da5f78948895699 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.050s user 0.001s sys 0.000s
I20250115 03:06:15.790874 14812 raft_consensus.cc:357] T 9808c7f96a3147848da5f78948895699 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.790977 14812 raft_consensus.cc:383] T 9808c7f96a3147848da5f78948895699 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.791013 14812 raft_consensus.cc:738] T 9808c7f96a3147848da5f78948895699 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.791124 14812 consensus_queue.cc:260] T 9808c7f96a3147848da5f78948895699 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.791399 14812 ts_tablet_manager.cc:1428] T 9808c7f96a3147848da5f78948895699 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.791550 14812 tablet_bootstrap.cc:492] T db1aa206e9994de6b9a355a621cb2ab3 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.791985 14820 ts_tablet_manager.cc:1905] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.792325 14812 tablet_bootstrap.cc:654] T db1aa206e9994de6b9a355a621cb2ab3 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.793692 14812 tablet_bootstrap.cc:492] T db1aa206e9994de6b9a355a621cb2ab3 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.793771 14812 ts_tablet_manager.cc:1397] T db1aa206e9994de6b9a355a621cb2ab3 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:15.794140 14812 raft_consensus.cc:357] T db1aa206e9994de6b9a355a621cb2ab3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.794225 14812 raft_consensus.cc:383] T db1aa206e9994de6b9a355a621cb2ab3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.794253 14812 raft_consensus.cc:738] T db1aa206e9994de6b9a355a621cb2ab3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.794514 14812 consensus_queue.cc:260] T db1aa206e9994de6b9a355a621cb2ab3 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.795297 14812 ts_tablet_manager.cc:1428] T db1aa206e9994de6b9a355a621cb2ab3 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.796123 14812 tablet_bootstrap.cc:492] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.796106 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2299c208a52a4d99ba20b1f3b5783fbd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.796406 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2299c208a52a4d99ba20b1f3b5783fbd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.797024 14812 tablet_bootstrap.cc:654] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.797241 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_71fadacb26bd4e4a8ddcd48e403350bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.797549 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_71fadacb26bd4e4a8ddcd48e403350bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.800000 14811 tablet_bootstrap.cc:492] T e368f87556944c46a6d8f160ead17ac9 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.800086 14811 ts_tablet_manager.cc:1397] T e368f87556944c46a6d8f160ead17ac9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.102s user 0.002s sys 0.000s
I20250115 03:06:15.800493 14811 raft_consensus.cc:357] T e368f87556944c46a6d8f160ead17ac9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.800583 14811 raft_consensus.cc:383] T e368f87556944c46a6d8f160ead17ac9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.800614 14811 raft_consensus.cc:738] T e368f87556944c46a6d8f160ead17ac9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.800729 14811 consensus_queue.cc:260] T e368f87556944c46a6d8f160ead17ac9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.800989 14811 ts_tablet_manager.cc:1428] T e368f87556944c46a6d8f160ead17ac9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.801157 14811 tablet_bootstrap.cc:492] T 2379bae3690c44949e658dde84c1712e P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.802017 14811 tablet_bootstrap.cc:654] T 2379bae3690c44949e658dde84c1712e P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.805737 15153 raft_consensus.cc:491] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.805814 15153 raft_consensus.cc:513] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.806083 15153 leader_election.cc:290] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.806356 14811 tablet_bootstrap.cc:492] T 2379bae3690c44949e658dde84c1712e P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.806432 14811 ts_tablet_manager.cc:1397] T 2379bae3690c44949e658dde84c1712e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:15.806403 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e368f87556944c46a6d8f160ead17ac9" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:15.806522 14421 raft_consensus.cc:2463] T e368f87556944c46a6d8f160ead17ac9 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:15.806504 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e368f87556944c46a6d8f160ead17ac9" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.806604 14554 raft_consensus.cc:2463] T e368f87556944c46a6d8f160ead17ac9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:15.806777 14628 leader_election.cc:304] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.806828 14811 raft_consensus.cc:357] T 2379bae3690c44949e658dde84c1712e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.806905 14811 raft_consensus.cc:383] T 2379bae3690c44949e658dde84c1712e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.806932 14811 raft_consensus.cc:738] T 2379bae3690c44949e658dde84c1712e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.807150 15153 raft_consensus.cc:2798] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:15.807124 14811 consensus_queue.cc:260] T 2379bae3690c44949e658dde84c1712e P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.807196 15153 raft_consensus.cc:491] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:15.807231 15153 raft_consensus.cc:3054] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.807374 14811 ts_tablet_manager.cc:1428] T 2379bae3690c44949e658dde84c1712e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.807533 14811 tablet_bootstrap.cc:492] T 99d9180ed9b24363a4c44d01ea1d6e9b P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.808410 15153 raft_consensus.cc:513] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.808732 15153 leader_election.cc:290] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.808830 14811 tablet_bootstrap.cc:654] T 99d9180ed9b24363a4c44d01ea1d6e9b P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.809276 14812 tablet_bootstrap.cc:492] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.809340 14812 ts_tablet_manager.cc:1397] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.001s
I20250115 03:06:15.809803 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e368f87556944c46a6d8f160ead17ac9" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:15.809899 14554 raft_consensus.cc:3054] T e368f87556944c46a6d8f160ead17ac9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.809975 14812 raft_consensus.cc:357] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.810051 14812 raft_consensus.cc:383] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.810078 14812 raft_consensus.cc:738] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.810155 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e368f87556944c46a6d8f160ead17ac9" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:15.810176 14812 consensus_queue.cc:260] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.810245 14421 raft_consensus.cc:3054] T e368f87556944c46a6d8f160ead17ac9 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.810403 14812 ts_tablet_manager.cc:1428] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:15.810583 14812 tablet_bootstrap.cc:492] T 79579515fb7e45a8b16fd36dbc6a965a P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.810729 14554 raft_consensus.cc:2463] T e368f87556944c46a6d8f160ead17ac9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
W20250115 03:06:15.811090 14171 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 054251151c294e06905092f2a9306849 consensus state has no leader
I20250115 03:06:15.811079 14421 raft_consensus.cc:2463] T e368f87556944c46a6d8f160ead17ac9 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:15.812311 14812 tablet_bootstrap.cc:654] T 79579515fb7e45a8b16fd36dbc6a965a P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.813114 14628 leader_election.cc:304] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.813361 15325 raft_consensus.cc:2798] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:15.813441 15325 raft_consensus.cc:695] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:15.813572 15325 consensus_queue.cc:237] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.814944 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 2379bae3690c44949e658dde84c1712e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.819105 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 68974a1815cb482d8b28d54f42634608 with delete_type TABLET_DATA_DELETED (replaced by tablet 420b7c8d43a04bae972eed2014d5634c) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.819108 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet f5af8b81e7ff4acab5aef13003656988 with delete_type TABLET_DATA_DELETED (replaced by tablet 3efc2fb8427f493ea933347bec34f88f) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.819271 14823 tablet_replica.cc:331] T 68974a1815cb482d8b28d54f42634608 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.819398 14823 raft_consensus.cc:2238] T 68974a1815cb482d8b28d54f42634608 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.819501 14823 raft_consensus.cc:2267] T 68974a1815cb482d8b28d54f42634608 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.819869 14823 ts_tablet_manager.cc:1905] T 68974a1815cb482d8b28d54f42634608 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.821939 14823 ts_tablet_manager.cc:1918] T 68974a1815cb482d8b28d54f42634608 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.822012 14823 log.cc:1198] T 68974a1815cb482d8b28d54f42634608 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/68974a1815cb482d8b28d54f42634608
I20250115 03:06:15.822193 14823 ts_tablet_manager.cc:1939] T 68974a1815cb482d8b28d54f42634608 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.822582 14812 tablet_bootstrap.cc:492] T 79579515fb7e45a8b16fd36dbc6a965a P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.822651 14812 ts_tablet_manager.cc:1397] T 79579515fb7e45a8b16fd36dbc6a965a P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.002s
I20250115 03:06:15.822623 14162 catalog_manager.cc:5526] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:15.823006 14812 raft_consensus.cc:357] T 79579515fb7e45a8b16fd36dbc6a965a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.823086 14812 raft_consensus.cc:383] T 79579515fb7e45a8b16fd36dbc6a965a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.823112 14812 raft_consensus.cc:738] T 79579515fb7e45a8b16fd36dbc6a965a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.823199 14812 consensus_queue.cc:260] T 79579515fb7e45a8b16fd36dbc6a965a P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.823402 14812 ts_tablet_manager.cc:1428] T 79579515fb7e45a8b16fd36dbc6a965a P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:15.823544 14812 tablet_bootstrap.cc:492] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.823918 14820 ts_tablet_manager.cc:1918] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.823977 14820 log.cc:1198] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/01aa12e3d8fa463cbaee1287e7b74dd4
I20250115 03:06:15.824191 14820 ts_tablet_manager.cc:1939] T 01aa12e3d8fa463cbaee1287e7b74dd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.824488 14812 tablet_bootstrap.cc:654] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.826155 14823 tablet_replica.cc:331] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.826308 14823 raft_consensus.cc:2238] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.826421 14823 raft_consensus.cc:2267] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.826767 14823 ts_tablet_manager.cc:1905] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.826813 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 68974a1815cb482d8b28d54f42634608 (table default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]) successfully deleted
I20250115 03:06:15.828727 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0337ad43f9b646a48c61c66c15f1844a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:15.829034 14823 ts_tablet_manager.cc:1918] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20250115 03:06:15.829102 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0337ad43f9b646a48c61c66c15f1844a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.829172 14823 log.cc:1198] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/f5af8b81e7ff4acab5aef13003656988
I20250115 03:06:15.829349 14823 ts_tablet_manager.cc:1939] T f5af8b81e7ff4acab5aef13003656988 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.829835 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet f5af8b81e7ff4acab5aef13003656988 (table default.Table_f4fc7a15692446fba7fcdb17b6b33d56 [id=364a7905f3f044f684e9c75927f7efda]) successfully deleted
I20250115 03:06:15.830458 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_1b5e80210299494eabf041f3be441b9e" } new_table_name: "default.Table_a63d6344914343c4af56eef21f467c6f" modify_external_catalogs: true
I20250115 03:06:15.831091 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet db1aa206e9994de6b9a355a621cb2ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.831200 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 9808c7f96a3147848da5f78948895699 with delete_type TABLET_DATA_DELETED (replaced by tablet 341b24d9b63f4761b2013035023fcd98) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.831301 14823 tablet_replica.cc:331] T db1aa206e9994de6b9a355a621cb2ab3 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.831389 14823 raft_consensus.cc:2238] T db1aa206e9994de6b9a355a621cb2ab3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.831466 14823 raft_consensus.cc:2267] T db1aa206e9994de6b9a355a621cb2ab3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.831736 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_c0bde98045a84e568d03c9608c7731fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.832015 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c0bde98045a84e568d03c9608c7731fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.832748 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_5e89af766dcb414fb8dfb770c99ede52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.833149 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5e89af766dcb414fb8dfb770c99ede52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.833753 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 79579515fb7e45a8b16fd36dbc6a965a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.833801 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_62fa8b23f4c44bb0938d6363bc2086ef" } new_table_name: "default.Table_7a132f92dbad4a868f47b81f0107d42e" modify_external_catalogs: true
I20250115 03:06:15.834578 14823 ts_tablet_manager.cc:1905] T db1aa206e9994de6b9a355a621cb2ab3 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.836436 14823 ts_tablet_manager.cc:1918] T db1aa206e9994de6b9a355a621cb2ab3 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.836519 14823 log.cc:1198] T db1aa206e9994de6b9a355a621cb2ab3 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/db1aa206e9994de6b9a355a621cb2ab3
I20250115 03:06:15.836701 14823 ts_tablet_manager.cc:1939] T db1aa206e9994de6b9a355a621cb2ab3 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.837246 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet db1aa206e9994de6b9a355a621cb2ab3 (table default.Table_e050a17d73d342fda8108a99a665f222 [id=eea2f4b389314abbac5f983a79d5cf7e]) successfully deleted
I20250115 03:06:15.837433 14823 tablet_replica.cc:331] T 9808c7f96a3147848da5f78948895699 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.837625 14823 raft_consensus.cc:2238] T 9808c7f96a3147848da5f78948895699 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.837723 14823 raft_consensus.cc:2267] T 9808c7f96a3147848da5f78948895699 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.838060 14823 ts_tablet_manager.cc:1905] T 9808c7f96a3147848da5f78948895699 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.839068 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 01aa12e3d8fa463cbaee1287e7b74dd4 (table default.Table_490cb72d634f46eaa4130e320f581469 [id=fe70549b0af7478ea3ddd2b658fb42fc]) successfully deleted
W20250115 03:06:15.839495 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 5d27310f8433411a85bf88d05a81bcf0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5d27310f8433411a85bf88d05a81bcf0 already in progress: creating tablet
I20250115 03:06:15.839742 14820 tablet_replica.cc:331] T 2379bae3690c44949e658dde84c1712e P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.839916 14820 raft_consensus.cc:2238] T 2379bae3690c44949e658dde84c1712e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.840016 14823 ts_tablet_manager.cc:1918] T 9808c7f96a3147848da5f78948895699 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.840075 14820 raft_consensus.cc:2267] T 2379bae3690c44949e658dde84c1712e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.840113 14823 log.cc:1198] T 9808c7f96a3147848da5f78948895699 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/9808c7f96a3147848da5f78948895699
I20250115 03:06:15.840327 14823 ts_tablet_manager.cc:1939] T 9808c7f96a3147848da5f78948895699 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.841199 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 9808c7f96a3147848da5f78948895699 (table default.Table_af3419632bb0478bab77f1fa32e9fb52 [id=369c42ec23ec4a27a71a54be23a410a6]) successfully deleted
I20250115 03:06:15.841867 14823 tablet_replica.cc:331] T 79579515fb7e45a8b16fd36dbc6a965a P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.842013 14823 raft_consensus.cc:2238] T 79579515fb7e45a8b16fd36dbc6a965a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.842219 14823 raft_consensus.cc:2267] T 79579515fb7e45a8b16fd36dbc6a965a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.842687 14823 ts_tablet_manager.cc:1905] T 79579515fb7e45a8b16fd36dbc6a965a P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.844259 14823 ts_tablet_manager.cc:1918] T 79579515fb7e45a8b16fd36dbc6a965a P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.844319 14823 log.cc:1198] T 79579515fb7e45a8b16fd36dbc6a965a P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/79579515fb7e45a8b16fd36dbc6a965a
I20250115 03:06:15.844494 14823 ts_tablet_manager.cc:1939] T 79579515fb7e45a8b16fd36dbc6a965a P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.845062 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 79579515fb7e45a8b16fd36dbc6a965a (table default.Table_b1775fbfc0eb4cf4a59a0f9a853a081e [id=ed57b2cba1db497fa0af4bba35580a35]) successfully deleted
I20250115 03:06:15.845161 14820 ts_tablet_manager.cc:1905] T 2379bae3690c44949e658dde84c1712e P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.846519 14811 tablet_bootstrap.cc:492] T 99d9180ed9b24363a4c44d01ea1d6e9b P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.846601 14811 ts_tablet_manager.cc:1397] T 99d9180ed9b24363a4c44d01ea1d6e9b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.039s user 0.001s sys 0.001s
I20250115 03:06:15.846973 15401 raft_consensus.cc:491] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.847046 15401 raft_consensus.cc:513] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.847373 15401 leader_election.cc:290] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:15.847666 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2fc04950cfe4c608a4712afd257ca76" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.847735 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2fc04950cfe4c608a4712afd257ca76" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:15.847831 14691 raft_consensus.cc:2463] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
W20250115 03:06:15.848071 14356 leader_election.cc:343] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.848255 14811 raft_consensus.cc:357] T 99d9180ed9b24363a4c44d01ea1d6e9b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.848343 14811 raft_consensus.cc:383] T 99d9180ed9b24363a4c44d01ea1d6e9b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.848373 14811 raft_consensus.cc:738] T 99d9180ed9b24363a4c44d01ea1d6e9b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.848501 14811 consensus_queue.cc:260] T 99d9180ed9b24363a4c44d01ea1d6e9b P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.848789 14811 ts_tablet_manager.cc:1428] T 99d9180ed9b24363a4c44d01ea1d6e9b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:15.848989 14811 tablet_bootstrap.cc:492] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.849458 14358 leader_election.cc:304] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:15.849848 14885 raft_consensus.cc:2798] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:15.849896 14885 raft_consensus.cc:491] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:15.849932 14885 raft_consensus.cc:3054] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.850152 14811 tablet_bootstrap.cc:654] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.850793 14885 raft_consensus.cc:513] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.850936 14809 tablet_bootstrap.cc:492] T b1644ba8616349dc9d10a790b6089571 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.851004 14809 ts_tablet_manager.cc:1397] T b1644ba8616349dc9d10a790b6089571 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.114s user 0.000s sys 0.002s
I20250115 03:06:15.851092 14885 leader_election.cc:290] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:15.851426 14809 raft_consensus.cc:357] T b1644ba8616349dc9d10a790b6089571 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.851516 14809 raft_consensus.cc:383] T b1644ba8616349dc9d10a790b6089571 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.851545 14809 raft_consensus.cc:738] T b1644ba8616349dc9d10a790b6089571 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.851646 14809 consensus_queue.cc:260] T b1644ba8616349dc9d10a790b6089571 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.851771 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2fc04950cfe4c608a4712afd257ca76" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:15.851866 14691 raft_consensus.cc:3054] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.851907 14809 ts_tablet_manager.cc:1428] T b1644ba8616349dc9d10a790b6089571 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:15.852097 14809 tablet_bootstrap.cc:492] T 82c2f41769d84bd48bfc1bc7f504be4b P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.852669 14691 raft_consensus.cc:2463] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:15.853920 14358 leader_election.cc:304] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.854023 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2fc04950cfe4c608a4712afd257ca76" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:15.854496 15401 raft_consensus.cc:2798] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
W20250115 03:06:15.854544 14356 leader_election.cc:343] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.854586 15401 raft_consensus.cc:695] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:15.854686 15401 consensus_queue.cc:237] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:15.855214 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6bcfde5fe3d4425e98edb4dc54096718 has no consensus state
I20250115 03:06:15.855738 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_92b7f10774fe4db0ac9872d33fceb7a7" } modify_external_catalogs: true
I20250115 03:06:15.855832 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_92b7f10774fe4db0ac9872d33fceb7a7" } modify_external_catalogs: true
I20250115 03:06:15.858613 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet b1644ba8616349dc9d10a790b6089571 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.858767 14819 tablet_replica.cc:331] T b1644ba8616349dc9d10a790b6089571 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.858899 14819 raft_consensus.cc:2238] T b1644ba8616349dc9d10a790b6089571 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.858990 14819 raft_consensus.cc:2267] T b1644ba8616349dc9d10a790b6089571 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.859145 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0d7244964d874f5eb1f508e5c0dabf1a" } modify_external_catalogs: true
I20250115 03:06:15.859236 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0d7244964d874f5eb1f508e5c0dabf1a" } modify_external_catalogs: true
I20250115 03:06:15.859273 14809 tablet_bootstrap.cc:654] T 82c2f41769d84bd48bfc1bc7f504be4b P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:15.860219 14162 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 90277f7d279e4c7a83c43f40766cd11b consensus state has no leader
I20250115 03:06:15.861737 14819 ts_tablet_manager.cc:1905] T b1644ba8616349dc9d10a790b6089571 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.863865 14819 ts_tablet_manager.cc:1918] T b1644ba8616349dc9d10a790b6089571 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.863930 14819 log.cc:1198] T b1644ba8616349dc9d10a790b6089571 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/b1644ba8616349dc9d10a790b6089571
I20250115 03:06:15.864107 14819 ts_tablet_manager.cc:1939] T b1644ba8616349dc9d10a790b6089571 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.864910 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet b1644ba8616349dc9d10a790b6089571 (table default.Table_632fc9d76fb3482c8a5e87feda0ceeed [id=dd1bb6f7163f46748711bc33cc8a3f85]) successfully deleted
I20250115 03:06:15.865211 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ef19dd8e16aa4bf9a97897ebc1c8eef5" } modify_external_catalogs: true
I20250115 03:06:15.865299 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ef19dd8e16aa4bf9a97897ebc1c8eef5" } modify_external_catalogs: true
I20250115 03:06:15.870242 15325 raft_consensus.cc:491] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.870323 15325 raft_consensus.cc:513] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.870648 15325 leader_election.cc:290] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.871018 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff51d514fb84809aa53fa9fc1010681" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.871239 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff51d514fb84809aa53fa9fc1010681" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:15.872196 14628 leader_election.cc:343] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.872599 14820 ts_tablet_manager.cc:1918] T 2379bae3690c44949e658dde84c1712e P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20250115 03:06:15.872722 14626 leader_election.cc:343] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.872789 14626 leader_election.cc:304] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:15.872953 15325 raft_consensus.cc:2743] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:15.873142 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet a75b9401cb4b4f8b92a4779c3704eb46 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:15.873394 14166 catalog_manager.cc:5526] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:15.873596 14166 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet a2fc04950cfe4c608a4712afd257ca76 (table default.Table_6b1f81ee12e04acda4bcf7d7f289a67a [id=8f811bda53fc4d76a88a3dcf69aa3dc6]). Expected version 1 got 0
I20250115 03:06:15.874588 14812 tablet_bootstrap.cc:492] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.874675 14812 ts_tablet_manager.cc:1397] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.051s user 0.001s sys 0.000s
I20250115 03:06:15.875087 14812 raft_consensus.cc:357] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.875181 14812 raft_consensus.cc:383] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.875214 14812 raft_consensus.cc:738] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.875324 14812 consensus_queue.cc:260] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.875597 14812 ts_tablet_manager.cc:1428] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:15.875751 14812 tablet_bootstrap.cc:492] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.876660 14812 tablet_bootstrap.cc:654] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.880088 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 054251151c294e06905092f2a9306849
I20250115 03:06:15.881204 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 054251151c294e06905092f2a9306849 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.881278 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 054251151c294e06905092f2a9306849 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.881867 14169 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 13ce129d8a9245baab7c772f0f5fbaa7 (table default.Table_d49d96cab47241d4836768cbc9cd552b [id=6cba4fb2f7104c6e953340b04e5dc494]). Expected version 1 got 0
W20250115 03:06:15.882397 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 054251151c294e06905092f2a9306849 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 054251151c294e06905092f2a9306849 already in progress: creating tablet
W20250115 03:06:15.882825 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 054251151c294e06905092f2a9306849 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 054251151c294e06905092f2a9306849 already in progress: creating tablet
I20250115 03:06:15.885151 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet 5d27310f8433411a85bf88d05a81bcf0 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:15.886370 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 5737f10aa50e4059a814585019346872 (DEFAULT_TABLE table=default.Table_7f271037abae4e2a979c2358897fc66f [id=17af824fb80a4c6b986f305d79e15031]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.886634 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5737f10aa50e4059a814585019346872. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.877012 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 99d9180ed9b24363a4c44d01ea1d6e9b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.880180 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 5737f10aa50e4059a814585019346872 (DEFAULT_TABLE table=default.Table_7f271037abae4e2a979c2358897fc66f [id=17af824fb80a4c6b986f305d79e15031]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.887943 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5737f10aa50e4059a814585019346872. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.889011 14812 tablet_bootstrap.cc:492] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.889094 14812 ts_tablet_manager.cc:1397] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20250115 03:06:15.889161 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 5737f10aa50e4059a814585019346872 (DEFAULT_TABLE table=default.Table_7f271037abae4e2a979c2358897fc66f [id=17af824fb80a4c6b986f305d79e15031]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.889281 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 3e46d4a75e9b4ca8b83a188d06280fcd
I20250115 03:06:15.889380 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5737f10aa50e4059a814585019346872. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.889576 14812 raft_consensus.cc:357] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.889669 14812 raft_consensus.cc:383] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.889698 14812 raft_consensus.cc:738] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.889801 14812 consensus_queue.cc:260] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.882166 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 054251151c294e06905092f2a9306849 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.889986 14820 log.cc:1198] T 2379bae3690c44949e658dde84c1712e P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/2379bae3690c44949e658dde84c1712e
I20250115 03:06:15.890031 14812 ts_tablet_manager.cc:1428] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.890093 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e46d4a75e9b4ca8b83a188d06280fcd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.890192 14812 tablet_bootstrap.cc:492] T 7f67cbb929cd4ae589a14606aa168b46 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.890228 14819 tablet_replica.cc:331] T 3e46d4a75e9b4ca8b83a188d06280fcd P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.890347 14819 raft_consensus.cc:2238] T 3e46d4a75e9b4ca8b83a188d06280fcd P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.890432 14819 raft_consensus.cc:2267] T 3e46d4a75e9b4ca8b83a188d06280fcd P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.890865 14811 tablet_bootstrap.cc:492] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.890865 14819 ts_tablet_manager.cc:1905] T 3e46d4a75e9b4ca8b83a188d06280fcd P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.890938 14811 ts_tablet_manager.cc:1397] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.042s user 0.001s sys 0.000s
I20250115 03:06:15.891114 14812 tablet_bootstrap.cc:654] T 7f67cbb929cd4ae589a14606aa168b46 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.891402 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e46d4a75e9b4ca8b83a188d06280fcd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.891675 14823 tablet_replica.cc:331] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.891785 14823 raft_consensus.cc:2238] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:15.891782 14811 raft_consensus.cc:357] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.891871 14811 raft_consensus.cc:383] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.891899 14811 raft_consensus.cc:738] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.891903 14823 pending_rounds.cc:62] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 1 pending ops.
I20250115 03:06:15.891930 14823 pending_rounds.cc:69] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384899122597888 op_type: NO_OP noop_request { }
I20250115 03:06:15.891976 14823 raft_consensus.cc:2883] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:15.892021 14823 raft_consensus.cc:2267] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.892002 14811 consensus_queue.cc:260] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.892259 14811 ts_tablet_manager.cc:1428] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.892551 14823 ts_tablet_manager.cc:1905] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.892830 14819 ts_tablet_manager.cc:1918] T 3e46d4a75e9b4ca8b83a188d06280fcd P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.892886 14819 log.cc:1198] T 3e46d4a75e9b4ca8b83a188d06280fcd P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/3e46d4a75e9b4ca8b83a188d06280fcd
I20250115 03:06:15.894537 14823 ts_tablet_manager.cc:1918] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.894603 14823 log.cc:1198] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/3e46d4a75e9b4ca8b83a188d06280fcd
I20250115 03:06:15.894824 14823 ts_tablet_manager.cc:1939] T 3e46d4a75e9b4ca8b83a188d06280fcd P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.895711 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 3e46d4a75e9b4ca8b83a188d06280fcd (table default.Table_0d7244964d874f5eb1f508e5c0dabf1a [id=15762a37e62f4e89852b2eb66a008dbb]) successfully deleted
I20250115 03:06:15.895802 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_a1c9e28bac9c491e948bc3fc9b173bad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:15.896067 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e46d4a75e9b4ca8b83a188d06280fcd with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:15.896122 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a1c9e28bac9c491e948bc3fc9b173bad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.896831 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b2f2461378b548f785fdc3595569c0db"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:15.897152 14819 ts_tablet_manager.cc:1939] T 3e46d4a75e9b4ca8b83a188d06280fcd P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
W20250115 03:06:15.897159 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b2f2461378b548f785fdc3595569c0db in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.897714 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 3e46d4a75e9b4ca8b83a188d06280fcd (table default.Table_0d7244964d874f5eb1f508e5c0dabf1a [id=15762a37e62f4e89852b2eb66a008dbb]) successfully deleted
I20250115 03:06:15.898463 14164 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 22cc0833bfde458ebe3985c05d29b8e6 (table default.Table_3db38338e5b04298954aded6d5482169 [id=f56a0de6e56441bfa620902f175feb4a]). Expected version 2 got 0
I20250115 03:06:15.898593 14663 tablet_service.cc:1467] Processing CreateTablet for tablet e5a8293f43134a54b3f8dfb5fb99d6ea (DEFAULT_TABLE table=default.Table_98f4dad994f5494daf0f44483bb15e33 [id=2f6a1091a89943c9bd6a44d3168c456d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.898794 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5a8293f43134a54b3f8dfb5fb99d6ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.898933 14396 tablet_service.cc:1467] Processing CreateTablet for tablet ca2093980c8e480e99ae0cd79606f318 (DEFAULT_TABLE table=default.Table_0aecd3e601c04f0497cefafda025e6e1 [id=bde9696f4c534fe58bb1de4fd7e00201]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.899145 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca2093980c8e480e99ae0cd79606f318. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.900025 14811 tablet_bootstrap.cc:492] T f3ac7f1e1bfb4383895dc2c2123a743b P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.900890 14809 tablet_bootstrap.cc:492] T 82c2f41769d84bd48bfc1bc7f504be4b P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:15.901019 14809 ts_tablet_manager.cc:1397] T 82c2f41769d84bd48bfc1bc7f504be4b P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.049s user 0.000s sys 0.002s
I20250115 03:06:15.901427 14809 raft_consensus.cc:357] T 82c2f41769d84bd48bfc1bc7f504be4b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.901525 14809 raft_consensus.cc:383] T 82c2f41769d84bd48bfc1bc7f504be4b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.901557 14809 raft_consensus.cc:738] T 82c2f41769d84bd48bfc1bc7f504be4b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.901556 14663 tablet_service.cc:1467] Processing CreateTablet for tablet ca2093980c8e480e99ae0cd79606f318 (DEFAULT_TABLE table=default.Table_0aecd3e601c04f0497cefafda025e6e1 [id=bde9696f4c534fe58bb1de4fd7e00201]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.901665 14809 consensus_queue.cc:260] T 82c2f41769d84bd48bfc1bc7f504be4b P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.901767 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca2093980c8e480e99ae0cd79606f318. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.901934 14809 ts_tablet_manager.cc:1428] T 82c2f41769d84bd48bfc1bc7f504be4b P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:15.902099 14809 tablet_bootstrap.cc:492] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:15.902238 14393 tablet_service.cc:1467] Processing CreateTablet for tablet e5a8293f43134a54b3f8dfb5fb99d6ea (DEFAULT_TABLE table=default.Table_98f4dad994f5494daf0f44483bb15e33 [id=2f6a1091a89943c9bd6a44d3168c456d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.902422 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5a8293f43134a54b3f8dfb5fb99d6ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.902789 14809 tablet_bootstrap.cc:654] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.903371 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 62ec9057d85447bdb6b86cdc1f9015ac (DEFAULT_TABLE table=default.Table_5a68d6c8c53b4746a7123860185fe0cc [id=84c18cf488cc4c28aaf93fa72334acb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.903654 14537 tablet_service.cc:1467] Processing CreateTablet for tablet ca2093980c8e480e99ae0cd79606f318 (DEFAULT_TABLE table=default.Table_0aecd3e601c04f0497cefafda025e6e1 [id=bde9696f4c534fe58bb1de4fd7e00201]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.903731 14536 tablet_service.cc:1467] Processing CreateTablet for tablet e5a8293f43134a54b3f8dfb5fb99d6ea (DEFAULT_TABLE table=default.Table_98f4dad994f5494daf0f44483bb15e33 [id=2f6a1091a89943c9bd6a44d3168c456d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.903877 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca2093980c8e480e99ae0cd79606f318. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.903978 15325 raft_consensus.cc:491] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.904052 15325 raft_consensus.cc:513] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.904353 15325 leader_election.cc:290] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.904721 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5a8293f43134a54b3f8dfb5fb99d6ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.904716 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17b5e362481c49d2a3a13b1550ceef23" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:15.904817 14421 raft_consensus.cc:2463] T 17b5e362481c49d2a3a13b1550ceef23 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:15.905189 14628 leader_election.cc:304] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.905337 15325 raft_consensus.cc:2798] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:15.905380 15325 raft_consensus.cc:491] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:15.905414 15325 raft_consensus.cc:3054] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.906216 15325 raft_consensus.cc:513] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.906559 15325 leader_election.cc:290] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.906872 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62ec9057d85447bdb6b86cdc1f9015ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.909974 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17b5e362481c49d2a3a13b1550ceef23" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:15.910069 14421 raft_consensus.cc:3054] T 17b5e362481c49d2a3a13b1550ceef23 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.910079 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_be6e30c629974876a5e7b6b25c4959dc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.910357 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_be6e30c629974876a5e7b6b25c4959dc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.910871 14421 raft_consensus.cc:2463] T 17b5e362481c49d2a3a13b1550ceef23 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:15.911370 14628 leader_election.cc:304] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.911700 15153 raft_consensus.cc:2798] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:15.911777 15153 raft_consensus.cc:695] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:15.911880 15153 consensus_queue.cc:237] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.912083 14811 tablet_bootstrap.cc:654] T f3ac7f1e1bfb4383895dc2c2123a743b P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.912731 14820 ts_tablet_manager.cc:1939] T 2379bae3690c44949e658dde84c1712e P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.913165 14812 tablet_bootstrap.cc:492] T 7f67cbb929cd4ae589a14606aa168b46 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.913236 14812 ts_tablet_manager.cc:1397] T 7f67cbb929cd4ae589a14606aa168b46 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s
I20250115 03:06:15.913257 14820 tablet_replica.cc:331] T 99d9180ed9b24363a4c44d01ea1d6e9b P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:15.913367 14820 raft_consensus.cc:2238] T 99d9180ed9b24363a4c44d01ea1d6e9b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.913448 14820 raft_consensus.cc:2267] T 99d9180ed9b24363a4c44d01ea1d6e9b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.913672 14812 raft_consensus.cc:357] T 7f67cbb929cd4ae589a14606aa168b46 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.913759 14812 raft_consensus.cc:383] T 7f67cbb929cd4ae589a14606aa168b46 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.913786 14812 raft_consensus.cc:738] T 7f67cbb929cd4ae589a14606aa168b46 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.913878 14812 consensus_queue.cc:260] T 7f67cbb929cd4ae589a14606aa168b46 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.914119 14812 ts_tablet_manager.cc:1428] T 7f67cbb929cd4ae589a14606aa168b46 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:15.914271 14812 tablet_bootstrap.cc:492] T 5b7022db9edc4de8909e16769a3ea136 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.915259 14812 tablet_bootstrap.cc:654] T 5b7022db9edc4de8909e16769a3ea136 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.915557 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 62ec9057d85447bdb6b86cdc1f9015ac (DEFAULT_TABLE table=default.Table_5a68d6c8c53b4746a7123860185fe0cc [id=84c18cf488cc4c28aaf93fa72334acb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.915761 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62ec9057d85447bdb6b86cdc1f9015ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.916693 14394 tablet_service.cc:1467] Processing CreateTablet for tablet da95daa40c5b4615a0219f6d8e3770f6 (DEFAULT_TABLE table=default.Table_4bde77af131940a099eabda16fa2ccb2 [id=4f952e57c86746f6bb1df72e13589832]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.916805 14399 tablet_service.cc:1467] Processing CreateTablet for tablet c251960c7c9a44429102a294f35f925c (DEFAULT_TABLE table=default.Table_90aedfcdb1274e1497cf36620b980768 [id=9e4f5de45eb24beabef550d15dcad8bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.916905 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da95daa40c5b4615a0219f6d8e3770f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.917588 14536 tablet_service.cc:1467] Processing CreateTablet for tablet c251960c7c9a44429102a294f35f925c (DEFAULT_TABLE table=default.Table_90aedfcdb1274e1497cf36620b980768 [id=9e4f5de45eb24beabef550d15dcad8bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.917670 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17b5e362481c49d2a3a13b1550ceef23" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.917810 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c251960c7c9a44429102a294f35f925c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.917996 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 62ec9057d85447bdb6b86cdc1f9015ac (DEFAULT_TABLE table=default.Table_5a68d6c8c53b4746a7123860185fe0cc [id=84c18cf488cc4c28aaf93fa72334acb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.918044 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c251960c7c9a44429102a294f35f925c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.918375 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 2379bae3690c44949e658dde84c1712e (table default.Table_ebca4429c164491c974b3ee244c3b1de [id=01a92c882c3640dcbc950b3e76f97c4b]) successfully deleted
I20250115 03:06:15.919023 14400 tablet_service.cc:1467] Processing CreateTablet for tablet e2c6db404348466ea5211ffa69c8f01c (DEFAULT_TABLE table=default.Table_ef19dd8e16aa4bf9a97897ebc1c8eef5 [id=38bfb0375c334d16b97c7251d6ddd76b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.919236 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2c6db404348466ea5211ffa69c8f01c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.919677 14527 tablet_service.cc:1467] Processing CreateTablet for tablet da95daa40c5b4615a0219f6d8e3770f6 (DEFAULT_TABLE table=default.Table_4bde77af131940a099eabda16fa2ccb2 [id=4f952e57c86746f6bb1df72e13589832]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.919875 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17b5e362481c49d2a3a13b1550ceef23" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:15.919883 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da95daa40c5b4615a0219f6d8e3770f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.920034 14537 tablet_service.cc:1467] Processing CreateTablet for tablet e2c6db404348466ea5211ffa69c8f01c (DEFAULT_TABLE table=default.Table_ef19dd8e16aa4bf9a97897ebc1c8eef5 [id=38bfb0375c334d16b97c7251d6ddd76b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.920248 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2c6db404348466ea5211ffa69c8f01c. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:15.921351 14626 leader_election.cc:343] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:15.921548 14626 leader_election.cc:343] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.921710 14666 tablet_service.cc:1467] Processing CreateTablet for tablet e2c6db404348466ea5211ffa69c8f01c (DEFAULT_TABLE table=default.Table_ef19dd8e16aa4bf9a97897ebc1c8eef5 [id=38bfb0375c334d16b97c7251d6ddd76b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.921921 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2c6db404348466ea5211ffa69c8f01c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.923972 14667 tablet_service.cc:1467] Processing CreateTablet for tablet da95daa40c5b4615a0219f6d8e3770f6 (DEFAULT_TABLE table=default.Table_4bde77af131940a099eabda16fa2ccb2 [id=4f952e57c86746f6bb1df72e13589832]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.924178 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da95daa40c5b4615a0219f6d8e3770f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.926163 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62ec9057d85447bdb6b86cdc1f9015ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.921561 14660 tablet_service.cc:1467] Processing CreateTablet for tablet c251960c7c9a44429102a294f35f925c (DEFAULT_TABLE table=default.Table_90aedfcdb1274e1497cf36620b980768 [id=9e4f5de45eb24beabef550d15dcad8bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.926985 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c251960c7c9a44429102a294f35f925c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.928848 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet b1644ba8616349dc9d10a790b6089571 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.930299 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet a75b9401cb4b4f8b92a4779c3704eb46
W20250115 03:06:15.930377 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet b1644ba8616349dc9d10a790b6089571 because the tablet was not found. No further retry: Not found: Tablet not found: b1644ba8616349dc9d10a790b6089571
I20250115 03:06:15.930567 14171 catalog_manager.cc:6596] ReplaceTablet: tablet a75b9401cb4b4f8b92a4779c3704eb46 deleted and replaced by tablet 21524aef80f74a6c899ea66b9b022389
I20250115 03:06:15.930701 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 82c2f41769d84bd48bfc1bc7f504be4b with delete_type TABLET_DATA_DELETED (replaced by tablet 80aa570e2c154172ad6da29d268bede7) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.930928 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet a75b9401cb4b4f8b92a4779c3704eb46 with delete_type TABLET_DATA_DELETED (replaced by tablet 21524aef80f74a6c899ea66b9b022389) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.931042 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet a75b9401cb4b4f8b92a4779c3704eb46 with delete_type TABLET_DATA_DELETED (replaced by tablet 21524aef80f74a6c899ea66b9b022389) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.931401 14819 tablet_replica.cc:331] T 82c2f41769d84bd48bfc1bc7f504be4b P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.931533 14819 raft_consensus.cc:2238] T 82c2f41769d84bd48bfc1bc7f504be4b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.931619 14819 raft_consensus.cc:2267] T 82c2f41769d84bd48bfc1bc7f504be4b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.931685 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet a75b9401cb4b4f8b92a4779c3704eb46 with delete_type TABLET_DATA_DELETED (replaced by tablet 21524aef80f74a6c899ea66b9b022389) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.931746 15325 raft_consensus.cc:491] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20250115 03:06:15.931773 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet a75b9401cb4b4f8b92a4779c3704eb46 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a75b9401cb4b4f8b92a4779c3704eb46 already in progress: creating tablet
I20250115 03:06:15.931823 15325 raft_consensus.cc:513] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.932159 15325 leader_election.cc:290] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.932286 14819 ts_tablet_manager.cc:1905] T 82c2f41769d84bd48bfc1bc7f504be4b P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.932417 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8baf86b37e9942ba813b099a0da87ca5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:15.932780 14628 leader_election.cc:343] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.933929 14171 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d4a2862d217743a181c6139aeeda1f1a" } new_table_name: "default.Table_070c9465ea944d50ad72cffc3a10681a" modify_external_catalogs: true
I20250115 03:06:15.934280 14819 ts_tablet_manager.cc:1918] T 82c2f41769d84bd48bfc1bc7f504be4b P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.934341 14819 log.cc:1198] T 82c2f41769d84bd48bfc1bc7f504be4b P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/82c2f41769d84bd48bfc1bc7f504be4b
I20250115 03:06:15.934515 14819 ts_tablet_manager.cc:1939] T 82c2f41769d84bd48bfc1bc7f504be4b P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.934705 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8baf86b37e9942ba813b099a0da87ca5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:15.935359 14626 leader_election.cc:343] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:15.935420 14626 leader_election.cc:304] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:15.935606 15325 raft_consensus.cc:2743] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:15.936408 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet e2c6db404348466ea5211ffa69c8f01c
I20250115 03:06:15.937692 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 82c2f41769d84bd48bfc1bc7f504be4b (table default.Table_24cbe6b962b248a9b340506b63c41b20 [id=9866275e65ab4dfcbfee6bf209c3654e]) successfully deleted
I20250115 03:06:15.937705 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet e2c6db404348466ea5211ffa69c8f01c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:15.937825 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet a75b9401cb4b4f8b92a4779c3704eb46 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a75b9401cb4b4f8b92a4779c3704eb46 already in progress: creating tablet
I20250115 03:06:15.938076 15325 raft_consensus.cc:491] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:15.938148 15325 raft_consensus.cc:513] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.938367 14812 tablet_bootstrap.cc:492] T 5b7022db9edc4de8909e16769a3ea136 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.938436 14812 ts_tablet_manager.cc:1397] T 5b7022db9edc4de8909e16769a3ea136 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.001s
I20250115 03:06:15.938827 14812 raft_consensus.cc:357] T 5b7022db9edc4de8909e16769a3ea136 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.938918 14812 raft_consensus.cc:383] T 5b7022db9edc4de8909e16769a3ea136 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.938946 14812 raft_consensus.cc:738] T 5b7022db9edc4de8909e16769a3ea136 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.938992 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22cc0833bfde458ebe3985c05d29b8e6" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:15.939049 14812 consensus_queue.cc:260] T 5b7022db9edc4de8909e16769a3ea136 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.939095 14556 raft_consensus.cc:2463] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
W20250115 03:06:15.939103 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet e2c6db404348466ea5211ffa69c8f01c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e2c6db404348466ea5211ffa69c8f01c already in progress: creating tablet
I20250115 03:06:15.939105 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22cc0833bfde458ebe3985c05d29b8e6" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:15.939203 14421 raft_consensus.cc:2463] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:15.939289 14812 ts_tablet_manager.cc:1428] T 5b7022db9edc4de8909e16769a3ea136 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:15.939353 14626 leader_election.cc:304] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:15.939476 14812 tablet_bootstrap.cc:492] T ec9f14c1ab014f29843a568c460a28d6 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.939630 15153 raft_consensus.cc:2798] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:15.939692 15153 raft_consensus.cc:491] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:15.939735 15153 raft_consensus.cc:3054] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.940265 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet e2c6db404348466ea5211ffa69c8f01c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.940371 14812 tablet_bootstrap.cc:654] T ec9f14c1ab014f29843a568c460a28d6 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.940554 15153 raft_consensus.cc:513] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.938439 15325 leader_election.cc:290] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.940869 15153 leader_election.cc:290] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:15.941172 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22cc0833bfde458ebe3985c05d29b8e6" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:15.941277 14421 raft_consensus.cc:3054] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:15.942071 14421 raft_consensus.cc:2463] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:15.942340 14628 leader_election.cc:304] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:15.942507 15153 raft_consensus.cc:2798] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:15.942592 15153 raft_consensus.cc:695] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:15.942709 15153 consensus_queue.cc:237] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.943595 14820 ts_tablet_manager.cc:1905] T 99d9180ed9b24363a4c44d01ea1d6e9b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.945374 14812 tablet_bootstrap.cc:492] T ec9f14c1ab014f29843a568c460a28d6 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.945444 14812 ts_tablet_manager.cc:1397] T ec9f14c1ab014f29843a568c460a28d6 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:15.945828 14812 raft_consensus.cc:357] T ec9f14c1ab014f29843a568c460a28d6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.945922 14812 raft_consensus.cc:383] T ec9f14c1ab014f29843a568c460a28d6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.945952 14812 raft_consensus.cc:738] T ec9f14c1ab014f29843a568c460a28d6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.946051 14812 consensus_queue.cc:260] T ec9f14c1ab014f29843a568c460a28d6 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.946275 14812 ts_tablet_manager.cc:1428] T ec9f14c1ab014f29843a568c460a28d6 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.946261 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6fe85b67783046f99e0b160677207732" } modify_external_catalogs: true
I20250115 03:06:15.946355 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6fe85b67783046f99e0b160677207732" } modify_external_catalogs: true
I20250115 03:06:15.946403 14812 tablet_bootstrap.cc:492] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.947135 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet e2c6db404348466ea5211ffa69c8f01c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.947220 14812 tablet_bootstrap.cc:654] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.948237 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22cc0833bfde458ebe3985c05d29b8e6" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:15.948343 14556 raft_consensus.cc:3054] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:15.950120 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet e2c6db404348466ea5211ffa69c8f01c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e2c6db404348466ea5211ffa69c8f01c already in progress: creating tablet
I20250115 03:06:15.950385 14556 raft_consensus.cc:2463] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:15.953418 14812 tablet_bootstrap.cc:492] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:15.953496 14812 ts_tablet_manager.cc:1397] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:15.953893 14812 raft_consensus.cc:357] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.953996 14812 raft_consensus.cc:383] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.954035 14812 raft_consensus.cc:738] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.954152 14812 consensus_queue.cc:260] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.954388 14812 ts_tablet_manager.cc:1428] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.954535 14812 tablet_bootstrap.cc:492] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:15.955451 14812 tablet_bootstrap.cc:654] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.956116 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_7a132f92dbad4a868f47b81f0107d42e" } modify_external_catalogs: true
I20250115 03:06:15.956205 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_7a132f92dbad4a868f47b81f0107d42e" } modify_external_catalogs: true
I20250115 03:06:15.958534 14163 catalog_manager.cc:5526] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:15.959148 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 21cc97960f7c49c7bf57499e66fd7190
I20250115 03:06:15.959280 15426 tablet.cc:1721] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:15.960019 15274 tablet.cc:1721] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:15.964039 15617 tablet.cc:1721] T 29ef42586d164315b5fd54ba91227bad P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:15.966243 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 312dcdda8cf042648db9fdf6228062d9 (DEFAULT_TABLE table=default.Table_5e89af766dcb414fb8dfb770c99ede52 [id=c096ce170874405ea740b9ea829f886e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.966627 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 623887ec17374f598f48054e17582f3f (DEFAULT_TABLE table=default.Table_0337ad43f9b646a48c61c66c15f1844a [id=7f5506ea0a5e465cba8de1ba2f5e75c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.966976 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 98913be4c8e74d7185957b0c4ad6abce (DEFAULT_TABLE table=default.Table_71fadacb26bd4e4a8ddcd48e403350bb [id=529f6a8366ca4afd8e51cc396a8bb7a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.967535 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 312dcdda8cf042648db9fdf6228062d9 (DEFAULT_TABLE table=default.Table_5e89af766dcb414fb8dfb770c99ede52 [id=c096ce170874405ea740b9ea829f886e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.967561 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 21cc97960f7c49c7bf57499e66fd7190 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.967778 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 312dcdda8cf042648db9fdf6228062d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.967818 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 15fbf8d5de904fc4b284af664f3bd60f (DEFAULT_TABLE table=default.Table_2299c208a52a4d99ba20b1f3b5783fbd [id=6ed01835736b40f8b4caeca7378770d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.968027 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15fbf8d5de904fc4b284af664f3bd60f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.968381 14397 tablet_service.cc:1467] Processing CreateTablet for tablet b56279bb16054869a97006d88574e381 (DEFAULT_TABLE table=default.Table_c0bde98045a84e568d03c9608c7731fb [id=080d0a4dfa174d3ba2979bb850d0604a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.968572 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b56279bb16054869a97006d88574e381. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.968626 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 6bcfde5fe3d4425e98edb4dc54096718 (DEFAULT_TABLE table=default.Table_a63d6344914343c4af56eef21f467c6f [id=f45053c781fc460ab55f592eed6fb946]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.968844 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bcfde5fe3d4425e98edb4dc54096718. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.971920 14811 tablet_bootstrap.cc:492] T f3ac7f1e1bfb4383895dc2c2123a743b P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:15.971997 14811 ts_tablet_manager.cc:1397] T f3ac7f1e1bfb4383895dc2c2123a743b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.072s user 0.001s sys 0.000s
I20250115 03:06:15.972347 14811 raft_consensus.cc:357] T f3ac7f1e1bfb4383895dc2c2123a743b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.972436 14811 raft_consensus.cc:383] T f3ac7f1e1bfb4383895dc2c2123a743b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:15.972463 14811 raft_consensus.cc:738] T f3ac7f1e1bfb4383895dc2c2123a743b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:15.972553 14811 consensus_queue.cc:260] T f3ac7f1e1bfb4383895dc2c2123a743b P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:15.972800 14811 ts_tablet_manager.cc:1428] T f3ac7f1e1bfb4383895dc2c2123a743b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:15.972991 14811 tablet_bootstrap.cc:492] T a2fc04950cfe4c608a4712afd257ca76 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:15.973206 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 21cc97960f7c49c7bf57499e66fd7190 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.973357 14823 tablet_replica.cc:331] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.973412 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 98913be4c8e74d7185957b0c4ad6abce (DEFAULT_TABLE table=default.Table_71fadacb26bd4e4a8ddcd48e403350bb [id=529f6a8366ca4afd8e51cc396a8bb7a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.973475 14823 raft_consensus.cc:2238] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:15.973613 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98913be4c8e74d7185957b0c4ad6abce. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.973621 14823 raft_consensus.cc:2267] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.973956 14823 ts_tablet_manager.cc:1905] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.976054 14823 ts_tablet_manager.cc:1918] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.976121 14823 log.cc:1198] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/21cc97960f7c49c7bf57499e66fd7190
I20250115 03:06:15.976356 14823 ts_tablet_manager.cc:1939] T 21cc97960f7c49c7bf57499e66fd7190 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:15.977161 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 312dcdda8cf042648db9fdf6228062d9 (DEFAULT_TABLE table=default.Table_5e89af766dcb414fb8dfb770c99ede52 [id=c096ce170874405ea740b9ea829f886e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.977324 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 90277f7d279e4c7a83c43f40766cd11b
I20250115 03:06:15.977375 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 312dcdda8cf042648db9fdf6228062d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.977246 14168 catalog_manager.cc:5526] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:15.977468 14168 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 22cc0833bfde458ebe3985c05d29b8e6 (table default.Table_3db38338e5b04298954aded6d5482169 [id=f56a0de6e56441bfa620902f175feb4a]). Expected version 2 got 0
I20250115 03:06:15.979487 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 98913be4c8e74d7185957b0c4ad6abce (DEFAULT_TABLE table=default.Table_71fadacb26bd4e4a8ddcd48e403350bb [id=529f6a8366ca4afd8e51cc396a8bb7a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.979704 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98913be4c8e74d7185957b0c4ad6abce. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.981775 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 15fbf8d5de904fc4b284af664f3bd60f (DEFAULT_TABLE table=default.Table_2299c208a52a4d99ba20b1f3b5783fbd [id=6ed01835736b40f8b4caeca7378770d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.981988 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15fbf8d5de904fc4b284af664f3bd60f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.981940 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_fd907927ba11487b907e7130bdfb6092"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:15.982244 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fd907927ba11487b907e7130bdfb6092 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:15.982650 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 623887ec17374f598f48054e17582f3f (DEFAULT_TABLE table=default.Table_0337ad43f9b646a48c61c66c15f1844a [id=7f5506ea0a5e465cba8de1ba2f5e75c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.982848 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 623887ec17374f598f48054e17582f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.982918 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_a7ee880814fc40f98518c5149b8b5b4b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:15.983129 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 90277f7d279e4c7a83c43f40766cd11b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:15.983179 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a7ee880814fc40f98518c5149b8b5b4b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:15.983656 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 90277f7d279e4c7a83c43f40766cd11b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 90277f7d279e4c7a83c43f40766cd11b already in progress: creating tablet
I20250115 03:06:15.983898 14163 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_070c9465ea944d50ad72cffc3a10681a [id=c7ebf406fe8c4b4f91ab34cae586b47a] alter complete (version 1)
I20250115 03:06:15.983903 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 90277f7d279e4c7a83c43f40766cd11b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.983963 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 21cc97960f7c49c7bf57499e66fd7190 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:15.984086 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 21cc97960f7c49c7bf57499e66fd7190 (table default.Table_6fe85b67783046f99e0b160677207732 [id=f89110bc2f1e4619a821be08eb35695f]) successfully deleted
I20250115 03:06:15.984155 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 6bcfde5fe3d4425e98edb4dc54096718 (DEFAULT_TABLE table=default.Table_a63d6344914343c4af56eef21f467c6f [id=f45053c781fc460ab55f592eed6fb946]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.984376 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bcfde5fe3d4425e98edb4dc54096718. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.984562 14819 tablet_replica.cc:331] T 21cc97960f7c49c7bf57499e66fd7190 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:15.984692 14819 raft_consensus.cc:2238] T 21cc97960f7c49c7bf57499e66fd7190 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.984784 14819 raft_consensus.cc:2267] T 21cc97960f7c49c7bf57499e66fd7190 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.985155 14819 ts_tablet_manager.cc:1905] T 21cc97960f7c49c7bf57499e66fd7190 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.986290 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 623887ec17374f598f48054e17582f3f (DEFAULT_TABLE table=default.Table_0337ad43f9b646a48c61c66c15f1844a [id=7f5506ea0a5e465cba8de1ba2f5e75c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.986496 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 623887ec17374f598f48054e17582f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.987044 14819 ts_tablet_manager.cc:1918] T 21cc97960f7c49c7bf57499e66fd7190 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:15.987110 14819 log.cc:1198] T 21cc97960f7c49c7bf57499e66fd7190 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/21cc97960f7c49c7bf57499e66fd7190
I20250115 03:06:15.987326 14819 ts_tablet_manager.cc:1939] T 21cc97960f7c49c7bf57499e66fd7190 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:15.988114 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 21cc97960f7c49c7bf57499e66fd7190 (table default.Table_6fe85b67783046f99e0b160677207732 [id=f89110bc2f1e4619a821be08eb35695f]) successfully deleted
I20250115 03:06:15.988960 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet ec9f14c1ab014f29843a568c460a28d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.989215 14823 tablet_replica.cc:331] T ec9f14c1ab014f29843a568c460a28d6 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:15.989305 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 6bcfde5fe3d4425e98edb4dc54096718 (DEFAULT_TABLE table=default.Table_a63d6344914343c4af56eef21f467c6f [id=f45053c781fc460ab55f592eed6fb946]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.989445 14528 tablet_service.cc:1467] Processing CreateTablet for tablet b56279bb16054869a97006d88574e381 (DEFAULT_TABLE table=default.Table_c0bde98045a84e568d03c9608c7731fb [id=080d0a4dfa174d3ba2979bb850d0604a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.989441 14823 raft_consensus.cc:2238] T ec9f14c1ab014f29843a568c460a28d6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:15.989598 14529 tablet_service.cc:1467] Processing CreateTablet for tablet 15fbf8d5de904fc4b284af664f3bd60f (DEFAULT_TABLE table=default.Table_2299c208a52a4d99ba20b1f3b5783fbd [id=6ed01835736b40f8b4caeca7378770d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.989773 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 90277f7d279e4c7a83c43f40766cd11b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:15.989647 14823 raft_consensus.cc:2267] T ec9f14c1ab014f29843a568c460a28d6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:15.990782 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f67cbb929cd4ae589a14606aa168b46 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:15.990923 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b7022db9edc4de8909e16769a3ea136 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:15.991248 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 90277f7d279e4c7a83c43f40766cd11b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 90277f7d279e4c7a83c43f40766cd11b already in progress: creating tablet
I20250115 03:06:15.991554 14823 ts_tablet_manager.cc:1905] T ec9f14c1ab014f29843a568c460a28d6 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:15.991555 14666 tablet_service.cc:1467] Processing CreateTablet for tablet b56279bb16054869a97006d88574e381 (DEFAULT_TABLE table=default.Table_c0bde98045a84e568d03c9608c7731fb [id=080d0a4dfa174d3ba2979bb850d0604a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:15.992563 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b56279bb16054869a97006d88574e381. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.994935 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 623887ec17374f598f48054e17582f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.995878 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bcfde5fe3d4425e98edb4dc54096718. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.996784 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98913be4c8e74d7185957b0c4ad6abce. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:15.996946 14820 ts_tablet_manager.cc:1918] T 99d9180ed9b24363a4c44d01ea1d6e9b P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:15.997007 14820 log.cc:1198] T 99d9180ed9b24363a4c44d01ea1d6e9b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/99d9180ed9b24363a4c44d01ea1d6e9b
I20250115 03:06:15.997181 14820 ts_tablet_manager.cc:1939] T 99d9180ed9b24363a4c44d01ea1d6e9b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:15.998078 14811 tablet_bootstrap.cc:654] T a2fc04950cfe4c608a4712afd257ca76 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:15.998414 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 312dcdda8cf042648db9fdf6228062d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.000909 14823 ts_tablet_manager.cc:1918] T ec9f14c1ab014f29843a568c460a28d6 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.001047 14823 log.cc:1198] T ec9f14c1ab014f29843a568c460a28d6 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/ec9f14c1ab014f29843a568c460a28d6
I20250115 03:06:16.001291 14823 ts_tablet_manager.cc:1939] T ec9f14c1ab014f29843a568c460a28d6 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.001802 14823 tablet_replica.cc:331] T 7f67cbb929cd4ae589a14606aa168b46 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.001956 14823 raft_consensus.cc:2238] T 7f67cbb929cd4ae589a14606aa168b46 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.002072 14823 raft_consensus.cc:2267] T 7f67cbb929cd4ae589a14606aa168b46 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.002494 14823 ts_tablet_manager.cc:1905] T 7f67cbb929cd4ae589a14606aa168b46 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.002943 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15fbf8d5de904fc4b284af664f3bd60f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.003755 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet ec9f14c1ab014f29843a568c460a28d6 (table default.Table_60943a26c80945ae84effaed75c0cc42 [id=d94e2b655f424f549af052bc8dee0430]) successfully deleted
I20250115 03:06:16.004249 14823 ts_tablet_manager.cc:1918] T 7f67cbb929cd4ae589a14606aa168b46 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.004302 14823 log.cc:1198] T 7f67cbb929cd4ae589a14606aa168b46 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/7f67cbb929cd4ae589a14606aa168b46
I20250115 03:06:16.004470 14823 ts_tablet_manager.cc:1939] T 7f67cbb929cd4ae589a14606aa168b46 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.005052 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 7f67cbb929cd4ae589a14606aa168b46 (table default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]) successfully deleted
I20250115 03:06:16.007705 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet 3a0bb9b5093b454d921bf1150d631565 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:16.007895 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ad312d1f941c45b5a0dc5324967d5105" } new_table_name: "default.Table_551ca9fefcb74ed5aa29803f19de6ab6" modify_external_catalogs: true
I20250115 03:06:16.008868 14162 master_service.cc:946] ReplaceTablet: received request to replace tablet 8baf86b37e9942ba813b099a0da87ca5 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:16.010031 14823 tablet_replica.cc:331] T 5b7022db9edc4de8909e16769a3ea136 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.010143 14823 raft_consensus.cc:2238] T 5b7022db9edc4de8909e16769a3ea136 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.010216 14823 raft_consensus.cc:2267] T 5b7022db9edc4de8909e16769a3ea136 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.010637 14823 ts_tablet_manager.cc:1905] T 5b7022db9edc4de8909e16769a3ea136 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.012298 14823 ts_tablet_manager.cc:1918] T 5b7022db9edc4de8909e16769a3ea136 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.012363 14823 log.cc:1198] T 5b7022db9edc4de8909e16769a3ea136 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/5b7022db9edc4de8909e16769a3ea136
I20250115 03:06:16.012524 14823 ts_tablet_manager.cc:1939] T 5b7022db9edc4de8909e16769a3ea136 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.013062 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 5b7022db9edc4de8909e16769a3ea136 (table default.Table_a5cd96954b1c48c68a3bef815c47d8b4 [id=f94088b0776f4be990ddcec04afbca43]) successfully deleted
I20250115 03:06:16.015892 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b56279bb16054869a97006d88574e381. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.018139 14809 tablet_bootstrap.cc:492] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.018218 14809 ts_tablet_manager.cc:1397] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.116s user 0.001s sys 0.000s
I20250115 03:06:16.018667 14809 raft_consensus.cc:357] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.018769 14809 raft_consensus.cc:383] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.018796 14809 raft_consensus.cc:738] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.018903 14809 consensus_queue.cc:260] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.019165 14809 ts_tablet_manager.cc:1428] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.019318 14809 tablet_bootstrap.cc:492] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.020176 14809 tablet_bootstrap.cc:654] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.022382 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 99d9180ed9b24363a4c44d01ea1d6e9b (table default.Table_6bb3a06c1ae749e593cdc3f2b3992eaf [id=89578bd1e7e64c0fa550f20f592ccc6f]) successfully deleted
W20250115 03:06:16.022984 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 054251151c294e06905092f2a9306849 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 054251151c294e06905092f2a9306849 already in progress: creating tablet
I20250115 03:06:16.023314 14820 tablet_replica.cc:331] T 3e46d4a75e9b4ca8b83a188d06280fcd P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.023664 14820 raft_consensus.cc:2238] T 3e46d4a75e9b4ca8b83a188d06280fcd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.026590 14170 master_service.cc:946] ReplaceTablet: received request to replace tablet 8baf86b37e9942ba813b099a0da87ca5 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:16.024019 14820 raft_consensus.cc:2267] T 3e46d4a75e9b4ca8b83a188d06280fcd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.028191 15153 consensus_queue.cc:1035] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.029302 14820 ts_tablet_manager.cc:1905] T 3e46d4a75e9b4ca8b83a188d06280fcd P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.029906 14809 tablet_bootstrap.cc:492] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.029980 14809 ts_tablet_manager.cc:1397] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20250115 03:06:16.030368 14809 raft_consensus.cc:357] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.030467 14809 raft_consensus.cc:383] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.030493 14809 raft_consensus.cc:738] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.030622 14809 consensus_queue.cc:260] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.031277 14809 ts_tablet_manager.cc:1428] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.031453 14809 tablet_bootstrap.cc:492] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.032004 14691 raft_consensus.cc:1270] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250115 03:06:16.032519 14356 consensus_peers.cc:487] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:16.034026 14809 tablet_bootstrap.cc:654] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.034828 15153 consensus_queue.cc:1035] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.034967 14809 tablet_bootstrap.cc:492] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.034960 14885 consensus_queue.cc:1035] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.035053 14809 ts_tablet_manager.cc:1397] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20250115 03:06:16.035491 14809 raft_consensus.cc:357] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.035591 14809 raft_consensus.cc:383] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.035629 14809 raft_consensus.cc:738] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.035744 14809 consensus_queue.cc:260] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.036006 14809 ts_tablet_manager.cc:1428] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.036155 14809 tablet_bootstrap.cc:492] T f5af8b81e7ff4acab5aef13003656988 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.036998 14809 tablet_bootstrap.cc:654] T f5af8b81e7ff4acab5aef13003656988 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.037724 15153 raft_consensus.cc:491] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.037798 15153 raft_consensus.cc:513] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.038103 15153 leader_election.cc:290] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:16.038421 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa08518530ae4d5487af386a650c4672" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:16.038743 14628 leader_election.cc:343] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.039086 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa08518530ae4d5487af386a650c4672" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:16.039445 14626 leader_election.cc:343] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.039515 14626 leader_election.cc:304] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.039686 15153 raft_consensus.cc:2743] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.041378 14820 ts_tablet_manager.cc:1918] T 3e46d4a75e9b4ca8b83a188d06280fcd P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.041508 14820 log.cc:1198] T 3e46d4a75e9b4ca8b83a188d06280fcd P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/3e46d4a75e9b4ca8b83a188d06280fcd
I20250115 03:06:16.041759 14820 ts_tablet_manager.cc:1939] T 3e46d4a75e9b4ca8b83a188d06280fcd P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.042356 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 3e46d4a75e9b4ca8b83a188d06280fcd (table default.Table_0d7244964d874f5eb1f508e5c0dabf1a [id=15762a37e62f4e89852b2eb66a008dbb]) successfully deleted
W20250115 03:06:16.042819 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet a75b9401cb4b4f8b92a4779c3704eb46 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a75b9401cb4b4f8b92a4779c3704eb46 already in progress: creating tablet
W20250115 03:06:16.043231 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet e2c6db404348466ea5211ffa69c8f01c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e2c6db404348466ea5211ffa69c8f01c already in progress: creating tablet
I20250115 03:06:16.043454 14820 tablet_replica.cc:331] T 21cc97960f7c49c7bf57499e66fd7190 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.043656 14820 raft_consensus.cc:2238] T 21cc97960f7c49c7bf57499e66fd7190 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.043762 14820 raft_consensus.cc:2267] T 21cc97960f7c49c7bf57499e66fd7190 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.044111 14820 ts_tablet_manager.cc:1905] T 21cc97960f7c49c7bf57499e66fd7190 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.044409 14809 tablet_bootstrap.cc:492] T f5af8b81e7ff4acab5aef13003656988 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.044476 14809 ts_tablet_manager.cc:1397] T f5af8b81e7ff4acab5aef13003656988 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20250115 03:06:16.044828 15153 raft_consensus.cc:491] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.044899 15153 raft_consensus.cc:513] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.044893 14809 raft_consensus.cc:357] T f5af8b81e7ff4acab5aef13003656988 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.045001 14809 raft_consensus.cc:383] T f5af8b81e7ff4acab5aef13003656988 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.045028 14809 raft_consensus.cc:738] T f5af8b81e7ff4acab5aef13003656988 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.045117 14809 consensus_queue.cc:260] T f5af8b81e7ff4acab5aef13003656988 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.045207 15153 leader_election.cc:290] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:16.045353 14809 ts_tablet_manager.cc:1428] T f5af8b81e7ff4acab5aef13003656988 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.045501 14809 tablet_bootstrap.cc:492] T 68974a1815cb482d8b28d54f42634608 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.045523 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ce129d8a9245baab7c772f0f5fbaa7" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:16.045820 14628 leader_election.cc:343] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.046306 14809 tablet_bootstrap.cc:654] T 68974a1815cb482d8b28d54f42634608 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.049691 15617 tablet.cc:1721] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:16.051196 14809 tablet_bootstrap.cc:492] T 68974a1815cb482d8b28d54f42634608 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.051271 14809 ts_tablet_manager.cc:1397] T 68974a1815cb482d8b28d54f42634608 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20250115 03:06:16.051358 15274 tablet.cc:1721] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:16.051667 14809 raft_consensus.cc:357] T 68974a1815cb482d8b28d54f42634608 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.051764 14809 raft_consensus.cc:383] T 68974a1815cb482d8b28d54f42634608 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.051797 14809 raft_consensus.cc:738] T 68974a1815cb482d8b28d54f42634608 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.051891 14809 consensus_queue.cc:260] T 68974a1815cb482d8b28d54f42634608 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.051970 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_39adda20d0304f8f8785171a07e25531"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:16.052134 14809 ts_tablet_manager.cc:1428] T 68974a1815cb482d8b28d54f42634608 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.052286 14809 tablet_bootstrap.cc:492] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
W20250115 03:06:16.052337 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_39adda20d0304f8f8785171a07e25531 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.052582 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d85908ffc4694aaabd57c6ae118cdb06" } new_table_name: "default.Table_76ef9277dc8142f1a86962bfc430f2b2" modify_external_catalogs: true
I20250115 03:06:16.054877 14809 tablet_bootstrap.cc:654] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.055562 14812 tablet_bootstrap.cc:492] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.055646 14812 ts_tablet_manager.cc:1397] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.101s user 0.001s sys 0.000s
I20250115 03:06:16.056068 14812 raft_consensus.cc:357] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.056164 14812 raft_consensus.cc:383] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.056191 14812 raft_consensus.cc:738] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.056293 14812 consensus_queue.cc:260] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.056581 14812 ts_tablet_manager.cc:1428] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.056751 14812 tablet_bootstrap.cc:492] T 84ce55f729224f78b420e57977419add P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.057624 14812 tablet_bootstrap.cc:654] T 84ce55f729224f78b420e57977419add P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.059576 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_70287f2d5a024cc88244993789eb747b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.059882 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_70287f2d5a024cc88244993789eb747b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.060781 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e73230225df744a2a11f246bea89b8b0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.061074 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e73230225df744a2a11f246bea89b8b0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.065461 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_1ad8c0817ede4bbd847c01814eebfa83" } new_table_name: "default.Table_d407111658854f02aa8387a067f33f37" modify_external_catalogs: true
I20250115 03:06:16.073972 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 8baf86b37e9942ba813b099a0da87ca5
I20250115 03:06:16.074213 14170 catalog_manager.cc:6596] ReplaceTablet: tablet 8baf86b37e9942ba813b099a0da87ca5 deleted and replaced by tablet bfc34dc76ff94b8eaeb5c2822e6498da
I20250115 03:06:16.074908 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_24b2f1f4bbb14d2fa7cba35e089f6707"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.075189 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_24b2f1f4bbb14d2fa7cba35e089f6707 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.075309 14170 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 4ff51d514fb84809aa53fa9fc1010681 (table default.Table_9cc1022af4f14e7a8a8e11cc1803ca87 [id=98f48acd89a6461bb4511141cab4d88b]). Expected version 2 got 0
I20250115 03:06:16.075845 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 3a0bb9b5093b454d921bf1150d631565
I20250115 03:06:16.076071 14169 catalog_manager.cc:6596] ReplaceTablet: tablet 3a0bb9b5093b454d921bf1150d631565 deleted and replaced by tablet 5589b497b0e24740ad49d1392f5a737b
I20250115 03:06:16.076239 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_27b5acf9bde44139b1b076e9579d88c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.076500 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_27b5acf9bde44139b1b076e9579d88c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.076855 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a0bb9b5093b454d921bf1150d631565 with delete_type TABLET_DATA_DELETED (replaced by tablet 5589b497b0e24740ad49d1392f5a737b) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.076869 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d49d96cab47241d4836768cbc9cd552b" } new_table_name: "default.Table_e5c55332e306452685635c28fb3d4c55" modify_external_catalogs: true
W20250115 03:06:16.077241 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 3a0bb9b5093b454d921bf1150d631565 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3a0bb9b5093b454d921bf1150d631565 already in progress: creating tablet
I20250115 03:06:16.077302 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 8baf86b37e9942ba813b099a0da87ca5 with delete_type TABLET_DATA_DELETED (replaced by tablet bfc34dc76ff94b8eaeb5c2822e6498da) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:16.077471 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a4c3ff892f9b4dc484ffc80d7c5934c4 consensus state has no leader
I20250115 03:06:16.077461 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a0bb9b5093b454d921bf1150d631565 with delete_type TABLET_DATA_DELETED (replaced by tablet 5589b497b0e24740ad49d1392f5a737b) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.077586 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet 8baf86b37e9942ba813b099a0da87ca5 with delete_type TABLET_DATA_DELETED (replaced by tablet bfc34dc76ff94b8eaeb5c2822e6498da) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.077775 14823 tablet_replica.cc:331] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.077883 14823 raft_consensus.cc:2238] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.077960 14823 raft_consensus.cc:2267] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:16.078102 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 8baf86b37e9942ba813b099a0da87ca5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8baf86b37e9942ba813b099a0da87ca5 already in progress: creating tablet
W20250115 03:06:16.078249 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 3a0bb9b5093b454d921bf1150d631565 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3a0bb9b5093b454d921bf1150d631565 already in progress: creating tablet
I20250115 03:06:16.078362 14823 ts_tablet_manager.cc:1905] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:16.080267 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5780da67ce984297ba33d3c6901c9e3a consensus state has no leader
I20250115 03:06:16.080531 14823 ts_tablet_manager.cc:1918] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.080600 14823 log.cc:1198] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/8baf86b37e9942ba813b099a0da87ca5
I20250115 03:06:16.080778 14823 ts_tablet_manager.cc:1939] T 8baf86b37e9942ba813b099a0da87ca5 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.081346 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 8baf86b37e9942ba813b099a0da87ca5 (table default.Table_a1547d1d923f4f4da05aaeb519f4cdf0 [id=e929ffaba567498c815393836b264a12]) successfully deleted
I20250115 03:06:16.082407 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5a68d6c8c53b4746a7123860185fe0cc" } modify_external_catalogs: true
I20250115 03:06:16.082510 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5a68d6c8c53b4746a7123860185fe0cc" } modify_external_catalogs: true
W20250115 03:06:16.085242 14163 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 304fab45b38e4426ae6961ae786e5659 consensus state has no leader
I20250115 03:06:16.046141 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ce129d8a9245baab7c772f0f5fbaa7" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.078951 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet 8baf86b37e9942ba813b099a0da87ca5 with delete_type TABLET_DATA_DELETED (replaced by tablet bfc34dc76ff94b8eaeb5c2822e6498da) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.079510 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a0bb9b5093b454d921bf1150d631565 with delete_type TABLET_DATA_DELETED (replaced by tablet 5589b497b0e24740ad49d1392f5a737b) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:16.088953 14626 leader_election.cc:343] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.089022 14626 leader_election.cc:304] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.088408 14811 tablet_bootstrap.cc:492] T a2fc04950cfe4c608a4712afd257ca76 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.089293 14811 ts_tablet_manager.cc:1397] T a2fc04950cfe4c608a4712afd257ca76 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.116s user 0.003s sys 0.000s
I20250115 03:06:16.089702 14811 raft_consensus.cc:357] T a2fc04950cfe4c608a4712afd257ca76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.089799 14811 raft_consensus.cc:383] T a2fc04950cfe4c608a4712afd257ca76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.089830 14811 raft_consensus.cc:738] T a2fc04950cfe4c608a4712afd257ca76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.089936 14811 consensus_queue.cc:260] T a2fc04950cfe4c608a4712afd257ca76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.090188 14811 ts_tablet_manager.cc:1428] T a2fc04950cfe4c608a4712afd257ca76 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.090341 14811 tablet_bootstrap.cc:492] T 17b5e362481c49d2a3a13b1550ceef23 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.091105 14811 tablet_bootstrap.cc:654] T 17b5e362481c49d2a3a13b1550ceef23 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.091678 15153 raft_consensus.cc:2743] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.094025 14163 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet a2fc04950cfe4c608a4712afd257ca76 (table default.Table_6b1f81ee12e04acda4bcf7d7f289a67a [id=8f811bda53fc4d76a88a3dcf69aa3dc6]). Expected version 1 got 0
I20250115 03:06:16.095875 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_79d7aa1c665d4695a2a17501157b3807" } modify_external_catalogs: true
I20250115 03:06:16.095963 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_79d7aa1c665d4695a2a17501157b3807" } modify_external_catalogs: true
I20250115 03:06:16.097200 14166 master_service.cc:946] ReplaceTablet: received request to replace tablet 30499e99ab7d4138a376dea40b768c32 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:16.097463 14811 tablet_bootstrap.cc:492] T 17b5e362481c49d2a3a13b1550ceef23 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.097560 14811 ts_tablet_manager.cc:1397] T 17b5e362481c49d2a3a13b1550ceef23 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:16.097785 14812 tablet_bootstrap.cc:492] T 84ce55f729224f78b420e57977419add P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.097857 14812 ts_tablet_manager.cc:1397] T 84ce55f729224f78b420e57977419add P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.041s user 0.001s sys 0.000s
I20250115 03:06:16.097968 14811 raft_consensus.cc:357] T 17b5e362481c49d2a3a13b1550ceef23 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.098052 14811 raft_consensus.cc:383] T 17b5e362481c49d2a3a13b1550ceef23 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.098083 14811 raft_consensus.cc:738] T 17b5e362481c49d2a3a13b1550ceef23 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.098233 14820 ts_tablet_manager.cc:1918] T 21cc97960f7c49c7bf57499e66fd7190 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:16.098279 14812 raft_consensus.cc:357] T 84ce55f729224f78b420e57977419add P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.098367 14812 raft_consensus.cc:383] T 84ce55f729224f78b420e57977419add P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.098394 14812 raft_consensus.cc:738] T 84ce55f729224f78b420e57977419add P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.098546 14812 consensus_queue.cc:260] T 84ce55f729224f78b420e57977419add P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.098811 14812 ts_tablet_manager.cc:1428] T 84ce55f729224f78b420e57977419add P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.098332 14811 consensus_queue.cc:260] T 17b5e362481c49d2a3a13b1550ceef23 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.099644 14811 ts_tablet_manager.cc:1428] T 17b5e362481c49d2a3a13b1550ceef23 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:16.099807 14811 tablet_bootstrap.cc:492] T 82c2f41769d84bd48bfc1bc7f504be4b P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.100623 14811 tablet_bootstrap.cc:654] T 82c2f41769d84bd48bfc1bc7f504be4b P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.101178 14812 tablet_bootstrap.cc:492] T 82be772e05d443749914004aea8af300 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.101815 14820 log.cc:1198] T 21cc97960f7c49c7bf57499e66fd7190 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/21cc97960f7c49c7bf57499e66fd7190
I20250115 03:06:16.102118 14820 ts_tablet_manager.cc:1939] T 21cc97960f7c49c7bf57499e66fd7190 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.105690 14812 tablet_bootstrap.cc:654] T 82be772e05d443749914004aea8af300 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.106323 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 21cc97960f7c49c7bf57499e66fd7190 (table default.Table_6fe85b67783046f99e0b160677207732 [id=f89110bc2f1e4619a821be08eb35695f]) successfully deleted
I20250115 03:06:16.107581 14170 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_6b1f81ee12e04acda4bcf7d7f289a67a [id=8f811bda53fc4d76a88a3dcf69aa3dc6] alter complete (version 1)
I20250115 03:06:16.107789 14691 raft_consensus.cc:3054] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.107892 14809 tablet_bootstrap.cc:492] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
W20250115 03:06:16.107878 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 90277f7d279e4c7a83c43f40766cd11b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 90277f7d279e4c7a83c43f40766cd11b already in progress: creating tablet
I20250115 03:06:16.107965 14809 ts_tablet_manager.cc:1397] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.056s user 0.001s sys 0.000s
W20250115 03:06:16.108295 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 8baf86b37e9942ba813b099a0da87ca5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8baf86b37e9942ba813b099a0da87ca5 already in progress: creating tablet
I20250115 03:06:16.108346 14809 raft_consensus.cc:357] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.108440 14809 raft_consensus.cc:383] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.108469 14809 raft_consensus.cc:738] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.108573 14809 consensus_queue.cc:260] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.108851 14809 ts_tablet_manager.cc:1428] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.109061 14809 tablet_bootstrap.cc:492] T 9808c7f96a3147848da5f78948895699 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.109328 15390 consensus_queue.cc:1035] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.109939 14809 tablet_bootstrap.cc:654] T 9808c7f96a3147848da5f78948895699 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:16.110276 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 3a0bb9b5093b454d921bf1150d631565 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3a0bb9b5093b454d921bf1150d631565 already in progress: creating tablet
I20250115 03:06:16.110726 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet f5af8b81e7ff4acab5aef13003656988 with delete_type TABLET_DATA_DELETED (replaced by tablet 3efc2fb8427f493ea933347bec34f88f) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.111053 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 68974a1815cb482d8b28d54f42634608 with delete_type TABLET_DATA_DELETED (replaced by tablet 420b7c8d43a04bae972eed2014d5634c) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.111707 14819 tablet_replica.cc:331] T f5af8b81e7ff4acab5aef13003656988 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.111831 14819 raft_consensus.cc:2238] T f5af8b81e7ff4acab5aef13003656988 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.111909 14819 raft_consensus.cc:2267] T f5af8b81e7ff4acab5aef13003656988 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.112438 14819 ts_tablet_manager.cc:1905] T f5af8b81e7ff4acab5aef13003656988 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.113143 14811 tablet_bootstrap.cc:492] T 82c2f41769d84bd48bfc1bc7f504be4b P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.113209 14811 ts_tablet_manager.cc:1397] T 82c2f41769d84bd48bfc1bc7f504be4b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.001s
I20250115 03:06:16.113595 14811 raft_consensus.cc:357] T 82c2f41769d84bd48bfc1bc7f504be4b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.113682 14811 raft_consensus.cc:383] T 82c2f41769d84bd48bfc1bc7f504be4b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.113711 14811 raft_consensus.cc:738] T 82c2f41769d84bd48bfc1bc7f504be4b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.113814 14811 consensus_queue.cc:260] T 82c2f41769d84bd48bfc1bc7f504be4b P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.114069 14811 ts_tablet_manager.cc:1428] T 82c2f41769d84bd48bfc1bc7f504be4b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.114225 14811 tablet_bootstrap.cc:492] T 2d78a7793f84424bad42a1aba89eab92 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.114404 14819 ts_tablet_manager.cc:1918] T f5af8b81e7ff4acab5aef13003656988 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.114456 14819 log.cc:1198] T f5af8b81e7ff4acab5aef13003656988 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/f5af8b81e7ff4acab5aef13003656988
I20250115 03:06:16.114620 14819 ts_tablet_manager.cc:1939] T f5af8b81e7ff4acab5aef13003656988 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.115190 14811 tablet_bootstrap.cc:654] T 2d78a7793f84424bad42a1aba89eab92 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.117071 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet f5af8b81e7ff4acab5aef13003656988 (table default.Table_f4fc7a15692446fba7fcdb17b6b33d56 [id=364a7905f3f044f684e9c75927f7efda]) successfully deleted
I20250115 03:06:16.117355 14554 raft_consensus.cc:1270] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer 5c0dae7aff4b44c699275b95ba639770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:16.117370 14819 tablet_replica.cc:331] T 68974a1815cb482d8b28d54f42634608 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.117480 14819 raft_consensus.cc:2238] T 68974a1815cb482d8b28d54f42634608 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.117558 14819 raft_consensus.cc:2267] T 68974a1815cb482d8b28d54f42634608 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.117738 14421 raft_consensus.cc:1270] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 5c0dae7aff4b44c699275b95ba639770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:16.118044 15325 raft_consensus.cc:491] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.118119 15325 raft_consensus.cc:513] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.118373 14819 ts_tablet_manager.cc:1905] T 68974a1815cb482d8b28d54f42634608 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.118433 15325 leader_election.cc:290] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:16.118516 15325 consensus_queue.cc:1035] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.119532 15401 consensus_queue.cc:1035] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.121016 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3ba1b2ff964d648e5f36f410107b5b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.121490 14819 ts_tablet_manager.cc:1918] T 68974a1815cb482d8b28d54f42634608 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.121563 14819 log.cc:1198] T 68974a1815cb482d8b28d54f42634608 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/68974a1815cb482d8b28d54f42634608
I20250115 03:06:16.121621 15153 consensus_queue.cc:1035] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.121760 14819 ts_tablet_manager.cc:1939] T 68974a1815cb482d8b28d54f42634608 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.122107 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3ba1b2ff964d648e5f36f410107b5b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:16.122154 14811 tablet_bootstrap.cc:492] T 2d78a7793f84424bad42a1aba89eab92 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.122223 14811 ts_tablet_manager.cc:1397] T 2d78a7793f84424bad42a1aba89eab92 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20250115 03:06:16.122380 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 68974a1815cb482d8b28d54f42634608 (table default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]) successfully deleted
I20250115 03:06:16.122594 14811 raft_consensus.cc:357] T 2d78a7793f84424bad42a1aba89eab92 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.122678 14811 raft_consensus.cc:383] T 2d78a7793f84424bad42a1aba89eab92 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.122707 14811 raft_consensus.cc:738] T 2d78a7793f84424bad42a1aba89eab92 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.122808 14811 consensus_queue.cc:260] T 2d78a7793f84424bad42a1aba89eab92 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.123065 14811 ts_tablet_manager.cc:1428] T 2d78a7793f84424bad42a1aba89eab92 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.123203 14811 tablet_bootstrap.cc:492] T b1644ba8616349dc9d10a790b6089571 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
W20250115 03:06:16.123385 14628 leader_election.cc:343] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.124029 14811 tablet_bootstrap.cc:654] T b1644ba8616349dc9d10a790b6089571 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:16.126152 14626 leader_election.cc:343] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.126216 14626 leader_election.cc:304] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.126662 15153 raft_consensus.cc:2743] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.128041 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet 82c2f41769d84bd48bfc1bc7f504be4b with delete_type TABLET_DATA_DELETED (replaced by tablet 80aa570e2c154172ad6da29d268bede7) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.129088 14820 tablet_replica.cc:331] T 82c2f41769d84bd48bfc1bc7f504be4b P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.129294 14820 raft_consensus.cc:2238] T 82c2f41769d84bd48bfc1bc7f504be4b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.129429 14820 raft_consensus.cc:2267] T 82c2f41769d84bd48bfc1bc7f504be4b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.129871 14820 ts_tablet_manager.cc:1905] T 82c2f41769d84bd48bfc1bc7f504be4b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.130391 14165 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 22cc0833bfde458ebe3985c05d29b8e6 (table default.Table_3db38338e5b04298954aded6d5482169 [id=f56a0de6e56441bfa620902f175feb4a]). Expected version 2 got 0
I20250115 03:06:16.131987 14820 ts_tablet_manager.cc:1918] T 82c2f41769d84bd48bfc1bc7f504be4b P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.132325 14820 log.cc:1198] T 82c2f41769d84bd48bfc1bc7f504be4b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/82c2f41769d84bd48bfc1bc7f504be4b
I20250115 03:06:16.132588 14820 ts_tablet_manager.cc:1939] T 82c2f41769d84bd48bfc1bc7f504be4b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.133246 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 82c2f41769d84bd48bfc1bc7f504be4b (table default.Table_24cbe6b962b248a9b340506b63c41b20 [id=9866275e65ab4dfcbfee6bf209c3654e]) successfully deleted
I20250115 03:06:16.133389 14809 tablet_bootstrap.cc:492] T 9808c7f96a3147848da5f78948895699 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.133450 14809 ts_tablet_manager.cc:1397] T 9808c7f96a3147848da5f78948895699 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.002s
I20250115 03:06:16.133474 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 22cc0833bfde458ebe3985c05d29b8e6 (table default.Table_3db38338e5b04298954aded6d5482169 [id=f56a0de6e56441bfa620902f175feb4a]). Expected version 2 got 0
I20250115 03:06:16.133778 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d78a7793f84424bad42a1aba89eab92 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.133817 14809 raft_consensus.cc:357] T 9808c7f96a3147848da5f78948895699 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.133900 14809 raft_consensus.cc:383] T 9808c7f96a3147848da5f78948895699 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.133922 14820 tablet_replica.cc:331] T 2d78a7793f84424bad42a1aba89eab92 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.133934 14809 raft_consensus.cc:738] T 9808c7f96a3147848da5f78948895699 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.134016 14820 raft_consensus.cc:2238] T 2d78a7793f84424bad42a1aba89eab92 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.134020 14809 consensus_queue.cc:260] T 9808c7f96a3147848da5f78948895699 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.134095 14820 raft_consensus.cc:2267] T 2d78a7793f84424bad42a1aba89eab92 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.134232 14809 ts_tablet_manager.cc:1428] T 9808c7f96a3147848da5f78948895699 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.134385 14809 tablet_bootstrap.cc:492] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.135208 14809 tablet_bootstrap.cc:654] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.136251 14809 tablet_bootstrap.cc:492] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.136318 14809 ts_tablet_manager.cc:1397] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:16.136657 14809 raft_consensus.cc:357] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.136741 14809 raft_consensus.cc:383] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.136766 14809 raft_consensus.cc:738] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.136864 14820 ts_tablet_manager.cc:1905] T 2d78a7793f84424bad42a1aba89eab92 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.136854 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 9808c7f96a3147848da5f78948895699 with delete_type TABLET_DATA_DELETED (replaced by tablet 341b24d9b63f4761b2013035023fcd98) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.137053 14819 tablet_replica.cc:331] T 9808c7f96a3147848da5f78948895699 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.137151 14819 raft_consensus.cc:2238] T 9808c7f96a3147848da5f78948895699 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.137214 14819 raft_consensus.cc:2267] T 9808c7f96a3147848da5f78948895699 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.136854 14809 consensus_queue.cc:260] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.137562 14809 ts_tablet_manager.cc:1428] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.137684 14809 tablet_bootstrap.cc:492] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.138464 14809 tablet_bootstrap.cc:654] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.138738 14820 ts_tablet_manager.cc:1918] T 2d78a7793f84424bad42a1aba89eab92 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.138800 14820 log.cc:1198] T 2d78a7793f84424bad42a1aba89eab92 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/2d78a7793f84424bad42a1aba89eab92
I20250115 03:06:16.138974 14820 ts_tablet_manager.cc:1939] T 2d78a7793f84424bad42a1aba89eab92 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.139503 14819 ts_tablet_manager.cc:1905] T 9808c7f96a3147848da5f78948895699 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.139595 14812 tablet_bootstrap.cc:492] T 82be772e05d443749914004aea8af300 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.139673 14812 ts_tablet_manager.cc:1397] T 82be772e05d443749914004aea8af300 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.000s
I20250115 03:06:16.140089 14812 raft_consensus.cc:357] T 82be772e05d443749914004aea8af300 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.140190 14812 raft_consensus.cc:383] T 82be772e05d443749914004aea8af300 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.140224 14812 raft_consensus.cc:738] T 82be772e05d443749914004aea8af300 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
W20250115 03:06:16.141009 14167 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 13ce129d8a9245baab7c772f0f5fbaa7 consensus state has no leader
I20250115 03:06:16.141032 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e83118015644482ca56543e76b9691bd" } modify_external_catalogs: true
I20250115 03:06:16.141170 14819 ts_tablet_manager.cc:1918] T 9808c7f96a3147848da5f78948895699 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.141222 14819 log.cc:1198] T 9808c7f96a3147848da5f78948895699 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/9808c7f96a3147848da5f78948895699
I20250115 03:06:16.141378 14819 ts_tablet_manager.cc:1939] T 9808c7f96a3147848da5f78948895699 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.140352 14812 consensus_queue.cc:260] T 82be772e05d443749914004aea8af300 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.142051 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 9808c7f96a3147848da5f78948895699 (table default.Table_af3419632bb0478bab77f1fa32e9fb52 [id=369c42ec23ec4a27a71a54be23a410a6]) successfully deleted
I20250115 03:06:16.143674 14809 tablet_bootstrap.cc:492] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.143749 14809 ts_tablet_manager.cc:1397] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:16.144080 14809 raft_consensus.cc:357] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.144156 14809 raft_consensus.cc:383] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.144182 14809 raft_consensus.cc:738] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.144290 14809 consensus_queue.cc:260] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.144521 14809 ts_tablet_manager.cc:1428] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:16.144526 14812 ts_tablet_manager.cc:1428] T 82be772e05d443749914004aea8af300 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:16.144649 14809 tablet_bootstrap.cc:492] T 79579515fb7e45a8b16fd36dbc6a965a P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.145462 14809 tablet_bootstrap.cc:654] T 79579515fb7e45a8b16fd36dbc6a965a P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.141109 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e83118015644482ca56543e76b9691bd" } modify_external_catalogs: true
I20250115 03:06:16.145874 14812 tablet_bootstrap.cc:492] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.146757 14812 tablet_bootstrap.cc:654] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.147303 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 858bbbb62fc249d5a915697c12362d0f
I20250115 03:06:16.147652 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 62ec9057d85447bdb6b86cdc1f9015ac
I20250115 03:06:16.148124 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 62ec9057d85447bdb6b86cdc1f9015ac with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.148202 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 858bbbb62fc249d5a915697c12362d0f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.148344 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 62ec9057d85447bdb6b86cdc1f9015ac with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:16.148514 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 62ec9057d85447bdb6b86cdc1f9015ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62ec9057d85447bdb6b86cdc1f9015ac already in progress: creating tablet
W20250115 03:06:16.148649 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 858bbbb62fc249d5a915697c12362d0f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 858bbbb62fc249d5a915697c12362d0f already in progress: creating tablet
W20250115 03:06:16.148756 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 62ec9057d85447bdb6b86cdc1f9015ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62ec9057d85447bdb6b86cdc1f9015ac already in progress: creating tablet
I20250115 03:06:16.148139 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet 858bbbb62fc249d5a915697c12362d0f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:16.149226 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 858bbbb62fc249d5a915697c12362d0f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 858bbbb62fc249d5a915697c12362d0f already in progress: creating tablet
I20250115 03:06:16.149454 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f2946c2c8c56478f9108ffeda4034086"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:16.149937 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 2d78a7793f84424bad42a1aba89eab92 (table default.Table_638e39bcf2aa4131ac57afbe63ac9b73 [id=50928432524e42d79855e9143c537b61]) successfully deleted
I20250115 03:06:16.150130 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_7eacd8191bf44769b6371caa6d4772f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.150408 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7eacd8191bf44769b6371caa6d4772f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.150524 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet 858bbbb62fc249d5a915697c12362d0f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.150656 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet 62ec9057d85447bdb6b86cdc1f9015ac with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:16.151001 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f2946c2c8c56478f9108ffeda4034086 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:16.151010 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 858bbbb62fc249d5a915697c12362d0f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 858bbbb62fc249d5a915697c12362d0f already in progress: creating tablet
W20250115 03:06:16.151172 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 62ec9057d85447bdb6b86cdc1f9015ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62ec9057d85447bdb6b86cdc1f9015ac already in progress: creating tablet
I20250115 03:06:16.152580 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0bc1b60fff5a46129e157cf221391d0d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.152853 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0bc1b60fff5a46129e157cf221391d0d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.154598 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5e89af766dcb414fb8dfb770c99ede52" } new_table_name: "default.Table_719936ebaae74a91821a75f9f8edc316" modify_external_catalogs: true
I20250115 03:06:16.157500 15153 raft_consensus.cc:491] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.157596 15153 raft_consensus.cc:513] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.157923 15153 leader_election.cc:290] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:16.158324 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "872ad705e3e94c0eab4397df9f28f9ca" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:16.158644 14628 leader_election.cc:343] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.160017 14809 tablet_bootstrap.cc:492] T 79579515fb7e45a8b16fd36dbc6a965a P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.160079 14809 ts_tablet_manager.cc:1397] T 79579515fb7e45a8b16fd36dbc6a965a P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20250115 03:06:16.160478 14809 raft_consensus.cc:357] T 79579515fb7e45a8b16fd36dbc6a965a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.160573 14809 raft_consensus.cc:383] T 79579515fb7e45a8b16fd36dbc6a965a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.160599 14809 raft_consensus.cc:738] T 79579515fb7e45a8b16fd36dbc6a965a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.160705 14809 consensus_queue.cc:260] T 79579515fb7e45a8b16fd36dbc6a965a P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.161000 14809 ts_tablet_manager.cc:1428] T 79579515fb7e45a8b16fd36dbc6a965a P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.161051 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "872ad705e3e94c0eab4397df9f28f9ca" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.161157 14809 tablet_bootstrap.cc:492] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
W20250115 03:06:16.161401 14626 leader_election.cc:343] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.161480 14626 leader_election.cc:304] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.161722 15153 raft_consensus.cc:2743] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.161984 14809 tablet_bootstrap.cc:654] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.164539 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet 17b5e362481c49d2a3a13b1550ceef23 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:16.164606 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f7b803cd6e0546c394296cb6b4b2dc83" } modify_external_catalogs: true
I20250115 03:06:16.164685 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f7b803cd6e0546c394296cb6b4b2dc83" } modify_external_catalogs: true
I20250115 03:06:16.167104 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 304fab45b38e4426ae6961ae786e5659 (table default.Table_d407111658854f02aa8387a067f33f37 [id=526370a40a0d4f7aa96c8348412d8b26]). Expected version 1 got 0
I20250115 03:06:16.167968 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet 84ce55f729224f78b420e57977419add with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.168246 14660 tablet_service.cc:1467] Processing CreateTablet for tablet fbd161e290914d9d8671c10becfeaee1 (DEFAULT_TABLE table=default.Table_be6e30c629974876a5e7b6b25c4959dc [id=151f92275af94ad4b585a488274c12ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.168501 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbd161e290914d9d8671c10becfeaee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.168738 14535 tablet_service.cc:1467] Processing CreateTablet for tablet fbd161e290914d9d8671c10becfeaee1 (DEFAULT_TABLE table=default.Table_be6e30c629974876a5e7b6b25c4959dc [id=151f92275af94ad4b585a488274c12ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.168972 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbd161e290914d9d8671c10becfeaee1. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:16.169756 15645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250115 03:06:16.169973 15645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250115 03:06:16.170028 15645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20250115 03:06:16.170634 14823 tablet_replica.cc:331] T 84ce55f729224f78b420e57977419add P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.170758 14823 raft_consensus.cc:2238] T 84ce55f729224f78b420e57977419add P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.170845 14823 raft_consensus.cc:2267] T 84ce55f729224f78b420e57977419add P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.171504 14399 tablet_service.cc:1467] Processing CreateTablet for tablet fbd161e290914d9d8671c10becfeaee1 (DEFAULT_TABLE table=default.Table_be6e30c629974876a5e7b6b25c4959dc [id=151f92275af94ad4b585a488274c12ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.171659 15274 tablet.cc:1721] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:16.171734 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbd161e290914d9d8671c10becfeaee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.171725 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 82be772e05d443749914004aea8af300 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:16.174257 15645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250115 03:06:16.174320 15645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250115 03:06:16.174350 15645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250115 03:06:16.174377 15645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250115 03:06:16.175176 15426 tablet.cc:1721] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:16.176398 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 4102586f32d94a998cf2807434ee0ebf (DEFAULT_TABLE table=default.Table_a1c9e28bac9c491e948bc3fc9b173bad [id=2fb8c68c76304deebd009b01eb078616]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.176635 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4102586f32d94a998cf2807434ee0ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.176885 14528 tablet_service.cc:1467] Processing CreateTablet for tablet d2564a20857f45d9a0c7e3b74eb58a91 (DEFAULT_TABLE table=default.Table_b2f2461378b548f785fdc3595569c0db [id=6a9388b294f24f358a5d5f34568e8a13]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.177120 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2564a20857f45d9a0c7e3b74eb58a91. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.179036 15645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.186.212:37407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.13.186.254:39901,127.13.186.253:35411,127.13.186.252:41855
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.186.252:41855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.186.252
--webserver_port=39163
--never_fsync=true
--heap_profile_path=/tmp/kudu.15645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 75b75ca2f58f1a5aba69207830e3558011033d99
build type DEBUG
built by None at 15 Jan 2025 02:43:26 UTC on 24a791456cd2
build id 4181
I20250115 03:06:16.179132 14396 tablet_service.cc:1467] Processing CreateTablet for tablet d2564a20857f45d9a0c7e3b74eb58a91 (DEFAULT_TABLE table=default.Table_b2f2461378b548f785fdc3595569c0db [id=6a9388b294f24f358a5d5f34568e8a13]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.179338 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2564a20857f45d9a0c7e3b74eb58a91. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.179435 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 21524aef80f74a6c899ea66b9b022389 (DEFAULT_TABLE table=default.Table_a0c249d69b904390952a780b10e1369b [id=22d05297fb9545aeab39b5b62248d0d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.179661 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21524aef80f74a6c899ea66b9b022389. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.180183 15645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250115 03:06:16.180243 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 22cc0833bfde458ebe3985c05d29b8e6 (table default.Table_3db38338e5b04298954aded6d5482169 [id=f56a0de6e56441bfa620902f175feb4a]). Expected version 2 got 0
I20250115 03:06:16.181124 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 21524aef80f74a6c899ea66b9b022389 (DEFAULT_TABLE table=default.Table_a0c249d69b904390952a780b10e1369b [id=22d05297fb9545aeab39b5b62248d0d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.181357 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21524aef80f74a6c899ea66b9b022389. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.181468 14823 ts_tablet_manager.cc:1905] T 84ce55f729224f78b420e57977419add P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.182482 14660 tablet_service.cc:1467] Processing CreateTablet for tablet d2564a20857f45d9a0c7e3b74eb58a91 (DEFAULT_TABLE table=default.Table_b2f2461378b548f785fdc3595569c0db [id=6a9388b294f24f358a5d5f34568e8a13]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.182693 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d2564a20857f45d9a0c7e3b74eb58a91. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.183271 15645 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250115 03:06:16.183315 14823 ts_tablet_manager.cc:1918] T 84ce55f729224f78b420e57977419add P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.183374 14823 log.cc:1198] T 84ce55f729224f78b420e57977419add P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/84ce55f729224f78b420e57977419add
I20250115 03:06:16.183583 14823 ts_tablet_manager.cc:1939] T 84ce55f729224f78b420e57977419add P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.183748 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 4102586f32d94a998cf2807434ee0ebf (DEFAULT_TABLE table=default.Table_a1c9e28bac9c491e948bc3fc9b173bad [id=2fb8c68c76304deebd009b01eb078616]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.183934 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4102586f32d94a998cf2807434ee0ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.184072 14823 tablet_replica.cc:331] T 82be772e05d443749914004aea8af300 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.184170 14823 raft_consensus.cc:2238] T 82be772e05d443749914004aea8af300 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.184248 14823 raft_consensus.cc:2267] T 82be772e05d443749914004aea8af300 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.184914 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 84ce55f729224f78b420e57977419add (table default.Table_ff21d7bc463d43cfa3330f7288ba7a55 [id=5de066e9f2174fff9303cba23eeec988]) successfully deleted
I20250115 03:06:16.189152 14823 ts_tablet_manager.cc:1905] T 82be772e05d443749914004aea8af300 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.189391 15453 raft_consensus.cc:491] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.189471 15453 raft_consensus.cc:513] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.189766 15453 leader_election.cc:290] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:16.190122 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c3ff892f9b4dc484ffc80d7c5934c4" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:16.190433 14628 leader_election.cc:343] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.190531 15401 raft_consensus.cc:491] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.190609 15401 raft_consensus.cc:513] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.190873 14528 tablet_service.cc:1467] Processing CreateTablet for tablet 21524aef80f74a6c899ea66b9b022389 (DEFAULT_TABLE table=default.Table_a0c249d69b904390952a780b10e1369b [id=22d05297fb9545aeab39b5b62248d0d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.190954 15401 leader_election.cc:290] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.190891 14823 ts_tablet_manager.cc:1918] T 82be772e05d443749914004aea8af300 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.191076 14823 log.cc:1198] T 82be772e05d443749914004aea8af300 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/82be772e05d443749914004aea8af300
I20250115 03:06:16.191083 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21524aef80f74a6c899ea66b9b022389. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.191251 14823 ts_tablet_manager.cc:1939] T 82be772e05d443749914004aea8af300 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
W20250115 03:06:16.191519 14358 leader_election.cc:343] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: db1aa206e9994de6b9a355a621cb2ab3
I20250115 03:06:16.192804 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db1aa206e9994de6b9a355a621cb2ab3" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:16.193176 14356 leader_election.cc:343] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.193243 14356 leader_election.cc:304] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:16.193270 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 82be772e05d443749914004aea8af300 (table default.Table_3863dfaf57694ee7b4a9ba1e400d7c9d [id=831e8e5e33ae41368950b0b8fce37e9b]) successfully deleted
I20250115 03:06:16.193400 15401 raft_consensus.cc:2743] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:16.193238 15696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 03:06:16.193713 15698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 03:06:16.193922 15695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250115 03:06:16.195194 14812 tablet_bootstrap.cc:492] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.195207 15617 tablet.cc:1721] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:16.195318 14812 ts_tablet_manager.cc:1397] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.049s user 0.001s sys 0.001s
I20250115 03:06:16.195863 14812 raft_consensus.cc:357] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.196019 14812 raft_consensus.cc:383] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.196094 14812 raft_consensus.cc:738] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.196266 14812 consensus_queue.cc:260] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.196718 14812 ts_tablet_manager.cc:1428] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.196916 14812 tablet_bootstrap.cc:492] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.197808 14812 tablet_bootstrap.cc:654] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.199033 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c3ff892f9b4dc484ffc80d7c5934c4" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.199363 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0aecd3e601c04f0497cefafda025e6e1" } modify_external_catalogs: true
I20250115 03:06:16.199452 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0aecd3e601c04f0497cefafda025e6e1" } modify_external_catalogs: true
I20250115 03:06:16.200181 14809 tablet_bootstrap.cc:492] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.200255 14809 ts_tablet_manager.cc:1397] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.039s user 0.000s sys 0.001s
I20250115 03:06:16.200462 14811 tablet_bootstrap.cc:492] T b1644ba8616349dc9d10a790b6089571 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.200541 14811 ts_tablet_manager.cc:1397] T b1644ba8616349dc9d10a790b6089571 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.077s user 0.001s sys 0.000s
I20250115 03:06:16.200646 14809 raft_consensus.cc:357] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.200727 14809 raft_consensus.cc:383] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.200765 14809 raft_consensus.cc:738] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.200871 14809 consensus_queue.cc:260] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.201148 14809 ts_tablet_manager.cc:1428] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.201301 14809 tablet_bootstrap.cc:492] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.201663 14812 tablet_bootstrap.cc:492] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.201632 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 4102586f32d94a998cf2807434ee0ebf (DEFAULT_TABLE table=default.Table_a1c9e28bac9c491e948bc3fc9b173bad [id=2fb8c68c76304deebd009b01eb078616]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.201738 14812 ts_tablet_manager.cc:1397] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20250115 03:06:16.201836 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4102586f32d94a998cf2807434ee0ebf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.202090 14812 raft_consensus.cc:357] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.202143 14809 tablet_bootstrap.cc:654] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.202164 14812 raft_consensus.cc:383] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.202191 14812 raft_consensus.cc:738] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.202319 14812 consensus_queue.cc:260] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.202564 14812 ts_tablet_manager.cc:1428] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.202695 14812 tablet_bootstrap.cc:492] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.203037 15401 raft_consensus.cc:491] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.203104 15401 raft_consensus.cc:513] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.203403 15401 leader_election.cc:290] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.203821 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3ba1b2ff964d648e5f36f410107b5b" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:16.203879 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3ba1b2ff964d648e5f36f410107b5b" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.203929 14691 raft_consensus.cc:2463] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:16.204173 14358 leader_election.cc:304] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
W20250115 03:06:16.204249 14356 leader_election.cc:343] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.204442 15401 raft_consensus.cc:2798] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:16.204495 15401 raft_consensus.cc:491] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:16.204530 15401 raft_consensus.cc:3054] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.204602 14812 tablet_bootstrap.cc:654] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:16.204874 14626 leader_election.cc:343] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.205431 15401 raft_consensus.cc:513] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.205719 15401 leader_election.cc:290] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.205773 14626 leader_election.cc:304] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.205857 15645 server_base.cc:1034] running on GCE node
I20250115 03:06:16.206205 15153 raft_consensus.cc:2743] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.206358 15645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250115 03:06:16.206354 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3ba1b2ff964d648e5f36f410107b5b" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:16.206441 14691 raft_consensus.cc:3054] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.207135 14811 raft_consensus.cc:357] T b1644ba8616349dc9d10a790b6089571 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.207286 14691 raft_consensus.cc:2463] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:16.207350 15645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250115 03:06:16.207669 15153 consensus_queue.cc:1035] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.207943 14358 leader_election.cc:304] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.208079 14885 raft_consensus.cc:491] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.208138 14885 raft_consensus.cc:513] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.208374 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet f70a69acbdc049beb9d98eada09d9eb8 (table default.Table_db8a2673977247e0b14b7995f7e09661 [id=f7e4534c1f6f48f09e10b84c1d5c458c]). Expected version 1 got 0
I20250115 03:06:16.208402 14885 leader_election.cc:290] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.208479 14885 raft_consensus.cc:2798] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:16.208537 14885 raft_consensus.cc:695] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:16.208639 14885 consensus_queue.cc:237] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:16.208714 14163 catalog_manager.cc:5718] Failed to send AlterTable RPC for tablet f70a69acbdc049beb9d98eada09d9eb8 (table default.Table_db8a2673977247e0b14b7995f7e09661 [id=f7e4534c1f6f48f09e10b84c1d5c458c], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet f70a69acbdc049beb9d98eada09d9eb8 consensus state has no leader
I20250115 03:06:16.209704 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa08518530ae4d5487af386a650c4672" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:16.209820 14691 raft_consensus.cc:2463] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:16.210026 14358 leader_election.cc:304] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.210171 15401 raft_consensus.cc:2798] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:16.210223 15401 raft_consensus.cc:491] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:16.210268 15401 raft_consensus.cc:3054] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.211047 15401 raft_consensus.cc:513] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.210934 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa08518530ae4d5487af386a650c4672" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.211369 15401 leader_election.cc:290] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.211577 14811 raft_consensus.cc:383] T b1644ba8616349dc9d10a790b6089571 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.211663 14811 raft_consensus.cc:738] T b1644ba8616349dc9d10a790b6089571 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.211699 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa08518530ae4d5487af386a650c4672" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:16.211802 14691 raft_consensus.cc:3054] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.211858 14811 consensus_queue.cc:260] T b1644ba8616349dc9d10a790b6089571 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.212550 14691 raft_consensus.cc:2463] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:16.212872 14358 leader_election.cc:304] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.211046 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3ba1b2ff964d648e5f36f410107b5b" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:16.213093 15401 raft_consensus.cc:2798] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:16.213168 15401 raft_consensus.cc:695] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:16.213272 15401 consensus_queue.cc:237] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.213434 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa08518530ae4d5487af386a650c4672" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:16.213681 14356 leader_election.cc:343] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:16.213835 14356 leader_election.cc:343] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.214406 14812 tablet_bootstrap.cc:492] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.214491 14812 ts_tablet_manager.cc:1397] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20250115 03:06:16.214891 14812 raft_consensus.cc:357] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.215065 14812 raft_consensus.cc:383] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.215117 14812 raft_consensus.cc:738] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.215250 14812 consensus_queue.cc:260] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:16.215467 14356 leader_election.cc:343] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.215575 14812 ts_tablet_manager.cc:1428] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.215725 14812 tablet_bootstrap.cc:492] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.216638 14812 tablet_bootstrap.cc:654] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.217757 14812 tablet_bootstrap.cc:492] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.217852 14812 ts_tablet_manager.cc:1397] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20250115 03:06:16.219074 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 6864f88bf6454ee980f624ec2f2d9fb6 (table default.Table_2e68c93eaa1647df9b269879bd4520ab [id=18e0f06feeda4bca860e6f1bc6eab15b]). Expected version 1 got 0
I20250115 03:06:16.219666 14812 raft_consensus.cc:357] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.219790 14812 raft_consensus.cc:383] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.219851 14812 raft_consensus.cc:738] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.220011 14812 consensus_queue.cc:260] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.220278 14812 ts_tablet_manager.cc:1428] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:16.220438 14812 tablet_bootstrap.cc:492] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.221385 14812 tablet_bootstrap.cc:654] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.222431 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 30499e99ab7d4138a376dea40b768c32
I20250115 03:06:16.222713 14166 catalog_manager.cc:6596] ReplaceTablet: tablet 30499e99ab7d4138a376dea40b768c32 deleted and replaced by tablet 80ede4e9696d491aa385b375fcb5cf3b
W20250115 03:06:16.222765 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 312dcdda8cf042648db9fdf6228062d9 consensus state has no leader
I20250115 03:06:16.222957 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 17b5e362481c49d2a3a13b1550ceef23
I20250115 03:06:16.223176 14171 catalog_manager.cc:6596] ReplaceTablet: tablet 17b5e362481c49d2a3a13b1550ceef23 deleted and replaced by tablet 1016cf85491d4ceda132ee57325b48a4
I20250115 03:06:16.223371 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 0f3ba1b2ff964d648e5f36f410107b5b
I20250115 03:06:16.223703 15645 hybrid_clock.cc:648] HybridClock initialized: now 1736910376223637 us; error 81 us; skew 500 ppm
I20250115 03:06:16.224591 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 30499e99ab7d4138a376dea40b768c32 with delete_type TABLET_DATA_DELETED (replaced by tablet 80ede4e9696d491aa385b375fcb5cf3b) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.224733 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet 17b5e362481c49d2a3a13b1550ceef23 with delete_type TABLET_DATA_DELETED (replaced by tablet 1016cf85491d4ceda132ee57325b48a4) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.225243 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3ba1b2ff964d648e5f36f410107b5b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.225334 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet f3ac7f1e1bfb4383895dc2c2123a743b
I20250115 03:06:16.225991 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet f3ac7f1e1bfb4383895dc2c2123a743b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.226428 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 30499e99ab7d4138a376dea40b768c32 with delete_type TABLET_DATA_DELETED (replaced by tablet 80ede4e9696d491aa385b375fcb5cf3b) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.226553 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 9808c7f96a3147848da5f78948895699 with delete_type TABLET_DATA_DELETED (replaced by tablet 341b24d9b63f4761b2013035023fcd98) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.226784 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet f3ac7f1e1bfb4383895dc2c2123a743b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.226891 14819 tablet_replica.cc:331] T f3ac7f1e1bfb4383895dc2c2123a743b P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.226991 14819 raft_consensus.cc:2238] T f3ac7f1e1bfb4383895dc2c2123a743b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.227061 14819 raft_consensus.cc:2267] T f3ac7f1e1bfb4383895dc2c2123a743b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.227159 14823 tablet_replica.cc:331] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.227273 14823 raft_consensus.cc:2238] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:16.227368 14819 ts_tablet_manager.cc:1905] T f3ac7f1e1bfb4383895dc2c2123a743b P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.227404 14823 pending_rounds.cc:62] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 1 pending ops.
I20250115 03:06:16.227435 14823 pending_rounds.cc:69] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384899736195072 op_type: NO_OP noop_request { }
I20250115 03:06:16.227483 14823 raft_consensus.cc:2883] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:16.227530 14823 raft_consensus.cc:2267] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.227741 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 17b5e362481c49d2a3a13b1550ceef23 with delete_type TABLET_DATA_DELETED (replaced by tablet 1016cf85491d4ceda132ee57325b48a4) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.227855 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3ba1b2ff964d648e5f36f410107b5b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.229194 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 17b5e362481c49d2a3a13b1550ceef23 with delete_type TABLET_DATA_DELETED (replaced by tablet 1016cf85491d4ceda132ee57325b48a4) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.229346 14819 ts_tablet_manager.cc:1918] T f3ac7f1e1bfb4383895dc2c2123a743b P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.229363 14529 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3ba1b2ff964d648e5f36f410107b5b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.229405 14819 log.cc:1198] T f3ac7f1e1bfb4383895dc2c2123a743b P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/f3ac7f1e1bfb4383895dc2c2123a743b
I20250115 03:06:16.229369 14820 tablet_replica.cc:331] T 17b5e362481c49d2a3a13b1550ceef23 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.229532 14820 raft_consensus.cc:2238] T 17b5e362481c49d2a3a13b1550ceef23 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.229593 14819 ts_tablet_manager.cc:1939] T f3ac7f1e1bfb4383895dc2c2123a743b P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.229633 14820 raft_consensus.cc:2267] T 17b5e362481c49d2a3a13b1550ceef23 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.229815 14823 ts_tablet_manager.cc:1905] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.230053 14819 tablet_replica.cc:331] T 17b5e362481c49d2a3a13b1550ceef23 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.230151 14820 ts_tablet_manager.cc:1905] T 17b5e362481c49d2a3a13b1550ceef23 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.230166 14819 raft_consensus.cc:2238] T 17b5e362481c49d2a3a13b1550ceef23 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.230239 14819 raft_consensus.cc:2267] T 17b5e362481c49d2a3a13b1550ceef23 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.230532 14819 ts_tablet_manager.cc:1905] T 17b5e362481c49d2a3a13b1550ceef23 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.231242 15325 consensus_queue.cc:1035] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.231849 14823 ts_tablet_manager.cc:1918] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:16.231904 14823 log.cc:1198] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/17b5e362481c49d2a3a13b1550ceef23
I20250115 03:06:16.232120 14823 ts_tablet_manager.cc:1939] T 17b5e362481c49d2a3a13b1550ceef23 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.232142 14819 ts_tablet_manager.cc:1918] T 17b5e362481c49d2a3a13b1550ceef23 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.232201 14819 log.cc:1198] T 17b5e362481c49d2a3a13b1550ceef23 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/17b5e362481c49d2a3a13b1550ceef23
I20250115 03:06:16.232272 14820 ts_tablet_manager.cc:1918] T 17b5e362481c49d2a3a13b1550ceef23 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.232324 14820 log.cc:1198] T 17b5e362481c49d2a3a13b1550ceef23 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/17b5e362481c49d2a3a13b1550ceef23
I20250115 03:06:16.232372 14819 ts_tablet_manager.cc:1939] T 17b5e362481c49d2a3a13b1550ceef23 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.232508 14820 ts_tablet_manager.cc:1939] T 17b5e362481c49d2a3a13b1550ceef23 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.232599 14823 tablet_replica.cc:331] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.232707 14823 raft_consensus.cc:2238] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.232838 14819 tablet_replica.cc:331] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.233003 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet f3ac7f1e1bfb4383895dc2c2123a743b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.233266 14820 tablet_replica.cc:331] T f3ac7f1e1bfb4383895dc2c2123a743b P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.233407 14820 raft_consensus.cc:2238] T f3ac7f1e1bfb4383895dc2c2123a743b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.233537 14820 raft_consensus.cc:2267] T f3ac7f1e1bfb4383895dc2c2123a743b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.233978 14820 ts_tablet_manager.cc:1905] T f3ac7f1e1bfb4383895dc2c2123a743b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.234212 15453 raft_consensus.cc:491] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.234309 15453 raft_consensus.cc:513] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.234421 15645 webserver.cc:458] Webserver started at http://127.13.186.252:39163/ using document root <none> and password file <none>
I20250115 03:06:16.234521 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 17b5e362481c49d2a3a13b1550ceef23 (table default.Table_2e115074bf7049bfbb1239b4d9c1ca2e [id=6cf622de00cb490bb794837223aa7bcb]) successfully deleted
I20250115 03:06:16.234632 15453 leader_election.cc:290] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
W20250115 03:06:16.234685 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 0f3ba1b2ff964d648e5f36f410107b5b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0f3ba1b2ff964d648e5f36f410107b5b already in progress: creating tablet
W20250115 03:06:16.234987 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 30499e99ab7d4138a376dea40b768c32 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 30499e99ab7d4138a376dea40b768c32 already in progress: creating tablet
I20250115 03:06:16.235126 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 17b5e362481c49d2a3a13b1550ceef23 (table default.Table_2e115074bf7049bfbb1239b4d9c1ca2e [id=6cf622de00cb490bb794837223aa7bcb]) successfully deleted
I20250115 03:06:16.235147 15645 fs_manager.cc:362] Metadata directory not provided
I20250115 03:06:16.235222 15645 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250115 03:06:16.236138 14820 ts_tablet_manager.cc:1918] T f3ac7f1e1bfb4383895dc2c2123a743b P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.236255 14820 log.cc:1198] T f3ac7f1e1bfb4383895dc2c2123a743b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/f3ac7f1e1bfb4383895dc2c2123a743b
I20250115 03:06:16.232805 14823 raft_consensus.cc:2267] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.236501 14820 ts_tablet_manager.cc:1939] T f3ac7f1e1bfb4383895dc2c2123a743b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.236543 15390 raft_consensus.cc:491] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.236608 15390 raft_consensus.cc:513] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.236943 15390 leader_election.cc:290] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.237010 14823 ts_tablet_manager.cc:1905] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.237471 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff51d514fb84809aa53fa9fc1010681" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:16.237571 14691 raft_consensus.cc:2463] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:16.237820 14358 leader_election.cc:304] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.237998 15390 raft_consensus.cc:2798] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:16.238046 15390 raft_consensus.cc:491] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:16.238082 15390 raft_consensus.cc:3054] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.238727 14823 ts_tablet_manager.cc:1918] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.238786 14823 log.cc:1198] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/0f3ba1b2ff964d648e5f36f410107b5b
I20250115 03:06:16.238876 15390 raft_consensus.cc:513] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.238951 14823 ts_tablet_manager.cc:1939] T 0f3ba1b2ff964d648e5f36f410107b5b P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.239202 15390 leader_election.cc:290] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.239444 14823 tablet_replica.cc:331] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.239508 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 0f3ba1b2ff964d648e5f36f410107b5b (table default.Table_e83118015644482ca56543e76b9691bd [id=089b3b2c658a4702869cf70ddd9baf73]) successfully deleted
I20250115 03:06:16.239547 14823 raft_consensus.cc:2238] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:16.239665 14823 pending_rounds.cc:62] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 1 pending ops.
I20250115 03:06:16.239693 14823 pending_rounds.cc:69] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384898772205568 op_type: NO_OP noop_request { }
I20250115 03:06:16.239756 14823 raft_consensus.cc:2883] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:16.239802 14823 raft_consensus.cc:2267] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.239789 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff51d514fb84809aa53fa9fc1010681" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:16.239866 14691 raft_consensus.cc:3054] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.237306 14811 ts_tablet_manager.cc:1428] T b1644ba8616349dc9d10a790b6089571 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.037s user 0.001s sys 0.000s
I20250115 03:06:16.240063 14811 tablet_bootstrap.cc:492] T e594ea5cff81475886cdceb8400645a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.240635 14691 raft_consensus.cc:2463] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:16.240901 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cbc91ad00b54ede804c3629bcfe7fe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.241014 14811 tablet_bootstrap.cc:654] T e594ea5cff81475886cdceb8400645a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.241153 14358 leader_election.cc:304] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.241909 14809 tablet_bootstrap.cc:492] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.241983 14809 ts_tablet_manager.cc:1397] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.041s user 0.001s sys 0.000s
W20250115 03:06:16.242020 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 30499e99ab7d4138a376dea40b768c32 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 30499e99ab7d4138a376dea40b768c32 already in progress: creating tablet
I20250115 03:06:16.242143 14823 ts_tablet_manager.cc:1905] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:16.242166 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 9808c7f96a3147848da5f78948895699 because the tablet was not found. No further retry: Not found: Tablet not found: 9808c7f96a3147848da5f78948895699
I20250115 03:06:16.242257 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet f3ac7f1e1bfb4383895dc2c2123a743b (table default.Table_f7b803cd6e0546c394296cb6b4b2dc83 [id=d4a8db3fb58b46cda80c8e652dbd33b8]) successfully deleted
I20250115 03:06:16.242360 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 17b5e362481c49d2a3a13b1550ceef23 (table default.Table_2e115074bf7049bfbb1239b4d9c1ca2e [id=6cf622de00cb490bb794837223aa7bcb]) successfully deleted
I20250115 03:06:16.242417 14809 raft_consensus.cc:357] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.242506 14809 raft_consensus.cc:383] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.242535 14809 raft_consensus.cc:738] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.238193 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff51d514fb84809aa53fa9fc1010681" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.242628 14809 consensus_queue.cc:260] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.239446 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff51d514fb84809aa53fa9fc1010681" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:16.239632 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet b1644ba8616349dc9d10a790b6089571 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.239706 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet 30499e99ab7d4138a376dea40b768c32 with delete_type TABLET_DATA_DELETED (replaced by tablet 80ede4e9696d491aa385b375fcb5cf3b) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.242887 14809 ts_tablet_manager.cc:1428] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.242916 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4f688e4c40d4653929432b13508348a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:16.243026 14809 tablet_bootstrap.cc:492] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
W20250115 03:06:16.243197 14628 leader_election.cc:343] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.243377 15390 raft_consensus.cc:2798] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:16.243466 15390 raft_consensus.cc:695] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:16.243569 15390 consensus_queue.cc:237] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.243827 14823 ts_tablet_manager.cc:1918] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:16.243883 14823 log.cc:1198] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/f3ac7f1e1bfb4383895dc2c2123a743b
I20250115 03:06:16.243887 14809 tablet_bootstrap.cc:654] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.244091 14823 ts_tablet_manager.cc:1939] T f3ac7f1e1bfb4383895dc2c2123a743b P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.244616 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet f3ac7f1e1bfb4383895dc2c2123a743b (table default.Table_f7b803cd6e0546c394296cb6b4b2dc83 [id=d4a8db3fb58b46cda80c8e652dbd33b8]) successfully deleted
I20250115 03:06:16.244632 15453 raft_consensus.cc:491] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.244701 15453 raft_consensus.cc:513] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.244791 14820 tablet_replica.cc:331] T b1644ba8616349dc9d10a790b6089571 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.244899 14820 raft_consensus.cc:2238] T b1644ba8616349dc9d10a790b6089571 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.244997 14820 raft_consensus.cc:2267] T b1644ba8616349dc9d10a790b6089571 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.245028 15453 leader_election.cc:290] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
W20250115 03:06:16.245363 14356 leader_election.cc:343] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:16.245527 14356 leader_election.cc:343] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.246687 14820 ts_tablet_manager.cc:1905] T b1644ba8616349dc9d10a790b6089571 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.247287 14885 raft_consensus.cc:491] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.247362 14885 raft_consensus.cc:513] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.247682 14885 leader_election.cc:290] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.248059 14819 raft_consensus.cc:2238] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:16.248121 14809 tablet_bootstrap.cc:492] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.248188 14809 ts_tablet_manager.cc:1397] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:16.248521 14820 ts_tablet_manager.cc:1918] T b1644ba8616349dc9d10a790b6089571 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.248575 14820 log.cc:1198] T b1644ba8616349dc9d10a790b6089571 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/b1644ba8616349dc9d10a790b6089571
I20250115 03:06:16.248582 14809 raft_consensus.cc:357] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.248668 14809 raft_consensus.cc:383] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.248699 14809 raft_consensus.cc:738] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.248719 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f67cbb929cd4ae589a14606aa168b46" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.248754 14820 ts_tablet_manager.cc:1939] T b1644ba8616349dc9d10a790b6089571 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.248808 14809 consensus_queue.cc:260] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.248944 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cbc91ad00b54ede804c3629bcfe7fe6" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:16.249082 14809 ts_tablet_manager.cc:1428] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.249243 14809 tablet_bootstrap.cc:492] T 5b7022db9edc4de8909e16769a3ea136 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.249356 14823 tablet_replica.cc:331] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.249440 14823 raft_consensus.cc:2238] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.249498 14823 raft_consensus.cc:2267] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.249593 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet ca2093980c8e480e99ae0cd79606f318
I20250115 03:06:16.249894 14823 ts_tablet_manager.cc:1905] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.250118 14809 tablet_bootstrap.cc:654] T 5b7022db9edc4de8909e16769a3ea136 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.250110 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet ca2093980c8e480e99ae0cd79606f318 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:16.250609 14628 leader_election.cc:343] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.250900 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet ca2093980c8e480e99ae0cd79606f318 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.251067 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet f3ac7f1e1bfb4383895dc2c2123a743b (table default.Table_f7b803cd6e0546c394296cb6b4b2dc83 [id=d4a8db3fb58b46cda80c8e652dbd33b8]) successfully deleted
I20250115 03:06:16.251632 14823 ts_tablet_manager.cc:1918] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.251693 14823 log.cc:1198] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/3cbc91ad00b54ede804c3629bcfe7fe6
W20250115 03:06:16.251674 14356 leader_election.cc:343] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.251842 14823 ts_tablet_manager.cc:1939] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.252138 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet b1644ba8616349dc9d10a790b6089571 (table default.Table_632fc9d76fb3482c8a5e87feda0ceeed [id=dd1bb6f7163f46748711bc33cc8a3f85]) successfully deleted
W20250115 03:06:16.252140 14358 leader_election.cc:343] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 7f67cbb929cd4ae589a14606aa168b46
I20250115 03:06:16.252216 14358 leader_election.cc:304] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:16.252213 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 3cbc91ad00b54ede804c3629bcfe7fe6 (table default.Table_271f46df55f14ffd9b7bb48352620543 [id=44488be7335b488a9ff7cf913ce4273a]) successfully deleted
W20250115 03:06:16.252285 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 30499e99ab7d4138a376dea40b768c32 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 30499e99ab7d4138a376dea40b768c32 already in progress: creating tablet
I20250115 03:06:16.252352 14885 raft_consensus.cc:2743] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:16.252465 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet ca2093980c8e480e99ae0cd79606f318 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ca2093980c8e480e99ae0cd79606f318 already in progress: creating tablet
I20250115 03:06:16.248190 14819 pending_rounds.cc:62] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75: Trying to abort 1 pending ops.
I20250115 03:06:16.252759 14819 pending_rounds.cc:69] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384900951621632 op_type: NO_OP noop_request { }
I20250115 03:06:16.252810 14819 raft_consensus.cc:2883] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:16.252854 14819 raft_consensus.cc:2267] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.253060 14811 tablet_bootstrap.cc:492] T e594ea5cff81475886cdceb8400645a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.253129 14811 ts_tablet_manager.cc:1397] T e594ea5cff81475886cdceb8400645a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.001s
I20250115 03:06:16.253476 14811 raft_consensus.cc:357] T e594ea5cff81475886cdceb8400645a6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.253556 14811 raft_consensus.cc:383] T e594ea5cff81475886cdceb8400645a6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.253578 14811 raft_consensus.cc:738] T e594ea5cff81475886cdceb8400645a6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.253685 14811 consensus_queue.cc:260] T e594ea5cff81475886cdceb8400645a6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.253892 14811 ts_tablet_manager.cc:1428] T e594ea5cff81475886cdceb8400645a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:16.253991 14819 ts_tablet_manager.cc:1905] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.254020 14811 tablet_bootstrap.cc:492] T a4484651d964411e90ed93c6caab0a13 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.254897 14811 tablet_bootstrap.cc:654] T a4484651d964411e90ed93c6caab0a13 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.255203 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet ca2093980c8e480e99ae0cd79606f318 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.255831 14819 ts_tablet_manager.cc:1918] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:16.255877 14811 tablet_bootstrap.cc:492] T a4484651d964411e90ed93c6caab0a13 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.255944 14811 ts_tablet_manager.cc:1397] T a4484651d964411e90ed93c6caab0a13 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:16.256314 14811 raft_consensus.cc:357] T a4484651d964411e90ed93c6caab0a13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.256390 14811 raft_consensus.cc:383] T a4484651d964411e90ed93c6caab0a13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.256414 14811 raft_consensus.cc:738] T a4484651d964411e90ed93c6caab0a13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.256510 14811 consensus_queue.cc:260] T a4484651d964411e90ed93c6caab0a13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.256731 14811 ts_tablet_manager.cc:1428] T a4484651d964411e90ed93c6caab0a13 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:16.256568 14819 log.cc:1198] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/0f3ba1b2ff964d648e5f36f410107b5b
I20250115 03:06:16.256868 14811 tablet_bootstrap.cc:492] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.257066 14819 ts_tablet_manager.cc:1939] T 0f3ba1b2ff964d648e5f36f410107b5b P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.257723 14811 tablet_bootstrap.cc:654] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.257762 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 0f3ba1b2ff964d648e5f36f410107b5b (table default.Table_e83118015644482ca56543e76b9691bd [id=089b3b2c658a4702869cf70ddd9baf73]) successfully deleted
W20250115 03:06:16.257905 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet ca2093980c8e480e99ae0cd79606f318 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ca2093980c8e480e99ae0cd79606f318 already in progress: creating tablet
I20250115 03:06:16.258462 14812 tablet_bootstrap.cc:492] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.258548 14812 ts_tablet_manager.cc:1397] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.038s user 0.001s sys 0.000s
I20250115 03:06:16.258942 14812 raft_consensus.cc:357] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.259034 14812 raft_consensus.cc:383] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.259063 14812 raft_consensus.cc:738] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.259166 14812 consensus_queue.cc:260] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.259415 14812 ts_tablet_manager.cc:1428] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.259420 14811 tablet_bootstrap.cc:492] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.259500 14811 ts_tablet_manager.cc:1397] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:16.259609 14812 tablet_bootstrap.cc:492] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.259869 14811 raft_consensus.cc:357] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.259953 14811 raft_consensus.cc:383] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.259980 14811 raft_consensus.cc:738] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.260075 14811 consensus_queue.cc:260] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.260306 14811 ts_tablet_manager.cc:1428] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:16.260375 14812 tablet_bootstrap.cc:654] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.260370 14168 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 13ce129d8a9245baab7c772f0f5fbaa7 (table default.Table_e5c55332e306452685635c28fb3d4c55 [id=6cba4fb2f7104c6e953340b04e5dc494]). Expected version 2 got 0
I20250115 03:06:16.260462 14811 tablet_bootstrap.cc:492] T f5af8b81e7ff4acab5aef13003656988 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.260538 14168 catalog_manager.cc:5526] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:16.261348 14811 tablet_bootstrap.cc:654] T f5af8b81e7ff4acab5aef13003656988 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.261694 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_09ddb549f5424be289e1ff6bdefbb152"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.262022 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_09ddb549f5424be289e1ff6bdefbb152 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.263893 14812 tablet_bootstrap.cc:492] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.263970 14812 ts_tablet_manager.cc:1397] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:16.264295 14812 raft_consensus.cc:357] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.264384 14812 raft_consensus.cc:383] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.264408 14812 raft_consensus.cc:738] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.264503 14812 consensus_queue.cc:260] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.264721 14812 ts_tablet_manager.cc:1428] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.264881 14812 tablet_bootstrap.cc:492] T 506198162e0b4e24924c408542707c4c P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.265161 14811 tablet_bootstrap.cc:492] T f5af8b81e7ff4acab5aef13003656988 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.265231 14811 ts_tablet_manager.cc:1397] T f5af8b81e7ff4acab5aef13003656988 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:16.265563 14812 tablet_bootstrap.cc:654] T 506198162e0b4e24924c408542707c4c P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.265538 14811 raft_consensus.cc:357] T f5af8b81e7ff4acab5aef13003656988 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.265605 14811 raft_consensus.cc:383] T f5af8b81e7ff4acab5aef13003656988 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.265631 14811 raft_consensus.cc:738] T f5af8b81e7ff4acab5aef13003656988 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.265769 14811 consensus_queue.cc:260] T f5af8b81e7ff4acab5aef13003656988 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.265992 14811 ts_tablet_manager.cc:1428] T f5af8b81e7ff4acab5aef13003656988 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.266155 14811 tablet_bootstrap.cc:492] T 68974a1815cb482d8b28d54f42634608 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.266687 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4f688e4c40d4653929432b13508348a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.267385 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3cbc91ad00b54ede804c3629bcfe7fe6" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.267544 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_1df66a4877bb4ba993ae442eb173bf82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:16.267903 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 79579515fb7e45a8b16fd36dbc6a965a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.267979 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3ba1b2ff964d648e5f36f410107b5b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.268019 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet db1aa206e9994de6b9a355a621cb2ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:16.268114 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1df66a4877bb4ba993ae442eb173bf82 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.268334 14853 raft_consensus.cc:491] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.268421 14853 raft_consensus.cc:513] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.267948 14400 tablet_service.cc:1467] Processing CreateTablet for tablet c7e9a1d1ada941b68688e0ed6742aa99 (DEFAULT_TABLE table=default.Table_0bc1b60fff5a46129e157cf221391d0d [id=a2689bcea23843c9bb9e1f08e335e58f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.268708 14853 leader_election.cc:290] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.268898 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7e9a1d1ada941b68688e0ed6742aa99. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.269111 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff51d514fb84809aa53fa9fc1010681" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:16.269227 14691 raft_consensus.cc:2388] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in current term 1: Already voted for candidate e8df7247ee9c43e0aa18216da2661f75 in this term.
I20250115 03:06:16.269497 14819 tablet_replica.cc:331] T 79579515fb7e45a8b16fd36dbc6a965a P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.269608 14819 raft_consensus.cc:2238] T 79579515fb7e45a8b16fd36dbc6a965a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.269691 14819 raft_consensus.cc:2267] T 79579515fb7e45a8b16fd36dbc6a965a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.270144 14819 ts_tablet_manager.cc:1905] T 79579515fb7e45a8b16fd36dbc6a965a P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.270170 14811 tablet_bootstrap.cc:654] T 68974a1815cb482d8b28d54f42634608 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.270618 14169 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet a1c93bfd3da44f9f8a8d585f2d21f1be (table default.Table_40b1953a4a364eb1ae58e3387fc65a91 [id=64d20b1792044c5e8add735aaf75a6d9]). Expected version 1 got 0
I20250115 03:06:16.271345 14885 raft_consensus.cc:491] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.271422 14885 raft_consensus.cc:513] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.271674 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff51d514fb84809aa53fa9fc1010681" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:16.271718 14885 leader_election.cc:290] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.271955 14493 leader_election.cc:304] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.272105 14853 raft_consensus.cc:3054] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.272204 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ce129d8a9245baab7c772f0f5fbaa7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:16.272300 14691 raft_consensus.cc:2463] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:16.272537 14358 leader_election.cc:304] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.272733 15453 raft_consensus.cc:491] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.272800 15453 raft_consensus.cc:513] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.273006 14853 raft_consensus.cc:2743] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.273135 15453 leader_election.cc:290] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:16.273190 14812 tablet_bootstrap.cc:492] T 506198162e0b4e24924c408542707c4c P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.273185 15401 raft_consensus.cc:2798] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:16.273231 15401 raft_consensus.cc:491] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:16.273258 14812 ts_tablet_manager.cc:1397] T 506198162e0b4e24924c408542707c4c P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20250115 03:06:16.273267 15401 raft_consensus.cc:3054] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.273644 14812 raft_consensus.cc:357] T 506198162e0b4e24924c408542707c4c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.273734 14812 raft_consensus.cc:383] T 506198162e0b4e24924c408542707c4c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.273772 14812 raft_consensus.cc:738] T 506198162e0b4e24924c408542707c4c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.273871 14812 consensus_queue.cc:260] T 506198162e0b4e24924c408542707c4c P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.273981 14819 ts_tablet_manager.cc:1918] T 79579515fb7e45a8b16fd36dbc6a965a P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.274039 14819 log.cc:1198] T 79579515fb7e45a8b16fd36dbc6a965a P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/79579515fb7e45a8b16fd36dbc6a965a
I20250115 03:06:16.274132 14812 ts_tablet_manager.cc:1428] T 506198162e0b4e24924c408542707c4c P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.274092 15401 raft_consensus.cc:513] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.274216 14819 ts_tablet_manager.cc:1939] T 79579515fb7e45a8b16fd36dbc6a965a P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.274439 15401 leader_election.cc:290] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.274469 14812 tablet_bootstrap.cc:492] T acb43d614111477abb10def48a1dfb3b P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.274983 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ce129d8a9245baab7c772f0f5fbaa7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:16.275074 14691 raft_consensus.cc:3054] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.275911 14691 raft_consensus.cc:2463] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:16.275960 14819 tablet_replica.cc:331] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.276059 14819 raft_consensus.cc:2238] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.276192 14358 leader_election.cc:304] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.276379 15401 raft_consensus.cc:2798] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:16.276458 15401 raft_consensus.cc:695] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:16.276562 15401 consensus_queue.cc:237] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.276605 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 79579515fb7e45a8b16fd36dbc6a965a (table default.Table_b1775fbfc0eb4cf4a59a0f9a853a081e [id=ed57b2cba1db497fa0af4bba35580a35]) successfully deleted
W20250115 03:06:16.276741 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 0f3ba1b2ff964d648e5f36f410107b5b because the tablet was not found. No further retry: Not found: Tablet not found: 0f3ba1b2ff964d648e5f36f410107b5b
I20250115 03:06:16.276909 14812 tablet_bootstrap.cc:654] T acb43d614111477abb10def48a1dfb3b P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.277380 14165 catalog_manager.cc:5526] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:16.277573 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 4ff51d514fb84809aa53fa9fc1010681 (table default.Table_9cc1022af4f14e7a8a8e11cc1803ca87 [id=98f48acd89a6461bb4511141cab4d88b]). Expected version 2 got 0
I20250115 03:06:16.279451 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ce129d8a9245baab7c772f0f5fbaa7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:16.280218 14812 tablet_bootstrap.cc:492] T acb43d614111477abb10def48a1dfb3b P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.280272 14812 ts_tablet_manager.cc:1397] T acb43d614111477abb10def48a1dfb3b P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:16.280619 14812 raft_consensus.cc:357] T acb43d614111477abb10def48a1dfb3b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.280725 14812 raft_consensus.cc:383] T acb43d614111477abb10def48a1dfb3b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.280757 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet 506198162e0b4e24924c408542707c4c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.280773 14812 raft_consensus.cc:738] T acb43d614111477abb10def48a1dfb3b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.280839 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ce129d8a9245baab7c772f0f5fbaa7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.280889 14823 tablet_replica.cc:331] T 506198162e0b4e24924c408542707c4c P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.281002 14823 raft_consensus.cc:2238] T 506198162e0b4e24924c408542707c4c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.281080 14823 raft_consensus.cc:2267] T 506198162e0b4e24924c408542707c4c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.281304 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "304fab45b38e4426ae6961ae786e5659" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:16.281333 14356 leader_election.cc:343] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.281469 14823 ts_tablet_manager.cc:1905] T 506198162e0b4e24924c408542707c4c P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:16.281594 14356 leader_election.cc:343] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:16.281608 14628 leader_election.cc:343] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.276186 14819 raft_consensus.cc:2267] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.283134 14823 ts_tablet_manager.cc:1918] T 506198162e0b4e24924c408542707c4c P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.283200 14823 log.cc:1198] T 506198162e0b4e24924c408542707c4c P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/506198162e0b4e24924c408542707c4c
I20250115 03:06:16.283346 14823 ts_tablet_manager.cc:1939] T 506198162e0b4e24924c408542707c4c P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.283896 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 506198162e0b4e24924c408542707c4c (table default.Table_75c071e03ba44140bce71e920f31715d [id=13ea98de708e409ba7b66e29bb49416d]) successfully deleted
I20250115 03:06:16.283792 14819 ts_tablet_manager.cc:1905] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.284152 14812 consensus_queue.cc:260] T acb43d614111477abb10def48a1dfb3b P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.284394 14812 ts_tablet_manager.cc:1428] T acb43d614111477abb10def48a1dfb3b P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:16.284548 14812 tablet_bootstrap.cc:492] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.285380 14812 tablet_bootstrap.cc:654] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.285802 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ff993b0824114ebdb66d40758eb3b827"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.286177 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff993b0824114ebdb66d40758eb3b827 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.287180 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet acb43d614111477abb10def48a1dfb3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.287317 14823 tablet_replica.cc:331] T acb43d614111477abb10def48a1dfb3b P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.287407 14823 raft_consensus.cc:2238] T acb43d614111477abb10def48a1dfb3b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.287380 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2068156fefae4299b2bada157604f581"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:16.287492 14823 raft_consensus.cc:2267] T acb43d614111477abb10def48a1dfb3b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:16.287670 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2068156fefae4299b2bada157604f581 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.287950 14823 ts_tablet_manager.cc:1905] T acb43d614111477abb10def48a1dfb3b P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.288070 14819 ts_tablet_manager.cc:1918] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.288128 14819 log.cc:1198] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/db1aa206e9994de6b9a355a621cb2ab3
I20250115 03:06:16.288303 14819 ts_tablet_manager.cc:1939] T db1aa206e9994de6b9a355a621cb2ab3 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.288740 14171 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ff9ccf42b3cc41f0b9b3059fab4c6768" } new_table_name: "default.Table_67f9f825d77c4874bb1d4a5c17dedde2" modify_external_catalogs: true
I20250115 03:06:16.288856 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet db1aa206e9994de6b9a355a621cb2ab3 (table default.Table_e050a17d73d342fda8108a99a665f222 [id=eea2f4b389314abbac5f983a79d5cf7e]) successfully deleted
I20250115 03:06:16.289583 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_eea85759fdf64d0bbfabd1e6cb16b125" } new_table_name: "default.Table_70d0b700eac34892b4ef769d739481fd" modify_external_catalogs: true
I20250115 03:06:16.289664 14823 ts_tablet_manager.cc:1918] T acb43d614111477abb10def48a1dfb3b P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.289731 14823 log.cc:1198] T acb43d614111477abb10def48a1dfb3b P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/acb43d614111477abb10def48a1dfb3b
I20250115 03:06:16.289916 14823 ts_tablet_manager.cc:1939] T acb43d614111477abb10def48a1dfb3b P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.289985 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0bc1b60fff5a46129e157cf221391d0d" } new_table_name: "default.Table_169e728924b749e4a9118c12d9b56eab" modify_external_catalogs: true
I20250115 03:06:16.290289 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_eec974cf89de4f45930dbf47fa754f2a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.290566 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eec974cf89de4f45930dbf47fa754f2a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:16.290657 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet ca2093980c8e480e99ae0cd79606f318 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ca2093980c8e480e99ae0cd79606f318 already in progress: creating tablet
W20250115 03:06:16.290761 14626 leader_election.cc:343] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.290820 14626 leader_election.cc:304] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.290959 15453 raft_consensus.cc:2743] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:16.291476 14626 leader_election.cc:343] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.291543 14626 leader_election.cc:304] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
W20250115 03:06:16.291627 14626 raft_consensus.cc:2681] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 5c0dae7aff4b44c699275b95ba639770: unable to run election callback: Service unavailable: Thread pool token was shut down
I20250115 03:06:16.292651 14528 tablet_service.cc:1467] Processing CreateTablet for tablet c7e9a1d1ada941b68688e0ed6742aa99 (DEFAULT_TABLE table=default.Table_0bc1b60fff5a46129e157cf221391d0d [id=a2689bcea23843c9bb9e1f08e335e58f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.292865 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7e9a1d1ada941b68688e0ed6742aa99. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.293435 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet acb43d614111477abb10def48a1dfb3b (table default.Table_85ea53c1d8b245cc94e6e81e157242aa [id=4d8e724e03dc4e2ab9a6e0281159aca8]) successfully deleted
I20250115 03:06:16.295967 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "304fab45b38e4426ae6961ae786e5659" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:16.296953 14626 leader_election.cc:343] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.297020 14626 leader_election.cc:304] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
W20250115 03:06:16.297186 14163 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c7e9a1d1ada941b68688e0ed6742aa99 consensus state has no leader
W20250115 03:06:16.297428 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 13f36a563f13403eab372deb10b67451 consensus state has no leader
W20250115 03:06:16.297812 14171 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 58130f2b27b745d08c0002ac96af6d72 consensus state has no leader
I20250115 03:06:16.298589 15453 raft_consensus.cc:2743] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.300336 14400 tablet_service.cc:1467] Processing CreateTablet for tablet ada45621f8864caab9189039fae0fae2 (DEFAULT_TABLE table=default.Table_f2946c2c8c56478f9108ffeda4034086 [id=2c3fb664638b434397824dec2147518c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.300429 14660 tablet_service.cc:1467] Processing CreateTablet for tablet ada45621f8864caab9189039fae0fae2 (DEFAULT_TABLE table=default.Table_f2946c2c8c56478f9108ffeda4034086 [id=2c3fb664638b434397824dec2147518c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.300561 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ada45621f8864caab9189039fae0fae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.300608 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ada45621f8864caab9189039fae0fae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.300642 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 2c69daff62144976ae9082c89cc58eca (DEFAULT_TABLE table=default.Table_24b2f1f4bbb14d2fa7cba35e089f6707 [id=9c79f5b643664aebbc02a5d815485f61]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.300861 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c69daff62144976ae9082c89cc58eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.302419 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 2c69daff62144976ae9082c89cc58eca (DEFAULT_TABLE table=default.Table_24b2f1f4bbb14d2fa7cba35e089f6707 [id=9c79f5b643664aebbc02a5d815485f61]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.302623 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c69daff62144976ae9082c89cc58eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.303875 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f67cbb929cd4ae589a14606aa168b46 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.304522 14666 tablet_service.cc:1467] Processing CreateTablet for tablet c7e9a1d1ada941b68688e0ed6742aa99 (DEFAULT_TABLE table=default.Table_169e728924b749e4a9118c12d9b56eab [id=a2689bcea23843c9bb9e1f08e335e58f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.304726 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7e9a1d1ada941b68688e0ed6742aa99. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.305680 14401 tablet_service.cc:1467] Processing CreateTablet for tablet bfc34dc76ff94b8eaeb5c2822e6498da (DEFAULT_TABLE table=default.Table_a1547d1d923f4f4da05aaeb519f4cdf0 [id=e929ffaba567498c815393836b264a12]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.305766 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 9febec144bb343a8a21817e6d3655ad7 (DEFAULT_TABLE table=default.Table_27b5acf9bde44139b1b076e9579d88c7 [id=c9ed0b9c92ef40efa5c51cfdd8389325]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.305877 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfc34dc76ff94b8eaeb5c2822e6498da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.306053 14398 tablet_service.cc:1467] Processing CreateTablet for tablet a73d45bf459c4137af6961b680d282ac (DEFAULT_TABLE table=default.Table_70287f2d5a024cc88244993789eb747b [id=88146f846aa74ecfb3977f01ea45df2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.306227 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a73d45bf459c4137af6961b680d282ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.306636 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 5589b497b0e24740ad49d1392f5a737b (DEFAULT_TABLE table=default.Table_3d196756e234426799a1de651bc112f2 [id=37d965d8a5eb408ea3f33313bd22c152]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.306849 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5589b497b0e24740ad49d1392f5a737b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.307946 14395 tablet_service.cc:1467] Processing CreateTablet for tablet 66b748fc853f499fafc1f476c8f79b8b (DEFAULT_TABLE table=default.Table_7eacd8191bf44769b6371caa6d4772f1 [id=ea9e7141223d43de8962caa78ab14914]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.308123 14395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66b748fc853f499fafc1f476c8f79b8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.308511 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 0521973e23984355a77a321b4676b479 (DEFAULT_TABLE table=default.Table_39adda20d0304f8f8785171a07e25531 [id=0fde357ee96a4bb6b1a96a26ef4e37fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.308696 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0521973e23984355a77a321b4676b479. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.309211 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9febec144bb343a8a21817e6d3655ad7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.309242 14165 catalog_manager.cc:5526] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:16.309409 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 13ce129d8a9245baab7c772f0f5fbaa7 (table default.Table_e5c55332e306452685635c28fb3d4c55 [id=6cba4fb2f7104c6e953340b04e5dc494]). Expected version 2 got 0
I20250115 03:06:16.309834 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 85b4eb3b20bb4908884e5716e7b2a369 (DEFAULT_TABLE table=default.Table_fd907927ba11487b907e7130bdfb6092 [id=65805b1944554162a6e964b010bca1d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.310025 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85b4eb3b20bb4908884e5716e7b2a369. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.310364 14809 tablet_bootstrap.cc:492] T 5b7022db9edc4de8909e16769a3ea136 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.310429 14809 ts_tablet_manager.cc:1397] T 5b7022db9edc4de8909e16769a3ea136 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.061s user 0.001s sys 0.000s
I20250115 03:06:16.310838 14809 raft_consensus.cc:357] T 5b7022db9edc4de8909e16769a3ea136 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.310923 14809 raft_consensus.cc:383] T 5b7022db9edc4de8909e16769a3ea136 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.310950 14809 raft_consensus.cc:738] T 5b7022db9edc4de8909e16769a3ea136 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.311041 14809 consensus_queue.cc:260] T 5b7022db9edc4de8909e16769a3ea136 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.311259 14809 ts_tablet_manager.cc:1428] T 5b7022db9edc4de8909e16769a3ea136 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.311439 14809 tablet_bootstrap.cc:492] T ec9f14c1ab014f29843a568c460a28d6 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.311393 14660 tablet_service.cc:1467] Processing CreateTablet for tablet adbb0ae719f24dcc96709a83981e213d (DEFAULT_TABLE table=default.Table_e73230225df744a2a11f246bea89b8b0 [id=407331115fba4ca28b006a587584ea0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.311587 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adbb0ae719f24dcc96709a83981e213d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.312297 14809 tablet_bootstrap.cc:654] T ec9f14c1ab014f29843a568c460a28d6 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.313810 14400 tablet_service.cc:1467] Processing CreateTablet for tablet adbb0ae719f24dcc96709a83981e213d (DEFAULT_TABLE table=default.Table_e73230225df744a2a11f246bea89b8b0 [id=407331115fba4ca28b006a587584ea0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.314005 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adbb0ae719f24dcc96709a83981e213d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.316017 14527 tablet_service.cc:1467] Processing CreateTablet for tablet adbb0ae719f24dcc96709a83981e213d (DEFAULT_TABLE table=default.Table_e73230225df744a2a11f246bea89b8b0 [id=407331115fba4ca28b006a587584ea0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.316167 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 0521973e23984355a77a321b4676b479 (DEFAULT_TABLE table=default.Table_39adda20d0304f8f8785171a07e25531 [id=0fde357ee96a4bb6b1a96a26ef4e37fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.316228 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adbb0ae719f24dcc96709a83981e213d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.316334 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0521973e23984355a77a321b4676b479. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.316540 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 5589b497b0e24740ad49d1392f5a737b (DEFAULT_TABLE table=default.Table_3d196756e234426799a1de651bc112f2 [id=37d965d8a5eb408ea3f33313bd22c152]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.316808 14531 tablet_service.cc:1467] Processing CreateTablet for tablet bfc34dc76ff94b8eaeb5c2822e6498da (DEFAULT_TABLE table=default.Table_a1547d1d923f4f4da05aaeb519f4cdf0 [id=e929ffaba567498c815393836b264a12]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.316970 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 2c69daff62144976ae9082c89cc58eca (DEFAULT_TABLE table=default.Table_24b2f1f4bbb14d2fa7cba35e089f6707 [id=9c79f5b643664aebbc02a5d815485f61]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.317055 14533 tablet_service.cc:1467] Processing CreateTablet for tablet a73d45bf459c4137af6961b680d282ac (DEFAULT_TABLE table=default.Table_70287f2d5a024cc88244993789eb747b [id=88146f846aa74ecfb3977f01ea45df2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.317198 14529 tablet_service.cc:1467] Processing CreateTablet for tablet c6fa98c6fb304dc49b8e0ced989cc936 (DEFAULT_TABLE table=default.Table_a7ee880814fc40f98518c5149b8b5b4b [id=8f805d745ece4ec8845a43798122dd4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.317279 14534 tablet_service.cc:1467] Processing CreateTablet for tablet 66b748fc853f499fafc1f476c8f79b8b (DEFAULT_TABLE table=default.Table_7eacd8191bf44769b6371caa6d4772f1 [id=ea9e7141223d43de8962caa78ab14914]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.317337 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 9febec144bb343a8a21817e6d3655ad7 (DEFAULT_TABLE table=default.Table_27b5acf9bde44139b1b076e9579d88c7 [id=c9ed0b9c92ef40efa5c51cfdd8389325]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.317407 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 85b4eb3b20bb4908884e5716e7b2a369 (DEFAULT_TABLE table=default.Table_fd907927ba11487b907e7130bdfb6092 [id=65805b1944554162a6e964b010bca1d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.317533 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9febec144bb343a8a21817e6d3655ad7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.317548 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 5589b497b0e24740ad49d1392f5a737b (DEFAULT_TABLE table=default.Table_3d196756e234426799a1de651bc112f2 [id=37d965d8a5eb408ea3f33313bd22c152]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.317628 14532 tablet_service.cc:1467] Processing CreateTablet for tablet 9febec144bb343a8a21817e6d3655ad7 (DEFAULT_TABLE table=default.Table_27b5acf9bde44139b1b076e9579d88c7 [id=c9ed0b9c92ef40efa5c51cfdd8389325]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.317746 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5589b497b0e24740ad49d1392f5a737b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.317878 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5589b497b0e24740ad49d1392f5a737b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.318291 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 0521973e23984355a77a321b4676b479 (DEFAULT_TABLE table=default.Table_39adda20d0304f8f8785171a07e25531 [id=0fde357ee96a4bb6b1a96a26ef4e37fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.318478 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0521973e23984355a77a321b4676b479. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.319567 14672 tablet_service.cc:1467] Processing CreateTablet for tablet bfc34dc76ff94b8eaeb5c2822e6498da (DEFAULT_TABLE table=default.Table_a1547d1d923f4f4da05aaeb519f4cdf0 [id=e929ffaba567498c815393836b264a12]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.319756 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfc34dc76ff94b8eaeb5c2822e6498da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.319926 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 85b4eb3b20bb4908884e5716e7b2a369 (DEFAULT_TABLE table=default.Table_fd907927ba11487b907e7130bdfb6092 [id=65805b1944554162a6e964b010bca1d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.320101 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85b4eb3b20bb4908884e5716e7b2a369. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.320360 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_5be4a6fd260b4cea811c9928d9ac01b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:16.320456 14396 tablet_service.cc:1467] Processing CreateTablet for tablet c6fa98c6fb304dc49b8e0ced989cc936 (DEFAULT_TABLE table=default.Table_a7ee880814fc40f98518c5149b8b5b4b [id=8f805d745ece4ec8845a43798122dd4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.320663 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6fa98c6fb304dc49b8e0ced989cc936. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:16.320647 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5be4a6fd260b4cea811c9928d9ac01b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.321569 14664 tablet_service.cc:1467] Processing CreateTablet for tablet a73d45bf459c4137af6961b680d282ac (DEFAULT_TABLE table=default.Table_70287f2d5a024cc88244993789eb747b [id=88146f846aa74ecfb3977f01ea45df2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.321749 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a73d45bf459c4137af6961b680d282ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.322093 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85b4eb3b20bb4908884e5716e7b2a369. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.322458 14660 tablet_service.cc:1467] Processing CreateTablet for tablet c6fa98c6fb304dc49b8e0ced989cc936 (DEFAULT_TABLE table=default.Table_a7ee880814fc40f98518c5149b8b5b4b [id=8f805d745ece4ec8845a43798122dd4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.322662 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6fa98c6fb304dc49b8e0ced989cc936. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.323563 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 66b748fc853f499fafc1f476c8f79b8b (DEFAULT_TABLE table=default.Table_7eacd8191bf44769b6371caa6d4772f1 [id=ea9e7141223d43de8962caa78ab14914]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.323753 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66b748fc853f499fafc1f476c8f79b8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.324048 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6fa98c6fb304dc49b8e0ced989cc936. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.324591 14819 tablet_replica.cc:331] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.324698 14819 raft_consensus.cc:2238] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.324790 14819 raft_consensus.cc:2267] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.325217 14819 ts_tablet_manager.cc:1905] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.316813 14528 tablet_service.cc:1467] Processing CreateTablet for tablet ada45621f8864caab9189039fae0fae2 (DEFAULT_TABLE table=default.Table_f2946c2c8c56478f9108ffeda4034086 [id=2c3fb664638b434397824dec2147518c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.325857 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9febec144bb343a8a21817e6d3655ad7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.326169 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66b748fc853f499fafc1f476c8f79b8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.326282 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c69daff62144976ae9082c89cc58eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.326411 14166 master_service.cc:946] ReplaceTablet: received request to replace tablet d4f770f63a7348afb71904b93c68c379 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:16.327041 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ada45621f8864caab9189039fae0fae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.328258 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a73d45bf459c4137af6961b680d282ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.328289 14819 ts_tablet_manager.cc:1918] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.328339 14819 log.cc:1198] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/7f67cbb929cd4ae589a14606aa168b46
I20250115 03:06:16.328521 14819 ts_tablet_manager.cc:1939] T 7f67cbb929cd4ae589a14606aa168b46 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.328836 14531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfc34dc76ff94b8eaeb5c2822e6498da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.329066 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 7f67cbb929cd4ae589a14606aa168b46 (table default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]) successfully deleted
I20250115 03:06:16.330020 14809 tablet_bootstrap.cc:492] T ec9f14c1ab014f29843a568c460a28d6 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.330088 14809 ts_tablet_manager.cc:1397] T ec9f14c1ab014f29843a568c460a28d6 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.000s
I20250115 03:06:16.330401 14556 raft_consensus.cc:3054] T a2fc04950cfe4c608a4712afd257ca76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.330462 14809 raft_consensus.cc:357] T ec9f14c1ab014f29843a568c460a28d6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.330559 14809 raft_consensus.cc:383] T ec9f14c1ab014f29843a568c460a28d6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.330591 14809 raft_consensus.cc:738] T ec9f14c1ab014f29843a568c460a28d6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.330691 14809 consensus_queue.cc:260] T ec9f14c1ab014f29843a568c460a28d6 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.330947 14809 ts_tablet_manager.cc:1428] T ec9f14c1ab014f29843a568c460a28d6 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.331100 14809 tablet_bootstrap.cc:492] T 8baf86b37e9942ba813b099a0da87ca5 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.332126 14809 tablet_bootstrap.cc:654] T 8baf86b37e9942ba813b099a0da87ca5 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.332118 14168 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 4ff51d514fb84809aa53fa9fc1010681 (table default.Table_9cc1022af4f14e7a8a8e11cc1803ca87 [id=98f48acd89a6461bb4511141cab4d88b]). Expected version 2 got 0
I20250115 03:06:16.332610 15426 tablet.cc:1721] T a2fc04950cfe4c608a4712afd257ca76 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:16.332983 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet 80aa570e2c154172ad6da29d268bede7 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:16.333748 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet f5af8b81e7ff4acab5aef13003656988 with delete_type TABLET_DATA_DELETED (replaced by tablet 3efc2fb8427f493ea933347bec34f88f) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.333695 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_72e82b3e776e4d398fa5d8240afde936"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:16.333967 14820 tablet_replica.cc:331] T f5af8b81e7ff4acab5aef13003656988 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
W20250115 03:06:16.334004 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_72e82b3e776e4d398fa5d8240afde936 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.334088 14820 raft_consensus.cc:2238] T f5af8b81e7ff4acab5aef13003656988 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.334192 14820 raft_consensus.cc:2267] T f5af8b81e7ff4acab5aef13003656988 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.334980 14809 tablet_bootstrap.cc:492] T 8baf86b37e9942ba813b099a0da87ca5 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.335069 14809 ts_tablet_manager.cc:1397] T 8baf86b37e9942ba813b099a0da87ca5 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20250115 03:06:16.335458 15645 fs_manager.cc:714] Time spent opening directory manager: real 0.091s user 0.002s sys 0.000s
I20250115 03:06:16.335506 14809 raft_consensus.cc:357] T 8baf86b37e9942ba813b099a0da87ca5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.335579 14812 tablet_bootstrap.cc:492] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.335590 14809 raft_consensus.cc:383] T 8baf86b37e9942ba813b099a0da87ca5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.335618 14809 raft_consensus.cc:738] T 8baf86b37e9942ba813b099a0da87ca5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.335651 14812 ts_tablet_manager.cc:1397] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.051s user 0.000s sys 0.002s
I20250115 03:06:16.336089 14809 consensus_queue.cc:260] T 8baf86b37e9942ba813b099a0da87ca5 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.336431 14812 raft_consensus.cc:357] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.336550 14812 raft_consensus.cc:383] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.336622 14812 raft_consensus.cc:738] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.336768 14812 consensus_queue.cc:260] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.337071 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet 312dcdda8cf042648db9fdf6228062d9 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:16.337174 14809 ts_tablet_manager.cc:1428] T 8baf86b37e9942ba813b099a0da87ca5 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:16.337293 14811 tablet_bootstrap.cc:492] T 68974a1815cb482d8b28d54f42634608 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.337378 14811 ts_tablet_manager.cc:1397] T 68974a1815cb482d8b28d54f42634608 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.071s user 0.000s sys 0.001s
I20250115 03:06:16.337811 14811 raft_consensus.cc:357] T 68974a1815cb482d8b28d54f42634608 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.337874 14809 tablet_bootstrap.cc:492] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.337911 14811 raft_consensus.cc:383] T 68974a1815cb482d8b28d54f42634608 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.337940 14811 raft_consensus.cc:738] T 68974a1815cb482d8b28d54f42634608 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.338039 14811 consensus_queue.cc:260] T 68974a1815cb482d8b28d54f42634608 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.338299 14811 ts_tablet_manager.cc:1428] T 68974a1815cb482d8b28d54f42634608 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.338445 14811 tablet_bootstrap.cc:492] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.338748 14809 tablet_bootstrap.cc:654] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.339286 14811 tablet_bootstrap.cc:654] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.340600 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet 68974a1815cb482d8b28d54f42634608 with delete_type TABLET_DATA_DELETED (replaced by tablet 420b7c8d43a04bae972eed2014d5634c) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.342934 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_67f7434767174808bef25bc2c55394fd" } new_table_name: "default.Table_ec5f1644f3f6456f8d0d5ee407d7e7cd" modify_external_catalogs: true
I20250115 03:06:16.343716 14811 tablet_bootstrap.cc:492] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.343789 14811 ts_tablet_manager.cc:1397] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:16.344107 14811 raft_consensus.cc:357] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.344184 14811 raft_consensus.cc:383] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.344211 14811 raft_consensus.cc:738] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.344316 14811 consensus_queue.cc:260] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.344542 14811 ts_tablet_manager.cc:1428] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.344696 14811 tablet_bootstrap.cc:492] T 0f3ba1b2ff964d648e5f36f410107b5b P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.345760 14165 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 745344db22da4102bad39c2f121548a1 (table default.Table_365313892043408c92be8460c31d958e [id=f22f08e9af4a4496b100e7d4b5982121]). Expected version 1 got 0
I20250115 03:06:16.348253 14811 tablet_bootstrap.cc:654] T 0f3ba1b2ff964d648e5f36f410107b5b P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.348335 14820 ts_tablet_manager.cc:1905] T f5af8b81e7ff4acab5aef13003656988 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.348969 15730 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250115 03:06:16.349031 14812 ts_tablet_manager.cc:1428] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.013s user 0.000s sys 0.001s
I20250115 03:06:16.349771 14809 tablet_bootstrap.cc:492] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.349838 14809 ts_tablet_manager.cc:1397] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20250115 03:06:16.350258 14809 raft_consensus.cc:357] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.350346 14809 raft_consensus.cc:383] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.350375 14809 raft_consensus.cc:738] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.350382 14820 ts_tablet_manager.cc:1918] T f5af8b81e7ff4acab5aef13003656988 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.350437 14820 log.cc:1198] T f5af8b81e7ff4acab5aef13003656988 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/f5af8b81e7ff4acab5aef13003656988
I20250115 03:06:16.350476 14809 consensus_queue.cc:260] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.350615 14820 ts_tablet_manager.cc:1939] T f5af8b81e7ff4acab5aef13003656988 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.350894 14812 tablet_bootstrap.cc:492] T 48aa039bf0364c02abb7f9677eae49e5 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.350914 14809 ts_tablet_manager.cc:1428] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.351052 14809 tablet_bootstrap.cc:492] T 6864f88bf6454ee980f624ec2f2d9fb6 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.351794 14812 tablet_bootstrap.cc:654] T 48aa039bf0364c02abb7f9677eae49e5 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.351891 14809 tablet_bootstrap.cc:654] T 6864f88bf6454ee980f624ec2f2d9fb6 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.352912 15401 raft_consensus.cc:491] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.352952 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4f241c86102a4bbfb509a2d5d76953e9" } modify_external_catalogs: true
I20250115 03:06:16.353047 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4f241c86102a4bbfb509a2d5d76953e9" } modify_external_catalogs: true
I20250115 03:06:16.353014 15401 raft_consensus.cc:513] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.353067 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 745344db22da4102bad39c2f121548a1 (table default.Table_365313892043408c92be8460c31d958e [id=f22f08e9af4a4496b100e7d4b5982121]). Expected version 1 got 0
I20250115 03:06:16.353327 15401 leader_election.cc:290] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.354614 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4484651d964411e90ed93c6caab0a13" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.354641 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4484651d964411e90ed93c6caab0a13" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:16.354727 14555 raft_consensus.cc:2463] T a4484651d964411e90ed93c6caab0a13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:16.354727 14691 raft_consensus.cc:2463] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:16.354990 14358 leader_election.cc:304] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.355156 15401 raft_consensus.cc:2798] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:16.355203 15401 raft_consensus.cc:491] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:16.355240 15401 raft_consensus.cc:3054] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.355345 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 80aa570e2c154172ad6da29d268bede7
I20250115 03:06:16.355460 14811 tablet_bootstrap.cc:492] T 0f3ba1b2ff964d648e5f36f410107b5b P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.355533 14811 ts_tablet_manager.cc:1397] T 0f3ba1b2ff964d648e5f36f410107b5b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20250115 03:06:16.355607 14163 catalog_manager.cc:6596] ReplaceTablet: tablet 80aa570e2c154172ad6da29d268bede7 deleted and replaced by tablet 815ef85ce5cf4ae0a56c5734837e03ba
I20250115 03:06:16.355888 14811 raft_consensus.cc:357] T 0f3ba1b2ff964d648e5f36f410107b5b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.355973 14811 raft_consensus.cc:383] T 0f3ba1b2ff964d648e5f36f410107b5b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.356002 14811 raft_consensus.cc:738] T 0f3ba1b2ff964d648e5f36f410107b5b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.356077 15401 raft_consensus.cc:513] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.356092 14811 consensus_queue.cc:260] T 0f3ba1b2ff964d648e5f36f410107b5b P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:16.356221 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cff95c69fb9045cf9ec0ebb128c9ef6b consensus state has no leader
I20250115 03:06:16.356324 14811 ts_tablet_manager.cc:1428] T 0f3ba1b2ff964d648e5f36f410107b5b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.356375 15401 leader_election.cc:290] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.356472 14811 tablet_bootstrap.cc:492] T 9808c7f96a3147848da5f78948895699 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.356488 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 80aa570e2c154172ad6da29d268bede7 with delete_type TABLET_DATA_DELETED (replaced by tablet 815ef85ce5cf4ae0a56c5734837e03ba) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.356848 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4484651d964411e90ed93c6caab0a13" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:16.356966 14691 raft_consensus.cc:3054] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.357091 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4484651d964411e90ed93c6caab0a13" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:16.357175 14556 raft_consensus.cc:3054] T a4484651d964411e90ed93c6caab0a13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.357313 14811 tablet_bootstrap.cc:654] T 9808c7f96a3147848da5f78948895699 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.357759 14556 raft_consensus.cc:2463] T a4484651d964411e90ed93c6caab0a13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:16.357872 14691 raft_consensus.cc:2463] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:16.357970 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet d4f770f63a7348afb71904b93c68c379
I20250115 03:06:16.358170 14358 leader_election.cc:304] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.358260 14166 catalog_manager.cc:6596] ReplaceTablet: tablet d4f770f63a7348afb71904b93c68c379 deleted and replaced by tablet a4547465f3124e108dc2c4c54e2bc751
I20250115 03:06:16.358696 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet d4f770f63a7348afb71904b93c68c379 with delete_type TABLET_DATA_DELETED (replaced by tablet a4547465f3124e108dc2c4c54e2bc751) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.358971 15401 raft_consensus.cc:2798] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:16.359045 15401 raft_consensus.cc:695] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:16.359061 14809 tablet_bootstrap.cc:492] T 6864f88bf6454ee980f624ec2f2d9fb6 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.359117 14809 ts_tablet_manager.cc:1397] T 6864f88bf6454ee980f624ec2f2d9fb6 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20250115 03:06:16.359146 15401 consensus_queue.cc:237] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.359494 14809 raft_consensus.cc:357] T 6864f88bf6454ee980f624ec2f2d9fb6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.359576 14809 raft_consensus.cc:383] T 6864f88bf6454ee980f624ec2f2d9fb6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.359606 14809 raft_consensus.cc:738] T 6864f88bf6454ee980f624ec2f2d9fb6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.359691 14809 consensus_queue.cc:260] T 6864f88bf6454ee980f624ec2f2d9fb6 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.359923 14809 ts_tablet_manager.cc:1428] T 6864f88bf6454ee980f624ec2f2d9fb6 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:16.360070 14809 tablet_bootstrap.cc:492] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.360852 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet d4f770f63a7348afb71904b93c68c379 with delete_type TABLET_DATA_DELETED (replaced by tablet a4547465f3124e108dc2c4c54e2bc751) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.361023 14823 tablet_replica.cc:331] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.361027 14809 tablet_bootstrap.cc:654] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.361131 14823 raft_consensus.cc:2238] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:16.361295 14823 raft_consensus.cc:2267] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.361563 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 93fa227c0de74a3fa86778641fdcda3d
I20250115 03:06:16.361649 14823 ts_tablet_manager.cc:1905] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.361932 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_35005559d2f34672bc3e70f6bbfb31d6" } modify_external_catalogs: true
I20250115 03:06:16.362025 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_35005559d2f34672bc3e70f6bbfb31d6" } modify_external_catalogs: true
I20250115 03:06:16.362985 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 93fa227c0de74a3fa86778641fdcda3d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.363499 14820 tablet_replica.cc:331] T 68974a1815cb482d8b28d54f42634608 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.363610 14820 raft_consensus.cc:2238] T 68974a1815cb482d8b28d54f42634608 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.363623 14823 ts_tablet_manager.cc:1918] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:16.363680 14823 log.cc:1198] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/d4f770f63a7348afb71904b93c68c379
I20250115 03:06:16.363698 14820 raft_consensus.cc:2267] T 68974a1815cb482d8b28d54f42634608 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.363932 14823 ts_tablet_manager.cc:1939] T d4f770f63a7348afb71904b93c68c379 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.364241 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet 80aa570e2c154172ad6da29d268bede7 with delete_type TABLET_DATA_DELETED (replaced by tablet 815ef85ce5cf4ae0a56c5734837e03ba) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.364289 14531 tablet_service.cc:1514] Processing DeleteTablet for tablet d4f770f63a7348afb71904b93c68c379 with delete_type TABLET_DATA_DELETED (replaced by tablet a4547465f3124e108dc2c4c54e2bc751) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.364379 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 93fa227c0de74a3fa86778641fdcda3d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.364404 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet f5af8b81e7ff4acab5aef13003656988 (table default.Table_f4fc7a15692446fba7fcdb17b6b33d56 [id=364a7905f3f044f684e9c75927f7efda]) successfully deleted
I20250115 03:06:16.364832 14820 ts_tablet_manager.cc:1905] T 68974a1815cb482d8b28d54f42634608 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.364972 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet d4f770f63a7348afb71904b93c68c379 (table default.Table_58e0901d0ec94ef9859515f126825621 [id=50c73c26b4ed47a4961348db433b7dd4]) successfully deleted
W20250115 03:06:16.365134 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 93fa227c0de74a3fa86778641fdcda3d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 93fa227c0de74a3fa86778641fdcda3d already in progress: creating tablet
I20250115 03:06:16.365648 14819 tablet_replica.cc:331] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.365756 14819 raft_consensus.cc:2238] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.365847 14819 raft_consensus.cc:2267] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.366158 14819 ts_tablet_manager.cc:1905] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:16.366338 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 80aa570e2c154172ad6da29d268bede7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 80aa570e2c154172ad6da29d268bede7 already in progress: creating tablet
I20250115 03:06:16.366449 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f641c3009a9045c79adb4e38c38e8f1c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.366762 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f641c3009a9045c79adb4e38c38e8f1c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.368098 14819 ts_tablet_manager.cc:1918] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:16.368170 14819 log.cc:1198] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/d4f770f63a7348afb71904b93c68c379
I20250115 03:06:16.368304 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 312dcdda8cf042648db9fdf6228062d9
I20250115 03:06:16.368376 14819 ts_tablet_manager.cc:1939] T d4f770f63a7348afb71904b93c68c379 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.368595 14171 catalog_manager.cc:6596] ReplaceTablet: tablet 312dcdda8cf042648db9fdf6228062d9 deleted and replaced by tablet 7b86e66428504217af08f667f660d304
I20250115 03:06:16.369146 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 80aa570e2c154172ad6da29d268bede7 with delete_type TABLET_DATA_DELETED (replaced by tablet 815ef85ce5cf4ae0a56c5734837e03ba) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.369160 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet d4f770f63a7348afb71904b93c68c379 (table default.Table_58e0901d0ec94ef9859515f126825621 [id=50c73c26b4ed47a4961348db433b7dd4]) successfully deleted
I20250115 03:06:16.369206 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 312dcdda8cf042648db9fdf6228062d9 with delete_type TABLET_DATA_DELETED (replaced by tablet 7b86e66428504217af08f667f660d304) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.369269 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 312dcdda8cf042648db9fdf6228062d9 with delete_type TABLET_DATA_DELETED (replaced by tablet 7b86e66428504217af08f667f660d304) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:16.369292 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 93fa227c0de74a3fa86778641fdcda3d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 93fa227c0de74a3fa86778641fdcda3d already in progress: creating tablet
I20250115 03:06:16.369608 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_426742b2ee4d41da8f060fb20309d184"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.369900 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_426742b2ee4d41da8f060fb20309d184 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.369943 14811 tablet_bootstrap.cc:492] T 9808c7f96a3147848da5f78948895699 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.370016 14811 ts_tablet_manager.cc:1397] T 9808c7f96a3147848da5f78948895699 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20250115 03:06:16.370414 14811 raft_consensus.cc:357] T 9808c7f96a3147848da5f78948895699 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:16.370474 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 312dcdda8cf042648db9fdf6228062d9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 312dcdda8cf042648db9fdf6228062d9 already in progress: creating tablet
I20250115 03:06:16.370523 14811 raft_consensus.cc:383] T 9808c7f96a3147848da5f78948895699 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.370559 14811 raft_consensus.cc:738] T 9808c7f96a3147848da5f78948895699 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.370666 14811 consensus_queue.cc:260] T 9808c7f96a3147848da5f78948895699 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.370926 14811 ts_tablet_manager.cc:1428] T 9808c7f96a3147848da5f78948895699 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.371096 14811 tablet_bootstrap.cc:492] T db1aa206e9994de6b9a355a621cb2ab3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.371093 14820 ts_tablet_manager.cc:1918] T 68974a1815cb482d8b28d54f42634608 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.371336 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_1f20c003e8664c2381603de8f2055749"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:16.371562 14820 log.cc:1198] T 68974a1815cb482d8b28d54f42634608 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/68974a1815cb482d8b28d54f42634608
W20250115 03:06:16.371609 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1f20c003e8664c2381603de8f2055749 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.371846 14820 ts_tablet_manager.cc:1939] T 68974a1815cb482d8b28d54f42634608 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.371855 15645 fs_manager.cc:730] Time spent opening block manager: real 0.035s user 0.001s sys 0.000s
I20250115 03:06:16.371946 14811 tablet_bootstrap.cc:654] T db1aa206e9994de6b9a355a621cb2ab3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.372210 14531 tablet_service.cc:1514] Processing DeleteTablet for tablet 312dcdda8cf042648db9fdf6228062d9 with delete_type TABLET_DATA_DELETED (replaced by tablet 7b86e66428504217af08f667f660d304) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.371974 15645 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/data,/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/wal
uuid: "2db8cff032af46db986bf52b21b52246"
format_stamp: "Formatted at 2025-01-15 03:06:10 on dist-test-slave-c74p"
I20250115 03:06:16.372491 15645 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250115 03:06:16.372694 14812 tablet_bootstrap.cc:492] T 48aa039bf0364c02abb7f9677eae49e5 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.372766 14812 ts_tablet_manager.cc:1397] T 48aa039bf0364c02abb7f9677eae49e5 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s
I20250115 03:06:16.373184 14812 raft_consensus.cc:357] T 48aa039bf0364c02abb7f9677eae49e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.373286 14812 raft_consensus.cc:383] T 48aa039bf0364c02abb7f9677eae49e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.373322 14812 raft_consensus.cc:738] T 48aa039bf0364c02abb7f9677eae49e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.373441 14812 consensus_queue.cc:260] T 48aa039bf0364c02abb7f9677eae49e5 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.373708 14812 ts_tablet_manager.cc:1428] T 48aa039bf0364c02abb7f9677eae49e5 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.373772 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 93fa227c0de74a3fa86778641fdcda3d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.373886 14812 tablet_bootstrap.cc:492] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.374608 14820 tablet_replica.cc:331] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.374711 14820 raft_consensus.cc:2238] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.374753 14812 tablet_bootstrap.cc:654] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.374814 14820 raft_consensus.cc:2267] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.376832 15453 raft_consensus.cc:491] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.376900 15453 raft_consensus.cc:513] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.377009 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3ba1b2ff964d648e5f36f410107b5b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.377202 15453 leader_election.cc:290] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:16.377453 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 68974a1815cb482d8b28d54f42634608 (table default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]) successfully deleted
W20250115 03:06:16.377586 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 80aa570e2c154172ad6da29d268bede7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 80aa570e2c154172ad6da29d268bede7 already in progress: creating tablet
I20250115 03:06:16.377743 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "745344db22da4102bad39c2f121548a1" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:16.377954 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 80aa570e2c154172ad6da29d268bede7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 80aa570e2c154172ad6da29d268bede7 already in progress: creating tablet
W20250115 03:06:16.378038 14628 leader_election.cc:343] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:16.378093 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 312dcdda8cf042648db9fdf6228062d9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 312dcdda8cf042648db9fdf6228062d9 already in progress: creating tablet
I20250115 03:06:16.378235 14820 ts_tablet_manager.cc:1905] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.378253 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "745344db22da4102bad39c2f121548a1" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.378413 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 9808c7f96a3147848da5f78948895699 with delete_type TABLET_DATA_DELETED (replaced by tablet 341b24d9b63f4761b2013035023fcd98) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.378582 15401 raft_consensus.cc:491] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.378648 15401 raft_consensus.cc:513] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.378965 15401 leader_election.cc:290] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.379370 14809 tablet_bootstrap.cc:492] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.379437 14809 ts_tablet_manager.cc:1397] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.001s
I20250115 03:06:16.379582 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet ec9f14c1ab014f29843a568c460a28d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:16.379798 14626 leader_election.cc:343] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.379841 14809 raft_consensus.cc:357] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.379849 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e594ea5cff81475886cdceb8400645a6" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:16.379912 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 8baf86b37e9942ba813b099a0da87ca5 with delete_type TABLET_DATA_DELETED (replaced by tablet bfc34dc76ff94b8eaeb5c2822e6498da) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.379956 14687 raft_consensus.cc:2463] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:16.379979 14819 tablet_replica.cc:331] T ec9f14c1ab014f29843a568c460a28d6 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.379858 14626 leader_election.cc:304] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.380069 14819 raft_consensus.cc:2238] T ec9f14c1ab014f29843a568c460a28d6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.380139 14819 raft_consensus.cc:2267] T ec9f14c1ab014f29843a568c460a28d6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.380185 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b7022db9edc4de8909e16769a3ea136 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.380198 14358 leader_election.cc:304] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.380249 15453 raft_consensus.cc:2743] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.379921 14809 raft_consensus.cc:383] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.380381 14809 raft_consensus.cc:738] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.380503 14809 consensus_queue.cc:260] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.380564 14819 ts_tablet_manager.cc:1905] T ec9f14c1ab014f29843a568c460a28d6 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.380743 14809 ts_tablet_manager.cc:1428] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.380811 14861 raft_consensus.cc:2798] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:16.380859 14861 raft_consensus.cc:491] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:16.380892 14861 raft_consensus.cc:3054] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.380903 14809 tablet_bootstrap.cc:492] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.381789 14809 tablet_bootstrap.cc:654] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.381783 14861 raft_consensus.cc:513] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.382097 14861 leader_election.cc:290] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.382701 14812 tablet_bootstrap.cc:492] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.382772 14812 ts_tablet_manager.cc:1397] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:16.383170 14812 raft_consensus.cc:357] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.383265 14812 raft_consensus.cc:383] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.383297 14812 raft_consensus.cc:738] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.383395 14812 consensus_queue.cc:260] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.383662 14812 ts_tablet_manager.cc:1428] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.383662 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e594ea5cff81475886cdceb8400645a6" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.383647 14169 catalog_manager.cc:5526] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:16.383752 14556 raft_consensus.cc:2463] T e594ea5cff81475886cdceb8400645a6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:16.383817 14812 tablet_bootstrap.cc:492] T 420b7c8d43a04bae972eed2014d5634c P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.384423 14819 ts_tablet_manager.cc:1918] T ec9f14c1ab014f29843a568c460a28d6 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.384479 14819 log.cc:1198] T ec9f14c1ab014f29843a568c460a28d6 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/ec9f14c1ab014f29843a568c460a28d6
I20250115 03:06:16.384652 14812 tablet_bootstrap.cc:654] T 420b7c8d43a04bae972eed2014d5634c P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.384656 14819 ts_tablet_manager.cc:1939] T ec9f14c1ab014f29843a568c460a28d6 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.383872 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet a4c3ff892f9b4dc484ffc80d7c5934c4 (table default.Table_76ef9277dc8142f1a86962bfc430f2b2 [id=f70558594b8348579a0ad5d4c8fd8885]). Expected version 1 got 0
I20250115 03:06:16.385018 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e594ea5cff81475886cdceb8400645a6" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:16.385107 14555 raft_consensus.cc:3054] T e594ea5cff81475886cdceb8400645a6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.385267 14819 tablet_replica.cc:331] T 8baf86b37e9942ba813b099a0da87ca5 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.385363 14819 raft_consensus.cc:2238] T 8baf86b37e9942ba813b099a0da87ca5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.385434 14819 raft_consensus.cc:2267] T 8baf86b37e9942ba813b099a0da87ca5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.385552 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet ec9f14c1ab014f29843a568c460a28d6 (table default.Table_60943a26c80945ae84effaed75c0cc42 [id=d94e2b655f424f549af052bc8dee0430]) successfully deleted
I20250115 03:06:16.385972 14555 raft_consensus.cc:2463] T e594ea5cff81475886cdceb8400645a6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:16.383368 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e594ea5cff81475886cdceb8400645a6" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:16.387301 14691 raft_consensus.cc:3054] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.388177 14691 raft_consensus.cc:2463] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:16.388517 14358 leader_election.cc:304] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.388844 15401 raft_consensus.cc:2798] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:16.388916 15401 raft_consensus.cc:695] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:16.389042 15401 consensus_queue.cc:237] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.389146 14819 ts_tablet_manager.cc:1905] T 8baf86b37e9942ba813b099a0da87ca5 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.390239 14820 ts_tablet_manager.cc:1918] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:16.390349 14820 log.cc:1198] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/d4f770f63a7348afb71904b93c68c379
I20250115 03:06:16.390623 14820 ts_tablet_manager.cc:1939] T d4f770f63a7348afb71904b93c68c379 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.391036 14819 ts_tablet_manager.cc:1918] T 8baf86b37e9942ba813b099a0da87ca5 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.391108 14819 log.cc:1198] T 8baf86b37e9942ba813b099a0da87ca5 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/8baf86b37e9942ba813b099a0da87ca5
I20250115 03:06:16.391311 14819 ts_tablet_manager.cc:1939] T 8baf86b37e9942ba813b099a0da87ca5 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.391479 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet d4f770f63a7348afb71904b93c68c379 (table default.Table_58e0901d0ec94ef9859515f126825621 [id=50c73c26b4ed47a4961348db433b7dd4]) successfully deleted
I20250115 03:06:16.391785 14819 tablet_replica.cc:331] T 5b7022db9edc4de8909e16769a3ea136 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
W20250115 03:06:16.391763 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 312dcdda8cf042648db9fdf6228062d9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 312dcdda8cf042648db9fdf6228062d9 already in progress: creating tablet
I20250115 03:06:16.391894 14819 raft_consensus.cc:2238] T 5b7022db9edc4de8909e16769a3ea136 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.391973 14819 raft_consensus.cc:2267] T 5b7022db9edc4de8909e16769a3ea136 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.392308 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 8baf86b37e9942ba813b099a0da87ca5 (table default.Table_a1547d1d923f4f4da05aaeb519f4cdf0 [id=e929ffaba567498c815393836b264a12]) successfully deleted
I20250115 03:06:16.392376 14819 ts_tablet_manager.cc:1905] T 5b7022db9edc4de8909e16769a3ea136 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:16.392902 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 93fa227c0de74a3fa86778641fdcda3d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 93fa227c0de74a3fa86778641fdcda3d already in progress: creating tablet
I20250115 03:06:16.393103 14820 tablet_replica.cc:331] T 0f3ba1b2ff964d648e5f36f410107b5b P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.393218 14820 raft_consensus.cc:2238] T 0f3ba1b2ff964d648e5f36f410107b5b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.393373 14820 raft_consensus.cc:2267] T 0f3ba1b2ff964d648e5f36f410107b5b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.393801 14820 ts_tablet_manager.cc:1905] T 0f3ba1b2ff964d648e5f36f410107b5b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.394893 14819 ts_tablet_manager.cc:1918] T 5b7022db9edc4de8909e16769a3ea136 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.394963 14819 log.cc:1198] T 5b7022db9edc4de8909e16769a3ea136 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/5b7022db9edc4de8909e16769a3ea136
I20250115 03:06:16.395139 14819 ts_tablet_manager.cc:1939] T 5b7022db9edc4de8909e16769a3ea136 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.395648 14820 ts_tablet_manager.cc:1918] T 0f3ba1b2ff964d648e5f36f410107b5b P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.395660 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 5b7022db9edc4de8909e16769a3ea136 (table default.Table_a5cd96954b1c48c68a3bef815c47d8b4 [id=f94088b0776f4be990ddcec04afbca43]) successfully deleted
I20250115 03:06:16.395717 14820 log.cc:1198] T 0f3ba1b2ff964d648e5f36f410107b5b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/0f3ba1b2ff964d648e5f36f410107b5b
I20250115 03:06:16.395915 14820 ts_tablet_manager.cc:1939] T 0f3ba1b2ff964d648e5f36f410107b5b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.396292 14820 tablet_replica.cc:331] T 9808c7f96a3147848da5f78948895699 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.396394 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 0f3ba1b2ff964d648e5f36f410107b5b (table default.Table_e83118015644482ca56543e76b9691bd [id=089b3b2c658a4702869cf70ddd9baf73]) successfully deleted
I20250115 03:06:16.396783 14820 raft_consensus.cc:2238] T 9808c7f96a3147848da5f78948895699 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.396857 14811 tablet_bootstrap.cc:492] T db1aa206e9994de6b9a355a621cb2ab3 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.396950 14811 ts_tablet_manager.cc:1397] T db1aa206e9994de6b9a355a621cb2ab3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20250115 03:06:16.396967 14820 raft_consensus.cc:2267] T 9808c7f96a3147848da5f78948895699 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.397033 14812 tablet_bootstrap.cc:492] T 420b7c8d43a04bae972eed2014d5634c P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.397099 14812 ts_tablet_manager.cc:1397] T 420b7c8d43a04bae972eed2014d5634c P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.001s
I20250115 03:06:16.397372 14820 ts_tablet_manager.cc:1905] T 9808c7f96a3147848da5f78948895699 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.397580 14812 raft_consensus.cc:357] T 420b7c8d43a04bae972eed2014d5634c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.397666 14812 raft_consensus.cc:383] T 420b7c8d43a04bae972eed2014d5634c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.397692 14812 raft_consensus.cc:738] T 420b7c8d43a04bae972eed2014d5634c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.397795 14812 consensus_queue.cc:260] T 420b7c8d43a04bae972eed2014d5634c P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.398041 14812 ts_tablet_manager.cc:1428] T 420b7c8d43a04bae972eed2014d5634c P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.397301 14811 raft_consensus.cc:357] T db1aa206e9994de6b9a355a621cb2ab3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.398108 14811 raft_consensus.cc:383] T db1aa206e9994de6b9a355a621cb2ab3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.398152 14811 raft_consensus.cc:738] T db1aa206e9994de6b9a355a621cb2ab3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.398195 14812 tablet_bootstrap.cc:492] T f8b90f854a5e4221a24986f8c71ca300 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.398265 14811 consensus_queue.cc:260] T db1aa206e9994de6b9a355a621cb2ab3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.398499 14811 ts_tablet_manager.cc:1428] T db1aa206e9994de6b9a355a621cb2ab3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:16.399080 14812 tablet_bootstrap.cc:654] T f8b90f854a5e4221a24986f8c71ca300 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.385051 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 6864f88bf6454ee980f624ec2f2d9fb6 (table default.Table_2e68c93eaa1647df9b269879bd4520ab [id=18e0f06feeda4bca860e6f1bc6eab15b]). Expected version 1 got 0
I20250115 03:06:16.398655 14811 tablet_bootstrap.cc:492] T 13ce129d8a9245baab7c772f0f5fbaa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.400362 14811 tablet_bootstrap.cc:654] T 13ce129d8a9245baab7c772f0f5fbaa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.400852 14812 tablet_bootstrap.cc:492] T f8b90f854a5e4221a24986f8c71ca300 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.400939 14812 ts_tablet_manager.cc:1397] T f8b90f854a5e4221a24986f8c71ca300 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:16.401336 14812 raft_consensus.cc:357] T f8b90f854a5e4221a24986f8c71ca300 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.401427 14812 raft_consensus.cc:383] T f8b90f854a5e4221a24986f8c71ca300 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.401458 14812 raft_consensus.cc:738] T f8b90f854a5e4221a24986f8c71ca300 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.401559 14812 consensus_queue.cc:260] T f8b90f854a5e4221a24986f8c71ca300 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.401795 14812 ts_tablet_manager.cc:1428] T f8b90f854a5e4221a24986f8c71ca300 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.401953 14812 tablet_bootstrap.cc:492] T 3a0bb9b5093b454d921bf1150d631565 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.402606 14809 tablet_bootstrap.cc:492] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.402678 14809 ts_tablet_manager.cc:1397] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s
I20250115 03:06:16.402802 14812 tablet_bootstrap.cc:654] T 3a0bb9b5093b454d921bf1150d631565 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.403098 14809 raft_consensus.cc:357] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.403191 14809 raft_consensus.cc:383] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.403223 14809 raft_consensus.cc:738] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.403326 14809 consensus_queue.cc:260] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.403577 14809 ts_tablet_manager.cc:1428] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.403744 14809 tablet_bootstrap.cc:492] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.404577 14809 tablet_bootstrap.cc:654] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.404843 14812 tablet_bootstrap.cc:492] T 3a0bb9b5093b454d921bf1150d631565 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.404918 14812 ts_tablet_manager.cc:1397] T 3a0bb9b5093b454d921bf1150d631565 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:16.405331 14812 raft_consensus.cc:357] T 3a0bb9b5093b454d921bf1150d631565 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.405412 14812 raft_consensus.cc:383] T 3a0bb9b5093b454d921bf1150d631565 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.405452 14812 raft_consensus.cc:738] T 3a0bb9b5093b454d921bf1150d631565 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.405545 14812 consensus_queue.cc:260] T 3a0bb9b5093b454d921bf1150d631565 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.405776 14812 ts_tablet_manager.cc:1428] T 3a0bb9b5093b454d921bf1150d631565 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.405936 14812 tablet_bootstrap.cc:492] T 33380937a3b7428995b06e477a896c02 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.406759 14812 tablet_bootstrap.cc:654] T 33380937a3b7428995b06e477a896c02 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.407996 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet db1aa206e9994de6b9a355a621cb2ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.409287 14812 tablet_bootstrap.cc:492] T 33380937a3b7428995b06e477a896c02 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.409359 14812 ts_tablet_manager.cc:1397] T 33380937a3b7428995b06e477a896c02 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20250115 03:06:16.409744 14812 raft_consensus.cc:357] T 33380937a3b7428995b06e477a896c02 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.409834 14812 raft_consensus.cc:383] T 33380937a3b7428995b06e477a896c02 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.409864 14812 raft_consensus.cc:738] T 33380937a3b7428995b06e477a896c02 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.409961 14812 consensus_queue.cc:260] T 33380937a3b7428995b06e477a896c02 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:16.410043 14157 catalog_manager.cc:4670] Async tablet task AlterTable RPC for tablet 58130f2b27b745d08c0002ac96af6d72 (table default.Table_67f9f825d77c4874bb1d4a5c17dedde2 [id=6aa86d8861174b9091f29b0cfd39fa55], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 58130f2b27b745d08c0002ac96af6d72 consensus state has no leader
I20250115 03:06:16.410197 14812 ts_tablet_manager.cc:1428] T 33380937a3b7428995b06e477a896c02 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.410331 14812 tablet_bootstrap.cc:492] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.411128 14812 tablet_bootstrap.cc:654] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.412778 14809 tablet_bootstrap.cc:492] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.412778 14820 ts_tablet_manager.cc:1918] T 9808c7f96a3147848da5f78948895699 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.412842 14820 log.cc:1198] T 9808c7f96a3147848da5f78948895699 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/9808c7f96a3147848da5f78948895699
I20250115 03:06:16.412863 14809 ts_tablet_manager.cc:1397] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20250115 03:06:16.413152 14820 ts_tablet_manager.cc:1939] T 9808c7f96a3147848da5f78948895699 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.413744 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 9808c7f96a3147848da5f78948895699 (table default.Table_af3419632bb0478bab77f1fa32e9fb52 [id=369c42ec23ec4a27a71a54be23a410a6]) successfully deleted
I20250115 03:06:16.413934 14820 tablet_replica.cc:331] T db1aa206e9994de6b9a355a621cb2ab3 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.414060 14820 raft_consensus.cc:2238] T db1aa206e9994de6b9a355a621cb2ab3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.414155 14820 raft_consensus.cc:2267] T db1aa206e9994de6b9a355a621cb2ab3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.414144 14809 raft_consensus.cc:357] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.414243 14809 raft_consensus.cc:383] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.414284 14809 raft_consensus.cc:738] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.414397 14809 consensus_queue.cc:260] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.414664 14809 ts_tablet_manager.cc:1428] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:16.414817 14809 tablet_bootstrap.cc:492] T 82be772e05d443749914004aea8af300 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.415588 14820 ts_tablet_manager.cc:1905] T db1aa206e9994de6b9a355a621cb2ab3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.415776 14809 tablet_bootstrap.cc:654] T 82be772e05d443749914004aea8af300 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.416966 14812 tablet_bootstrap.cc:492] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.417047 14812 ts_tablet_manager.cc:1397] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:16.417431 14812 raft_consensus.cc:357] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.417541 14812 raft_consensus.cc:383] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.417580 14812 raft_consensus.cc:738] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.417693 14812 consensus_queue.cc:260] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.417949 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 48aa039bf0364c02abb7f9677eae49e5
I20250115 03:06:16.417965 14812 ts_tablet_manager.cc:1428] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.418120 14812 tablet_bootstrap.cc:492] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.418684 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 48aa039bf0364c02abb7f9677eae49e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.418906 14812 tablet_bootstrap.cc:654] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.419814 14820 ts_tablet_manager.cc:1918] T db1aa206e9994de6b9a355a621cb2ab3 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.419878 14820 log.cc:1198] T db1aa206e9994de6b9a355a621cb2ab3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/db1aa206e9994de6b9a355a621cb2ab3
I20250115 03:06:16.420044 14820 ts_tablet_manager.cc:1939] T db1aa206e9994de6b9a355a621cb2ab3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.420811 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 48aa039bf0364c02abb7f9677eae49e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.420949 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 48aa039bf0364c02abb7f9677eae49e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:16.421309 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 48aa039bf0364c02abb7f9677eae49e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 48aa039bf0364c02abb7f9677eae49e5 already in progress: creating tablet
I20250115 03:06:16.421695 14823 tablet_replica.cc:331] T 48aa039bf0364c02abb7f9677eae49e5 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.421808 14823 raft_consensus.cc:2238] T 48aa039bf0364c02abb7f9677eae49e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.421880 14823 raft_consensus.cc:2267] T 48aa039bf0364c02abb7f9677eae49e5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.422154 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet db1aa206e9994de6b9a355a621cb2ab3 (table default.Table_e050a17d73d342fda8108a99a665f222 [id=eea2f4b389314abbac5f983a79d5cf7e]) successfully deleted
I20250115 03:06:16.422284 14823 ts_tablet_manager.cc:1905] T 48aa039bf0364c02abb7f9677eae49e5 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:16.422539 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 48aa039bf0364c02abb7f9677eae49e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 48aa039bf0364c02abb7f9677eae49e5 already in progress: creating tablet
I20250115 03:06:16.423122 14809 tablet_bootstrap.cc:492] T 82be772e05d443749914004aea8af300 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.423199 14809 ts_tablet_manager.cc:1397] T 82be772e05d443749914004aea8af300 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20250115 03:06:16.423614 14809 raft_consensus.cc:357] T 82be772e05d443749914004aea8af300 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.423708 14809 raft_consensus.cc:383] T 82be772e05d443749914004aea8af300 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.423748 14809 raft_consensus.cc:738] T 82be772e05d443749914004aea8af300 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.423848 14809 consensus_queue.cc:260] T 82be772e05d443749914004aea8af300 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.424131 14809 ts_tablet_manager.cc:1428] T 82be772e05d443749914004aea8af300 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.424358 14823 ts_tablet_manager.cc:1918] T 48aa039bf0364c02abb7f9677eae49e5 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.424427 14823 log.cc:1198] T 48aa039bf0364c02abb7f9677eae49e5 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/48aa039bf0364c02abb7f9677eae49e5
I20250115 03:06:16.424618 14823 ts_tablet_manager.cc:1939] T 48aa039bf0364c02abb7f9677eae49e5 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.424592 14166 catalog_manager.cc:5526] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:16.425199 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 48aa039bf0364c02abb7f9677eae49e5 (table default.Table_35005559d2f34672bc3e70f6bbfb31d6 [id=d73a4eb552114e20887f9599f244e6ed]) successfully deleted
I20250115 03:06:16.425482 14809 tablet_bootstrap.cc:492] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.426108 14809 tablet_bootstrap.cc:654] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.426914 14811 tablet_bootstrap.cc:492] T 13ce129d8a9245baab7c772f0f5fbaa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.426980 14809 tablet_bootstrap.cc:492] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.426995 14812 tablet_bootstrap.cc:492] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.426996 14811 ts_tablet_manager.cc:1397] T 13ce129d8a9245baab7c772f0f5fbaa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.000s
I20250115 03:06:16.427052 14812 ts_tablet_manager.cc:1397] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:16.427048 14809 ts_tablet_manager.cc:1397] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:16.427414 14812 raft_consensus.cc:357] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.427403 14811 raft_consensus.cc:357] T 13ce129d8a9245baab7c772f0f5fbaa7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.427414 14809 raft_consensus.cc:357] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.427500 14812 raft_consensus.cc:383] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.427507 14809 raft_consensus.cc:383] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.427508 14811 raft_consensus.cc:383] T 13ce129d8a9245baab7c772f0f5fbaa7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.427531 14812 raft_consensus.cc:738] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.427539 14809 raft_consensus.cc:738] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.427543 14811 raft_consensus.cc:738] T 13ce129d8a9245baab7c772f0f5fbaa7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.427635 14812 consensus_queue.cc:260] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.427652 14811 consensus_queue.cc:260] T 13ce129d8a9245baab7c772f0f5fbaa7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.427702 14809 consensus_queue.cc:260] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.427886 14812 ts_tablet_manager.cc:1428] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.427904 14811 ts_tablet_manager.cc:1428] T 13ce129d8a9245baab7c772f0f5fbaa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.428071 14811 tablet_bootstrap.cc:492] T 872ad705e3e94c0eab4397df9f28f9ca P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.428067 14812 tablet_bootstrap.cc:492] T 054251151c294e06905092f2a9306849 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.427944 14809 ts_tablet_manager.cc:1428] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.428969 14812 tablet_bootstrap.cc:654] T 054251151c294e06905092f2a9306849 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.428671 14809 tablet_bootstrap.cc:492] T 3cbc91ad00b54ede804c3629bcfe7fe6 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.429343 14861 raft_consensus.cc:491] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.429414 14861 raft_consensus.cc:513] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.429718 14861 leader_election.cc:290] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.430022 14809 tablet_bootstrap.cc:654] T 3cbc91ad00b54ede804c3629bcfe7fe6 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.430398 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 84ce55f729224f78b420e57977419add with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.430547 14819 tablet_replica.cc:331] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.430645 14819 raft_consensus.cc:2238] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.430729 14819 raft_consensus.cc:2267] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.431097 14819 ts_tablet_manager.cc:1905] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.431644 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 3b5710268643446b91d04e2f4c7f999e (DEFAULT_TABLE table=default.Table_2068156fefae4299b2bada157604f581 [id=dd96ea350b374f4ba87ca09269f92b5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.431877 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b5710268643446b91d04e2f4c7f999e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.432075 14168 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet eeff2cd0825c4419aacfc73ebe267a8e (table default.Table_590b26746c6a4006bb430ba9dcbacafc [id=021f5685630741a8880d2471f7d2d1d2]). Expected version 1 got 0
I20250115 03:06:16.432195 14168 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet df44c5f5a9ae4d1aa7647dcbe2303a39 (table default.Table_098c9884c0494d4592f142b4a197f38c [id=8bf1da8d633940258affd0e11c334786]). Expected version 1 got 0
I20250115 03:06:16.432261 14168 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet f8b90f854a5e4221a24986f8c71ca300 (table default.Table_4e74d34a621f4b1898241f62a20ffc10 [id=d8084de64a264d28a3370971617a694c]). Expected version 2 got 0
W20250115 03:06:16.432879 14358 leader_election.cc:343] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 84ce55f729224f78b420e57977419add
I20250115 03:06:16.433285 14819 ts_tablet_manager.cc:1918] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.433346 14819 log.cc:1198] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/84ce55f729224f78b420e57977419add
I20250115 03:06:16.433529 14819 ts_tablet_manager.cc:1939] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.433743 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84ce55f729224f78b420e57977419add" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.434240 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_90aedfcdb1274e1497cf36620b980768" } modify_external_catalogs: true
I20250115 03:06:16.434335 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_90aedfcdb1274e1497cf36620b980768" } modify_external_catalogs: true
I20250115 03:06:16.435052 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e5c55332e306452685635c28fb3d4c55" } new_table_name: "default.Table_d58c19b62a194b46858ee8955ebf3f82" modify_external_catalogs: true
W20250115 03:06:16.435159 14356 leader_election.cc:343] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.435220 14356 leader_election.cc:304] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
W20250115 03:06:16.435294 14356 raft_consensus.cc:2681] T 84ce55f729224f78b420e57977419add P e8df7247ee9c43e0aa18216da2661f75: unable to run election callback: Service unavailable: Thread pool token was shut down
I20250115 03:06:16.435451 14809 tablet_bootstrap.cc:492] T 3cbc91ad00b54ede804c3629bcfe7fe6 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.435528 14809 ts_tablet_manager.cc:1397] T 3cbc91ad00b54ede804c3629bcfe7fe6 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:16.435598 14811 tablet_bootstrap.cc:654] T 872ad705e3e94c0eab4397df9f28f9ca P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.435918 14809 raft_consensus.cc:357] T 3cbc91ad00b54ede804c3629bcfe7fe6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.436002 14809 raft_consensus.cc:383] T 3cbc91ad00b54ede804c3629bcfe7fe6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.436031 14809 raft_consensus.cc:738] T 3cbc91ad00b54ede804c3629bcfe7fe6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.436133 14809 consensus_queue.cc:260] T 3cbc91ad00b54ede804c3629bcfe7fe6 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.436153 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5be4a6fd260b4cea811c9928d9ac01b1" } modify_external_catalogs: true
I20250115 03:06:16.436237 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5be4a6fd260b4cea811c9928d9ac01b1" } modify_external_catalogs: true
I20250115 03:06:16.436368 14809 ts_tablet_manager.cc:1428] T 3cbc91ad00b54ede804c3629bcfe7fe6 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.439333 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 3b5710268643446b91d04e2f4c7f999e (DEFAULT_TABLE table=default.Table_2068156fefae4299b2bada157604f581 [id=dd96ea350b374f4ba87ca09269f92b5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.439709 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 84ce55f729224f78b420e57977419add (table default.Table_ff21d7bc463d43cfa3330f7288ba7a55 [id=5de066e9f2174fff9303cba23eeec988]) successfully deleted
I20250115 03:06:16.439992 14809 tablet_bootstrap.cc:492] T b1c0f6a5611647eb884c77f7a6d1cd4e P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.440063 14170 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet f70a69acbdc049beb9d98eada09d9eb8 (table default.Table_db8a2673977247e0b14b7995f7e09661 [id=f7e4534c1f6f48f09e10b84c1d5c458c]). Expected version 1 got 0
I20250115 03:06:16.440150 14170 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 304fab45b38e4426ae6961ae786e5659 (table default.Table_d407111658854f02aa8387a067f33f37 [id=526370a40a0d4f7aa96c8348412d8b26]). Expected version 1 got 0
I20250115 03:06:16.440421 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 33380937a3b7428995b06e477a896c02 with delete_type TABLET_DATA_DELETED (replaced by tablet c251960c7c9a44429102a294f35f925c) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.440541 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a0bb9b5093b454d921bf1150d631565 with delete_type TABLET_DATA_DELETED (replaced by tablet 5589b497b0e24740ad49d1392f5a737b) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.440973 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 82be772e05d443749914004aea8af300 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.440984 14809 tablet_bootstrap.cc:654] T b1c0f6a5611647eb884c77f7a6d1cd4e P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.441058 14823 tablet_replica.cc:331] T 33380937a3b7428995b06e477a896c02 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.441159 14823 raft_consensus.cc:2238] T 33380937a3b7428995b06e477a896c02 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.441241 14823 raft_consensus.cc:2267] T 33380937a3b7428995b06e477a896c02 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.441619 14819 tablet_replica.cc:331] T 82be772e05d443749914004aea8af300 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.441720 14819 raft_consensus.cc:2238] T 82be772e05d443749914004aea8af300 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.441790 14819 raft_consensus.cc:2267] T 82be772e05d443749914004aea8af300 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.442099 14823 ts_tablet_manager.cc:1905] T 33380937a3b7428995b06e477a896c02 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.443995 14823 ts_tablet_manager.cc:1918] T 33380937a3b7428995b06e477a896c02 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.444056 14823 log.cc:1198] T 33380937a3b7428995b06e477a896c02 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/33380937a3b7428995b06e477a896c02
I20250115 03:06:16.444224 14823 ts_tablet_manager.cc:1939] T 33380937a3b7428995b06e477a896c02 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.444236 15453 raft_consensus.cc:491] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.444314 15453 raft_consensus.cc:513] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.444645 15453 leader_election.cc:290] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:16.444762 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 33380937a3b7428995b06e477a896c02 (table default.Table_90aedfcdb1274e1497cf36620b980768 [id=9e4f5de45eb24beabef550d15dcad8bf]) successfully deleted
I20250115 03:06:16.444897 14170 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet df44c5f5a9ae4d1aa7647dcbe2303a39 (table default.Table_098c9884c0494d4592f142b4a197f38c [id=8bf1da8d633940258affd0e11c334786]). Expected version 1 got 0
I20250115 03:06:16.445593 14691 raft_consensus.cc:1270] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:16.445612 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c93bfd3da44f9f8a8d585f2d21f1be" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:16.445870 14628 leader_election.cc:343] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.446079 15401 consensus_queue.cc:1035] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.446148 14823 tablet_replica.cc:331] T 3a0bb9b5093b454d921bf1150d631565 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.446236 14823 raft_consensus.cc:2238] T 3a0bb9b5093b454d921bf1150d631565 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.446317 14823 raft_consensus.cc:2267] T 3a0bb9b5093b454d921bf1150d631565 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.446565 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c93bfd3da44f9f8a8d585f2d21f1be" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.447005 14823 ts_tablet_manager.cc:1905] T 3a0bb9b5093b454d921bf1150d631565 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:16.447633 14626 leader_election.cc:343] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.447691 14626 leader_election.cc:304] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.447876 14555 raft_consensus.cc:3054] T 13ce129d8a9245baab7c772f0f5fbaa7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.448117 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b5710268643446b91d04e2f4c7f999e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.448721 14555 raft_consensus.cc:1270] T 13ce129d8a9245baab7c772f0f5fbaa7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:16.450011 15453 raft_consensus.cc:2743] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.450080 14823 ts_tablet_manager.cc:1918] T 3a0bb9b5093b454d921bf1150d631565 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.450170 14823 log.cc:1198] T 3a0bb9b5093b454d921bf1150d631565 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/3a0bb9b5093b454d921bf1150d631565
I20250115 03:06:16.450261 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cbc91ad00b54ede804c3629bcfe7fe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.450501 15401 consensus_queue.cc:1035] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.452118 14823 ts_tablet_manager.cc:1939] T 3a0bb9b5093b454d921bf1150d631565 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.452497 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet 623887ec17374f598f48054e17582f3f from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:16.452746 14687 raft_consensus.cc:1270] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:16.452903 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 3a0bb9b5093b454d921bf1150d631565 (table default.Table_3d196756e234426799a1de651bc112f2 [id=37d965d8a5eb408ea3f33313bd22c152]) successfully deleted
I20250115 03:06:16.453197 14861 consensus_queue.cc:1035] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.453351 14819 ts_tablet_manager.cc:1905] T 82be772e05d443749914004aea8af300 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.453609 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 13ce129d8a9245baab7c772f0f5fbaa7 (table default.Table_d58c19b62a194b46858ee8955ebf3f82 [id=6cba4fb2f7104c6e953340b04e5dc494]). Expected version 3 got 0
I20250115 03:06:16.453866 14556 raft_consensus.cc:1270] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:16.453879 14165 master_service.cc:946] ReplaceTablet: received request to replace tablet bfc34dc76ff94b8eaeb5c2822e6498da from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:16.454653 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d76368b5971a4d08a167c7e7bac8144e" } modify_external_catalogs: true
I20250115 03:06:16.454748 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d76368b5971a4d08a167c7e7bac8144e" } modify_external_catalogs: true
I20250115 03:06:16.455138 14861 consensus_queue.cc:1035] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.455328 14819 ts_tablet_manager.cc:1918] T 82be772e05d443749914004aea8af300 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.455385 14819 log.cc:1198] T 82be772e05d443749914004aea8af300 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/82be772e05d443749914004aea8af300
I20250115 03:06:16.455576 14819 ts_tablet_manager.cc:1939] T 82be772e05d443749914004aea8af300 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.456094 14819 tablet_replica.cc:331] T 3cbc91ad00b54ede804c3629bcfe7fe6 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.456204 14819 raft_consensus.cc:2238] T 3cbc91ad00b54ede804c3629bcfe7fe6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.456341 14861 raft_consensus.cc:491] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.456411 14861 raft_consensus.cc:513] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.456498 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 82be772e05d443749914004aea8af300 (table default.Table_3863dfaf57694ee7b4a9ba1e400d7c9d [id=831e8e5e33ae41368950b0b8fce37e9b]) successfully deleted
I20250115 03:06:16.456645 14819 raft_consensus.cc:2267] T 3cbc91ad00b54ede804c3629bcfe7fe6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.456727 14861 leader_election.cc:290] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.457080 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f70a69acbdc049beb9d98eada09d9eb8" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:16.457201 14689 raft_consensus.cc:2463] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:16.457226 14819 ts_tablet_manager.cc:1905] T 3cbc91ad00b54ede804c3629bcfe7fe6 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.457587 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f70a69acbdc049beb9d98eada09d9eb8" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.457751 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 4ff51d514fb84809aa53fa9fc1010681 (table default.Table_9cc1022af4f14e7a8a8e11cc1803ca87 [id=98f48acd89a6461bb4511141cab4d88b]). Expected version 2 got 0
I20250115 03:06:16.457957 14358 leader_election.cc:304] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
W20250115 03:06:16.458122 14356 leader_election.cc:343] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.458273 14861 raft_consensus.cc:2798] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:16.458279 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet c251960c7c9a44429102a294f35f925c
I20250115 03:06:16.458317 14861 raft_consensus.cc:491] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:16.458338 14169 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet 969f56f426484933af17e9673f777515
I20250115 03:06:16.458353 14861 raft_consensus.cc:3054] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.458895 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet c251960c7c9a44429102a294f35f925c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.459178 14861 raft_consensus.cc:513] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.459407 14819 ts_tablet_manager.cc:1918] T 3cbc91ad00b54ede804c3629bcfe7fe6 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.459462 14819 log.cc:1198] T 3cbc91ad00b54ede804c3629bcfe7fe6 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/3cbc91ad00b54ede804c3629bcfe7fe6
I20250115 03:06:16.459484 14861 leader_election.cc:290] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.459712 14819 ts_tablet_manager.cc:1939] T 3cbc91ad00b54ede804c3629bcfe7fe6 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.459702 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet c251960c7c9a44429102a294f35f925c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.459818 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet c251960c7c9a44429102a294f35f925c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.459966 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f70a69acbdc049beb9d98eada09d9eb8" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:16.460062 14689 raft_consensus.cc:3054] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.460258 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 3cbc91ad00b54ede804c3629bcfe7fe6 (table default.Table_271f46df55f14ffd9b7bb48352620543 [id=44488be7335b488a9ff7cf913ce4273a]) successfully deleted
W20250115 03:06:16.460408 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet c251960c7c9a44429102a294f35f925c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c251960c7c9a44429102a294f35f925c already in progress: creating tablet
W20250115 03:06:16.460536 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet c251960c7c9a44429102a294f35f925c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c251960c7c9a44429102a294f35f925c already in progress: creating tablet
I20250115 03:06:16.460953 14689 raft_consensus.cc:2463] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:16.461783 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 13ce129d8a9245baab7c772f0f5fbaa7 (table default.Table_d58c19b62a194b46858ee8955ebf3f82 [id=6cba4fb2f7104c6e953340b04e5dc494]). Expected version 3 got 0
I20250115 03:06:16.461894 14358 leader_election.cc:304] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.462044 14861 raft_consensus.cc:2798] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:16.462641 15617 tablet.cc:1721] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:16.462882 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 623887ec17374f598f48054e17582f3f
I20250115 03:06:16.462903 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f70a69acbdc049beb9d98eada09d9eb8" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:16.463124 14171 catalog_manager.cc:6596] ReplaceTablet: tablet 623887ec17374f598f48054e17582f3f deleted and replaced by tablet 12678da7ffee4a46b4690ac3e94cb19b
I20250115 03:06:16.463119 15614 mvcc.cc:204] Tried to move back new op lower bound from 7114384901917569024 to 7114384901229846528. Current Snapshot: MvccSnapshot[applied={T|T < 7114384901917569024}]
I20250115 03:06:16.463434 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 623887ec17374f598f48054e17582f3f with delete_type TABLET_DATA_DELETED (replaced by tablet 12678da7ffee4a46b4690ac3e94cb19b) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.463639 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 623887ec17374f598f48054e17582f3f with delete_type TABLET_DATA_DELETED (replaced by tablet 12678da7ffee4a46b4690ac3e94cb19b) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:16.463922 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet c251960c7c9a44429102a294f35f925c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c251960c7c9a44429102a294f35f925c already in progress: creating tablet
W20250115 03:06:16.464066 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 623887ec17374f598f48054e17582f3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 623887ec17374f598f48054e17582f3f already in progress: creating tablet
I20250115 03:06:16.464164 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 623887ec17374f598f48054e17582f3f with delete_type TABLET_DATA_DELETED (replaced by tablet 12678da7ffee4a46b4690ac3e94cb19b) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.464663 14171 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 13ce129d8a9245baab7c772f0f5fbaa7 (table default.Table_d58c19b62a194b46858ee8955ebf3f82 [id=6cba4fb2f7104c6e953340b04e5dc494]). Expected version 3 got 0
I20250115 03:06:16.464757 14171 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 4ff51d514fb84809aa53fa9fc1010681 (table default.Table_9cc1022af4f14e7a8a8e11cc1803ca87 [id=98f48acd89a6461bb4511141cab4d88b]). Expected version 2 got 0
W20250115 03:06:16.464867 14356 leader_election.cc:343] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.465286 15274 tablet.cc:1721] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20250115 03:06:16.467422 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 623887ec17374f598f48054e17582f3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 623887ec17374f598f48054e17582f3f already in progress: creating tablet
I20250115 03:06:16.467519 15426 tablet.cc:1721] T 13ce129d8a9245baab7c772f0f5fbaa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:16.468909 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 3b5710268643446b91d04e2f4c7f999e (DEFAULT_TABLE table=default.Table_2068156fefae4299b2bada157604f581 [id=dd96ea350b374f4ba87ca09269f92b5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.469184 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b5710268643446b91d04e2f4c7f999e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.469197 14861 raft_consensus.cc:695] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:16.469255 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 3bc1f0f72a3b4c79847c0a91fe98ccd1 (DEFAULT_TABLE table=default.Table_ff993b0824114ebdb66d40758eb3b827 [id=dfe0ca35843c4fc6b975dd1ab87e9a99]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.469314 14861 consensus_queue.cc:237] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.469466 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bc1f0f72a3b4c79847c0a91fe98ccd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.472141 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 3bc1f0f72a3b4c79847c0a91fe98ccd1 (DEFAULT_TABLE table=default.Table_ff993b0824114ebdb66d40758eb3b827 [id=dfe0ca35843c4fc6b975dd1ab87e9a99]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.472141 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 676ec8b9dc8241d4ae40488d3ce8f61e (DEFAULT_TABLE table=default.Table_09ddb549f5424be289e1ff6bdefbb152 [id=63edd09bfaaa4d56b4314ce760c87d3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.472251 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 3bc1f0f72a3b4c79847c0a91fe98ccd1 (DEFAULT_TABLE table=default.Table_ff993b0824114ebdb66d40758eb3b827 [id=dfe0ca35843c4fc6b975dd1ab87e9a99]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.472350 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bc1f0f72a3b4c79847c0a91fe98ccd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.472350 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676ec8b9dc8241d4ae40488d3ce8f61e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.472452 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bc1f0f72a3b4c79847c0a91fe98ccd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.473862 15617 tablet.cc:1721] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:16.474349 14531 tablet_service.cc:1467] Processing CreateTablet for tablet 676ec8b9dc8241d4ae40488d3ce8f61e (DEFAULT_TABLE table=default.Table_09ddb549f5424be289e1ff6bdefbb152 [id=63edd09bfaaa4d56b4314ce760c87d3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.474575 14531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676ec8b9dc8241d4ae40488d3ce8f61e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.475988 14534 tablet_service.cc:1467] Processing CreateTablet for tablet 1016cf85491d4ceda132ee57325b48a4 (DEFAULT_TABLE table=default.Table_2e115074bf7049bfbb1239b4d9c1ca2e [id=6cf622de00cb490bb794837223aa7bcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.476189 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1016cf85491d4ceda132ee57325b48a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.477444 15451 mvcc.cc:204] Tried to move back new op lower bound from 7114384901946978304 to 7114384901094572032. Current Snapshot: MvccSnapshot[applied={T|T < 7114384901946978304}]
W20250115 03:06:16.478101 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 623887ec17374f598f48054e17582f3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 623887ec17374f598f48054e17582f3f already in progress: creating tablet
I20250115 03:06:16.478365 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 676ec8b9dc8241d4ae40488d3ce8f61e (DEFAULT_TABLE table=default.Table_09ddb549f5424be289e1ff6bdefbb152 [id=63edd09bfaaa4d56b4314ce760c87d3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.478462 14395 tablet_service.cc:1467] Processing CreateTablet for tablet b8ec24d2937d44e79b839276bf615fa0 (DEFAULT_TABLE table=default.Table_1df66a4877bb4ba993ae442eb173bf82 [id=7bec6149db914c939bc6fd50e47097b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.478585 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676ec8b9dc8241d4ae40488d3ce8f61e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.478757 14395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8ec24d2937d44e79b839276bf615fa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.478799 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 1016cf85491d4ceda132ee57325b48a4 (DEFAULT_TABLE table=default.Table_2e115074bf7049bfbb1239b4d9c1ca2e [id=6cf622de00cb490bb794837223aa7bcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.478971 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1016cf85491d4ceda132ee57325b48a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.480912 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet bfc34dc76ff94b8eaeb5c2822e6498da
I20250115 03:06:16.481230 14165 catalog_manager.cc:6596] ReplaceTablet: tablet bfc34dc76ff94b8eaeb5c2822e6498da deleted and replaced by tablet f2ca06c402cd46dc8311c8a47da92846
I20250115 03:06:16.481266 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 69533c30b57e45dab0b00239dfd90f13
I20250115 03:06:16.481375 15426 tablet.cc:1721] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:16.481735 14527 tablet_service.cc:1467] Processing CreateTablet for tablet b8ec24d2937d44e79b839276bf615fa0 (DEFAULT_TABLE table=default.Table_1df66a4877bb4ba993ae442eb173bf82 [id=7bec6149db914c939bc6fd50e47097b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.481839 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet bfc34dc76ff94b8eaeb5c2822e6498da with delete_type TABLET_DATA_DELETED (replaced by tablet f2ca06c402cd46dc8311c8a47da92846) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.481932 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8ec24d2937d44e79b839276bf615fa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.482488 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet bfc34dc76ff94b8eaeb5c2822e6498da with delete_type TABLET_DATA_DELETED (replaced by tablet f2ca06c402cd46dc8311c8a47da92846) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.482564 14395 tablet_service.cc:1514] Processing DeleteTablet for tablet 69533c30b57e45dab0b00239dfd90f13 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:16.482690 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet bfc34dc76ff94b8eaeb5c2822e6498da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bfc34dc76ff94b8eaeb5c2822e6498da already in progress: creating tablet
I20250115 03:06:16.482798 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet bfc34dc76ff94b8eaeb5c2822e6498da with delete_type TABLET_DATA_DELETED (replaced by tablet f2ca06c402cd46dc8311c8a47da92846) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.483273 14660 tablet_service.cc:1467] Processing CreateTablet for tablet b8ec24d2937d44e79b839276bf615fa0 (DEFAULT_TABLE table=default.Table_1df66a4877bb4ba993ae442eb173bf82 [id=7bec6149db914c939bc6fd50e47097b2]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:16.483310 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet bfc34dc76ff94b8eaeb5c2822e6498da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bfc34dc76ff94b8eaeb5c2822e6498da already in progress: creating tablet
W20250115 03:06:16.483448 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 69533c30b57e45dab0b00239dfd90f13 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 69533c30b57e45dab0b00239dfd90f13 already in progress: creating tablet
I20250115 03:06:16.483500 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8ec24d2937d44e79b839276bf615fa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.484042 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 69533c30b57e45dab0b00239dfd90f13 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:16.484445 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 69533c30b57e45dab0b00239dfd90f13 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 69533c30b57e45dab0b00239dfd90f13 already in progress: creating tablet
I20250115 03:06:16.484764 14531 tablet_service.cc:1514] Processing DeleteTablet for tablet 69533c30b57e45dab0b00239dfd90f13 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.485628 14170 catalog_manager.cc:5526] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:16.485787 14170 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet f70a69acbdc049beb9d98eada09d9eb8 (table default.Table_db8a2673977247e0b14b7995f7e09661 [id=f7e4534c1f6f48f09e10b84c1d5c458c]). Expected version 1 got 0
I20250115 03:06:16.486003 14171 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_d58c19b62a194b46858ee8955ebf3f82 [id=6cba4fb2f7104c6e953340b04e5dc494] alter complete (version 3)
I20250115 03:06:16.486901 15453 raft_consensus.cc:491] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.486975 15453 raft_consensus.cc:513] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.487282 15453 leader_election.cc:290] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:16.487898 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "341b24d9b63f4761b2013035023fcd98" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:16.488265 14628 leader_election.cc:343] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.488605 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 85cceb5a3f234d85937ad0bb75348991 (DEFAULT_TABLE table=default.Table_eec974cf89de4f45930dbf47fa754f2a [id=36660de3890e411e974c41b7f8f85946]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.488802 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85cceb5a3f234d85937ad0bb75348991. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.488847 14395 tablet_service.cc:1467] Processing CreateTablet for tablet 85cceb5a3f234d85937ad0bb75348991 (DEFAULT_TABLE table=default.Table_eec974cf89de4f45930dbf47fa754f2a [id=36660de3890e411e974c41b7f8f85946]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.489094 14395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85cceb5a3f234d85937ad0bb75348991. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:16.489378 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet bfc34dc76ff94b8eaeb5c2822e6498da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bfc34dc76ff94b8eaeb5c2822e6498da already in progress: creating tablet
W20250115 03:06:16.489506 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 69533c30b57e45dab0b00239dfd90f13 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 69533c30b57e45dab0b00239dfd90f13 already in progress: creating tablet
I20250115 03:06:16.490653 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "341b24d9b63f4761b2013035023fcd98" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.491499 14531 tablet_service.cc:1467] Processing CreateTablet for tablet 85cceb5a3f234d85937ad0bb75348991 (DEFAULT_TABLE table=default.Table_eec974cf89de4f45930dbf47fa754f2a [id=36660de3890e411e974c41b7f8f85946]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.491705 14531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85cceb5a3f234d85937ad0bb75348991. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.491885 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 80ede4e9696d491aa385b375fcb5cf3b (DEFAULT_TABLE table=default.Table_75cd7b01fc534652975eb593912b88bc [id=111fc67094164d3f99965c246ae12b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.492075 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80ede4e9696d491aa385b375fcb5cf3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.492336 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_4f083ad1bef14359a5e1713d9dabbdca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.492645 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f083ad1bef14359a5e1713d9dabbdca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:16.493935 14626 leader_election.cc:343] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.494004 14626 leader_election.cc:304] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.494128 15453 raft_consensus.cc:2743] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.494912 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e8a58d3776bc440b90fe3d310db9b131"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.495173 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e8a58d3776bc440b90fe3d310db9b131 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.496488 14168 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_9cc1022af4f14e7a8a8e11cc1803ca87 [id=98f48acd89a6461bb4511141cab4d88b] alter complete (version 2)
I20250115 03:06:16.497205 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 80ede4e9696d491aa385b375fcb5cf3b (DEFAULT_TABLE table=default.Table_75cd7b01fc534652975eb593912b88bc [id=111fc67094164d3f99965c246ae12b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.497406 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80ede4e9696d491aa385b375fcb5cf3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.497681 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 1016cf85491d4ceda132ee57325b48a4 (DEFAULT_TABLE table=default.Table_2e115074bf7049bfbb1239b4d9c1ca2e [id=6cf622de00cb490bb794837223aa7bcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.497890 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1016cf85491d4ceda132ee57325b48a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.499504 15274 tablet.cc:1721] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:16.499933 14531 tablet_service.cc:1467] Processing CreateTablet for tablet 80ede4e9696d491aa385b375fcb5cf3b (DEFAULT_TABLE table=default.Table_75cd7b01fc534652975eb593912b88bc [id=111fc67094164d3f99965c246ae12b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.500133 14531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80ede4e9696d491aa385b375fcb5cf3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.507767 14812 tablet_bootstrap.cc:492] T 054251151c294e06905092f2a9306849 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.507855 14812 ts_tablet_manager.cc:1397] T 054251151c294e06905092f2a9306849 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.080s user 0.002s sys 0.000s
I20250115 03:06:16.507977 14811 tablet_bootstrap.cc:492] T 872ad705e3e94c0eab4397df9f28f9ca P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.508069 14811 ts_tablet_manager.cc:1397] T 872ad705e3e94c0eab4397df9f28f9ca P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.080s user 0.001s sys 0.000s
I20250115 03:06:16.508293 14812 raft_consensus.cc:357] T 054251151c294e06905092f2a9306849 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.508390 14812 raft_consensus.cc:383] T 054251151c294e06905092f2a9306849 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.508424 14812 raft_consensus.cc:738] T 054251151c294e06905092f2a9306849 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.508528 14812 consensus_queue.cc:260] T 054251151c294e06905092f2a9306849 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.508546 14811 raft_consensus.cc:357] T 872ad705e3e94c0eab4397df9f28f9ca P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.508654 14811 raft_consensus.cc:383] T 872ad705e3e94c0eab4397df9f28f9ca P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.508684 14811 raft_consensus.cc:738] T 872ad705e3e94c0eab4397df9f28f9ca P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.508775 14812 ts_tablet_manager.cc:1428] T 054251151c294e06905092f2a9306849 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.508798 14811 consensus_queue.cc:260] T 872ad705e3e94c0eab4397df9f28f9ca P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.508956 14812 tablet_bootstrap.cc:492] T b0a5d554764249f8a7eefdc943271090 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.509097 14811 ts_tablet_manager.cc:1428] T 872ad705e3e94c0eab4397df9f28f9ca P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.509235 14811 tablet_bootstrap.cc:492] T 79579515fb7e45a8b16fd36dbc6a965a P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.510103 14811 tablet_bootstrap.cc:654] T 79579515fb7e45a8b16fd36dbc6a965a P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.510236 14812 tablet_bootstrap.cc:654] T b0a5d554764249f8a7eefdc943271090 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.511099 14812 tablet_bootstrap.cc:492] T b0a5d554764249f8a7eefdc943271090 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.511381 14812 ts_tablet_manager.cc:1397] T b0a5d554764249f8a7eefdc943271090 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:16.512014 14812 raft_consensus.cc:357] T b0a5d554764249f8a7eefdc943271090 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.512110 14812 raft_consensus.cc:383] T b0a5d554764249f8a7eefdc943271090 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.512147 14812 raft_consensus.cc:738] T b0a5d554764249f8a7eefdc943271090 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.512255 14812 consensus_queue.cc:260] T b0a5d554764249f8a7eefdc943271090 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.512763 14812 ts_tablet_manager.cc:1428] T b0a5d554764249f8a7eefdc943271090 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.513500 14811 tablet_bootstrap.cc:492] T 79579515fb7e45a8b16fd36dbc6a965a P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.513569 14811 ts_tablet_manager.cc:1397] T 79579515fb7e45a8b16fd36dbc6a965a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250115 03:06:16.513906 14811 raft_consensus.cc:357] T 79579515fb7e45a8b16fd36dbc6a965a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.513979 14811 raft_consensus.cc:383] T 79579515fb7e45a8b16fd36dbc6a965a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.514003 14811 raft_consensus.cc:738] T 79579515fb7e45a8b16fd36dbc6a965a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.514089 14811 consensus_queue.cc:260] T 79579515fb7e45a8b16fd36dbc6a965a P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.514312 14811 ts_tablet_manager.cc:1428] T 79579515fb7e45a8b16fd36dbc6a965a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.514447 14811 tablet_bootstrap.cc:492] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.514793 14812 tablet_bootstrap.cc:492] T a75b9401cb4b4f8b92a4779c3704eb46 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.515434 14811 tablet_bootstrap.cc:654] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.517048 14812 tablet_bootstrap.cc:654] T a75b9401cb4b4f8b92a4779c3704eb46 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.517895 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet b0a5d554764249f8a7eefdc943271090 with delete_type TABLET_DATA_DELETED (replaced by tablet d895f75911b7456a88e7a5159db7d60b) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.518093 14823 tablet_replica.cc:331] T b0a5d554764249f8a7eefdc943271090 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.518218 14823 raft_consensus.cc:2238] T b0a5d554764249f8a7eefdc943271090 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.518306 14823 raft_consensus.cc:2267] T b0a5d554764249f8a7eefdc943271090 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.518431 14811 tablet_bootstrap.cc:492] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.518502 14811 ts_tablet_manager.cc:1397] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250115 03:06:16.518883 14811 raft_consensus.cc:357] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.518970 14811 raft_consensus.cc:383] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.519001 14811 raft_consensus.cc:738] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.519014 14823 ts_tablet_manager.cc:1905] T b0a5d554764249f8a7eefdc943271090 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.519091 14811 consensus_queue.cc:260] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.519142 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3d196756e234426799a1de651bc112f2" } new_table_name: "default.Table_8daa0abab4d1464488bec21bb7ad375b" modify_external_catalogs: true
I20250115 03:06:16.519340 14811 ts_tablet_manager.cc:1428] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.519492 14811 tablet_bootstrap.cc:492] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.520032 14809 tablet_bootstrap.cc:492] T b1c0f6a5611647eb884c77f7a6d1cd4e P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.520105 14809 ts_tablet_manager.cc:1397] T b1c0f6a5611647eb884c77f7a6d1cd4e P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.080s user 0.002s sys 0.000s
I20250115 03:06:16.520323 14811 tablet_bootstrap.cc:654] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.520515 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_204aaf913d224b93adf8ce86999572c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:16.520541 14809 raft_consensus.cc:357] T b1c0f6a5611647eb884c77f7a6d1cd4e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.520639 14809 raft_consensus.cc:383] T b1c0f6a5611647eb884c77f7a6d1cd4e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.520668 14809 raft_consensus.cc:738] T b1c0f6a5611647eb884c77f7a6d1cd4e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.520774 14809 consensus_queue.cc:260] T b1c0f6a5611647eb884c77f7a6d1cd4e P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:16.520839 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_204aaf913d224b93adf8ce86999572c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.521023 14823 ts_tablet_manager.cc:1918] T b0a5d554764249f8a7eefdc943271090 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.521075 14823 log.cc:1198] T b0a5d554764249f8a7eefdc943271090 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/b0a5d554764249f8a7eefdc943271090
I20250115 03:06:16.521064 14809 ts_tablet_manager.cc:1428] T b1c0f6a5611647eb884c77f7a6d1cd4e P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.521248 14823 ts_tablet_manager.cc:1939] T b0a5d554764249f8a7eefdc943271090 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.521255 14809 tablet_bootstrap.cc:492] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.521792 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet b0a5d554764249f8a7eefdc943271090 (table default.Table_39e0d37b84fc42c0807a4cbffb7bae0a [id=4b6ce1381c024498a78d91e346f488f2]) successfully deleted
I20250115 03:06:16.521636 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 054251151c294e06905092f2a9306849 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.522142 14809 tablet_bootstrap.cc:654] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.522171 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_27b5acf9bde44139b1b076e9579d88c7" } new_table_name: "default.Table_4464f446819641ebab4c3f23f0632251" modify_external_catalogs: true
I20250115 03:06:16.522279 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet b0a5d554764249f8a7eefdc943271090 with delete_type TABLET_DATA_DELETED (replaced by tablet d895f75911b7456a88e7a5159db7d60b) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.523178 14823 tablet_replica.cc:331] T 054251151c294e06905092f2a9306849 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.523283 14823 raft_consensus.cc:2238] T 054251151c294e06905092f2a9306849 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.523360 14823 raft_consensus.cc:2267] T 054251151c294e06905092f2a9306849 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.524711 14687 raft_consensus.cc:1270] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:16.525198 14885 consensus_queue.cc:1035] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.525274 14823 ts_tablet_manager.cc:1905] T 054251151c294e06905092f2a9306849 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.527304 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_51d58281d1844f48b14d851ff78eafe9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.527424 14356 consensus_peers.cc:487] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20250115 03:06:16.527604 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_51d58281d1844f48b14d851ff78eafe9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.528183 14812 tablet_bootstrap.cc:492] T a75b9401cb4b4f8b92a4779c3704eb46 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.528249 14812 ts_tablet_manager.cc:1397] T a75b9401cb4b4f8b92a4779c3704eb46 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.001s
I20250115 03:06:16.528626 14812 raft_consensus.cc:357] T a75b9401cb4b4f8b92a4779c3704eb46 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.528713 14812 raft_consensus.cc:383] T a75b9401cb4b4f8b92a4779c3704eb46 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.528755 14812 raft_consensus.cc:738] T a75b9401cb4b4f8b92a4779c3704eb46 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
W20250115 03:06:16.528810 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5589b497b0e24740ad49d1392f5a737b consensus state has no leader
I20250115 03:06:16.528864 14823 ts_tablet_manager.cc:1918] T 054251151c294e06905092f2a9306849 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.529050 14823 log.cc:1198] T 054251151c294e06905092f2a9306849 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/054251151c294e06905092f2a9306849
I20250115 03:06:16.529129 14812 consensus_queue.cc:260] T a75b9401cb4b4f8b92a4779c3704eb46 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.529337 14823 ts_tablet_manager.cc:1939] T 054251151c294e06905092f2a9306849 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
W20250115 03:06:16.529443 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9febec144bb343a8a21817e6d3655ad7 consensus state has no leader
I20250115 03:06:16.529577 14812 ts_tablet_manager.cc:1428] T a75b9401cb4b4f8b92a4779c3704eb46 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.529721 14812 tablet_bootstrap.cc:492] T 90277f7d279e4c7a83c43f40766cd11b P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.530063 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 054251151c294e06905092f2a9306849 (table default.Table_92b7f10774fe4db0ac9872d33fceb7a7 [id=54fd71a4b30d41b7b59b8dfc261a616d]) successfully deleted
W20250115 03:06:16.530210 14158 catalog_manager.cc:4839] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet b0a5d554764249f8a7eefdc943271090 because the tablet was not found. No further retry: Not found: Tablet not found: b0a5d554764249f8a7eefdc943271090
I20250115 03:06:16.531317 14812 tablet_bootstrap.cc:654] T 90277f7d279e4c7a83c43f40766cd11b P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.531814 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_aa81abdb583a417ca417be1d66854e8f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.532110 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa81abdb583a417ca417be1d66854e8f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.533345 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_adff4ab8d6594baba1f08a15bf8486eb" } new_table_name: "default.Table_b94f9ef5ede64627b994e94e674b1cfa" modify_external_catalogs: true
I20250115 03:06:16.534400 15617 tablet.cc:1721] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:16.536171 14811 tablet_bootstrap.cc:492] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.536250 14811 ts_tablet_manager.cc:1397] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20250115 03:06:16.536670 15274 tablet.cc:1721] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:16.537597 14809 tablet_bootstrap.cc:492] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.537698 14809 ts_tablet_manager.cc:1397] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20250115 03:06:16.538427 14809 raft_consensus.cc:357] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.538530 14809 raft_consensus.cc:383] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.538565 14809 raft_consensus.cc:738] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.538669 14809 consensus_queue.cc:260] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.538944 14809 ts_tablet_manager.cc:1428] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.539098 14809 tablet_bootstrap.cc:492] T 506198162e0b4e24924c408542707c4c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.539076 14811 raft_consensus.cc:357] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.539165 14811 raft_consensus.cc:383] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.539192 14811 raft_consensus.cc:738] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.539292 14811 consensus_queue.cc:260] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.539531 14811 ts_tablet_manager.cc:1428] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:16.539685 14811 tablet_bootstrap.cc:492] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.540563 14811 tablet_bootstrap.cc:654] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.541985 14809 tablet_bootstrap.cc:654] T 506198162e0b4e24924c408542707c4c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.544641 14809 tablet_bootstrap.cc:492] T 506198162e0b4e24924c408542707c4c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.544718 14809 ts_tablet_manager.cc:1397] T 506198162e0b4e24924c408542707c4c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250115 03:06:16.545341 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet a75b9401cb4b4f8b92a4779c3704eb46 with delete_type TABLET_DATA_DELETED (replaced by tablet 21524aef80f74a6c899ea66b9b022389) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.545229 14809 raft_consensus.cc:357] T 506198162e0b4e24924c408542707c4c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.545497 14823 tablet_replica.cc:331] T a75b9401cb4b4f8b92a4779c3704eb46 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.545578 14809 raft_consensus.cc:383] T 506198162e0b4e24924c408542707c4c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.545609 14823 raft_consensus.cc:2238] T a75b9401cb4b4f8b92a4779c3704eb46 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.545683 14809 raft_consensus.cc:738] T 506198162e0b4e24924c408542707c4c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.545703 14823 raft_consensus.cc:2267] T a75b9401cb4b4f8b92a4779c3704eb46 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.546454 14809 consensus_queue.cc:260] T 506198162e0b4e24924c408542707c4c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.547799 15645 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250115 03:06:16.547988 14823 ts_tablet_manager.cc:1905] T a75b9401cb4b4f8b92a4779c3704eb46 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.548534 15645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250115 03:06:16.548688 15645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250115 03:06:16.549952 14823 ts_tablet_manager.cc:1918] T a75b9401cb4b4f8b92a4779c3704eb46 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.550009 14823 log.cc:1198] T a75b9401cb4b4f8b92a4779c3704eb46 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/a75b9401cb4b4f8b92a4779c3704eb46
I20250115 03:06:16.550189 14823 ts_tablet_manager.cc:1939] T a75b9401cb4b4f8b92a4779c3704eb46 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.550768 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet a75b9401cb4b4f8b92a4779c3704eb46 (table default.Table_a0c249d69b904390952a780b10e1369b [id=22d05297fb9545aeab39b5b62248d0d4]) successfully deleted
I20250115 03:06:16.551126 14809 ts_tablet_manager.cc:1428] T 506198162e0b4e24924c408542707c4c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:16.551474 14809 tablet_bootstrap.cc:492] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.552446 14809 tablet_bootstrap.cc:654] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.553361 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_9194fcb0d915415ea9220cc06cb2a37d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:16.553659 14811 tablet_bootstrap.cc:492] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
W20250115 03:06:16.553682 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9194fcb0d915415ea9220cc06cb2a37d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.553747 14811 ts_tablet_manager.cc:1397] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20250115 03:06:16.554201 14811 raft_consensus.cc:357] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.554294 14811 raft_consensus.cc:383] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.554322 14811 raft_consensus.cc:738] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.554422 14811 consensus_queue.cc:260] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.554661 14811 ts_tablet_manager.cc:1428] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:16.554840 14811 tablet_bootstrap.cc:492] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.555121 14809 tablet_bootstrap.cc:492] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.555197 14809 ts_tablet_manager.cc:1397] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:16.555797 14811 tablet_bootstrap.cc:654] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.556757 14809 raft_consensus.cc:357] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.556852 14809 raft_consensus.cc:383] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.556882 14809 raft_consensus.cc:738] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.557046 14812 tablet_bootstrap.cc:492] T 90277f7d279e4c7a83c43f40766cd11b P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.557114 14812 ts_tablet_manager.cc:1397] T 90277f7d279e4c7a83c43f40766cd11b P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20250115 03:06:16.557530 14812 raft_consensus.cc:357] T 90277f7d279e4c7a83c43f40766cd11b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.557632 14812 raft_consensus.cc:383] T 90277f7d279e4c7a83c43f40766cd11b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.557663 14812 raft_consensus.cc:738] T 90277f7d279e4c7a83c43f40766cd11b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.557792 14812 consensus_queue.cc:260] T 90277f7d279e4c7a83c43f40766cd11b P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:16.557888 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 56c600fe18504e05bab2bcde3c89861d consensus state has no leader
I20250115 03:06:16.558053 14809 consensus_queue.cc:260] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.558646 14809 ts_tablet_manager.cc:1428] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20250115 03:06:16.558801 14809 tablet_bootstrap.cc:492] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.559837 14812 ts_tablet_manager.cc:1428] T 90277f7d279e4c7a83c43f40766cd11b P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:16.560107 14812 tablet_bootstrap.cc:492] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.562547 14812 tablet_bootstrap.cc:654] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.564154 14809 tablet_bootstrap.cc:654] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.564957 14811 tablet_bootstrap.cc:492] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.564961 14809 tablet_bootstrap.cc:492] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.565032 14809 ts_tablet_manager.cc:1397] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20250115 03:06:16.565032 14811 ts_tablet_manager.cc:1397] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20250115 03:06:16.565395 14811 raft_consensus.cc:357] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.565395 14809 raft_consensus.cc:357] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.565479 14809 raft_consensus.cc:383] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.565479 14811 raft_consensus.cc:383] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.565508 14809 raft_consensus.cc:738] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.565521 14811 raft_consensus.cc:738] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.565622 14811 consensus_queue.cc:260] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.565630 14809 consensus_queue.cc:260] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.565883 14811 ts_tablet_manager.cc:1428] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:16.566022 14809 ts_tablet_manager.cc:1428] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.566040 14811 tablet_bootstrap.cc:492] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.566159 14809 tablet_bootstrap.cc:492] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.568192 14809 tablet_bootstrap.cc:654] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.568297 14811 tablet_bootstrap.cc:654] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.568763 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet 420b7c8d43a04bae972eed2014d5634c from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:16.569195 14809 tablet_bootstrap.cc:492] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.569278 14809 ts_tablet_manager.cc:1397] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:16.569774 14809 raft_consensus.cc:357] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.569890 14809 raft_consensus.cc:383] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.569919 14809 raft_consensus.cc:738] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.570008 14809 consensus_queue.cc:260] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.570238 14809 ts_tablet_manager.cc:1428] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.570376 14809 tablet_bootstrap.cc:492] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.571182 14809 tablet_bootstrap.cc:654] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.571377 14858 raft_consensus.cc:491] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.571452 14858 raft_consensus.cc:513] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.571766 14858 leader_election.cc:290] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.572250 15645 rpc_server.cc:307] RPC server started. Bound to: 127.13.186.252:41855
W20250115 03:06:16.572371 14493 leader_election.cc:343] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 7f67cbb929cd4ae589a14606aa168b46
I20250115 03:06:16.574209 14811 tablet_bootstrap.cc:492] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.574209 14812 tablet_bootstrap.cc:492] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.574281 14812 ts_tablet_manager.cc:1397] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20250115 03:06:16.574281 14811 ts_tablet_manager.cc:1397] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20250115 03:06:16.574663 14812 raft_consensus.cc:357] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.574681 14811 raft_consensus.cc:357] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.574751 14812 raft_consensus.cc:383] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.574764 14811 raft_consensus.cc:383] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.574780 14812 raft_consensus.cc:738] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.574792 14811 raft_consensus.cc:738] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.574882 14811 consensus_queue.cc:260] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.574882 14812 consensus_queue.cc:260] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.575132 14812 ts_tablet_manager.cc:1428] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.575131 14811 ts_tablet_manager.cc:1428] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.575287 14812 tablet_bootstrap.cc:492] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.575873 15645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-2/data/info.pb
I20250115 03:06:16.576123 14812 tablet_bootstrap.cc:654] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.576455 14811 tablet_bootstrap.cc:492] T e4f688e4c40d4653929432b13508348a P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
W20250115 03:06:16.577451 14493 leader_election.cc:343] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 7f67cbb929cd4ae589a14606aa168b46
I20250115 03:06:16.577520 14493 leader_election.cc:304] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.577965 14853 raft_consensus.cc:2743] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.578058 15453 raft_consensus.cc:491] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.578123 15453 raft_consensus.cc:513] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.578383 15453 leader_election.cc:290] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:16.578691 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6864f88bf6454ee980f624ec2f2d9fb6" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:16.577519 14811 tablet_bootstrap.cc:654] T e4f688e4c40d4653929432b13508348a P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.578800 14421 raft_consensus.cc:2463] T 6864f88bf6454ee980f624ec2f2d9fb6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:16.579773 14809 tablet_bootstrap.cc:492] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.579845 14809 ts_tablet_manager.cc:1397] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20250115 03:06:16.580121 14628 leader_election.cc:304] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.580226 14809 raft_consensus.cc:357] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.580317 14809 raft_consensus.cc:383] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.580351 14809 raft_consensus.cc:738] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.580456 14809 consensus_queue.cc:260] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.580727 14809 ts_tablet_manager.cc:1428] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.580762 15453 raft_consensus.cc:2798] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:16.580811 15453 raft_consensus.cc:491] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:16.580842 15453 raft_consensus.cc:3054] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.580894 14809 tablet_bootstrap.cc:492] T 420b7c8d43a04bae972eed2014d5634c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.581722 14809 tablet_bootstrap.cc:654] T 420b7c8d43a04bae972eed2014d5634c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.581758 15453 raft_consensus.cc:513] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.582038 15453 leader_election.cc:290] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:16.582485 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6864f88bf6454ee980f624ec2f2d9fb6" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:16.582576 14421 raft_consensus.cc:3054] T 6864f88bf6454ee980f624ec2f2d9fb6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.582940 14811 tablet_bootstrap.cc:492] T e4f688e4c40d4653929432b13508348a P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.583412 14421 raft_consensus.cc:2463] T 6864f88bf6454ee980f624ec2f2d9fb6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:16.583746 14628 leader_election.cc:304] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.583866 14059 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu as pid 15645
I20250115 03:06:16.583004 14811 ts_tablet_manager.cc:1397] T e4f688e4c40d4653929432b13508348a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:16.584467 14858 raft_consensus.cc:491] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.584546 14858 raft_consensus.cc:513] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.584829 14858 leader_election.cc:290] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.584913 15453 raft_consensus.cc:2798] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:16.584981 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6864f88bf6454ee980f624ec2f2d9fb6" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.585012 15453 raft_consensus.cc:695] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:16.585115 15453 consensus_queue.cc:237] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.585186 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6864f88bf6454ee980f624ec2f2d9fb6" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:16.585371 15833 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.186.252:41855 every 8 connection(s)
I20250115 03:06:16.585796 14811 raft_consensus.cc:357] T e4f688e4c40d4653929432b13508348a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.585889 14811 raft_consensus.cc:383] T e4f688e4c40d4653929432b13508348a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.585918 14811 raft_consensus.cc:738] T e4f688e4c40d4653929432b13508348a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.586038 14811 consensus_queue.cc:260] T e4f688e4c40d4653929432b13508348a P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:16.586064 14493 leader_election.cc:343] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: ec9f14c1ab014f29843a568c460a28d6
I20250115 03:06:16.586338 14811 ts_tablet_manager.cc:1428] T e4f688e4c40d4653929432b13508348a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:16.586757 14812 tablet_bootstrap.cc:492] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.586828 14812 ts_tablet_manager.cc:1397] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20250115 03:06:16.587198 14812 raft_consensus.cc:357] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.587312 14812 raft_consensus.cc:383] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.587342 14812 raft_consensus.cc:738] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.587440 14812 consensus_queue.cc:260] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:16.587654 14493 leader_election.cc:343] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: ec9f14c1ab014f29843a568c460a28d6
I20250115 03:06:16.587697 14812 ts_tablet_manager.cc:1428] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.587721 14493 leader_election.cc:304] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.587836 14812 tablet_bootstrap.cc:492] T 6d10731df02f4e9682b8de6962256635 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:16.587992 14626 leader_election.cc:343] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:16.588126 14626 leader_election.cc:343] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.588793 14812 tablet_bootstrap.cc:654] T 6d10731df02f4e9682b8de6962256635 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.591308 14858 raft_consensus.cc:2743] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.591981 14811 tablet_bootstrap.cc:492] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.593914 14811 tablet_bootstrap.cc:654] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.595038 14809 tablet_bootstrap.cc:492] T 420b7c8d43a04bae972eed2014d5634c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.595038 14812 tablet_bootstrap.cc:492] T 6d10731df02f4e9682b8de6962256635 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.595113 14809 ts_tablet_manager.cc:1397] T 420b7c8d43a04bae972eed2014d5634c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.002s
I20250115 03:06:16.595185 14812 ts_tablet_manager.cc:1397] T 6d10731df02f4e9682b8de6962256635 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20250115 03:06:16.595494 14809 raft_consensus.cc:357] T 420b7c8d43a04bae972eed2014d5634c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.595535 14812 raft_consensus.cc:357] T 6d10731df02f4e9682b8de6962256635 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.595582 14809 raft_consensus.cc:383] T 420b7c8d43a04bae972eed2014d5634c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.595607 14812 raft_consensus.cc:383] T 6d10731df02f4e9682b8de6962256635 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.595610 14809 raft_consensus.cc:738] T 420b7c8d43a04bae972eed2014d5634c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.595635 14812 raft_consensus.cc:738] T 6d10731df02f4e9682b8de6962256635 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.595713 14809 consensus_queue.cc:260] T 420b7c8d43a04bae972eed2014d5634c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.595727 14812 consensus_queue.cc:260] T 6d10731df02f4e9682b8de6962256635 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.595975 14809 ts_tablet_manager.cc:1428] T 420b7c8d43a04bae972eed2014d5634c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.596837 14809 tablet_bootstrap.cc:492] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.597640 14812 ts_tablet_manager.cc:1428] T 6d10731df02f4e9682b8de6962256635 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:16.597709 14809 tablet_bootstrap.cc:654] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.597806 14812 tablet_bootstrap.cc:492] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.599460 14812 tablet_bootstrap.cc:654] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.602412 14812 tablet_bootstrap.cc:492] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.602497 14812 ts_tablet_manager.cc:1397] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20250115 03:06:16.602911 14812 raft_consensus.cc:357] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.603009 14812 raft_consensus.cc:383] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.603042 14812 raft_consensus.cc:738] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.603154 14812 consensus_queue.cc:260] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.603513 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0964201dda974eaabef3d5694c40389c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.603854 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0964201dda974eaabef3d5694c40389c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.604310 14812 ts_tablet_manager.cc:1428] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:16.604458 14812 tablet_bootstrap.cc:492] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.605182 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_10e94765088044309cd6f627ab7a293b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:16.605309 14812 tablet_bootstrap.cc:654] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:16.605453 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_10e94765088044309cd6f627ab7a293b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.605552 14811 tablet_bootstrap.cc:492] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.605623 14811 ts_tablet_manager.cc:1397] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20250115 03:06:16.606025 14811 raft_consensus.cc:357] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.606112 14811 raft_consensus.cc:383] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.606076 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e51b0b04bb044b36be347afe90845b14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:16.606137 14811 raft_consensus.cc:738] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.606262 14811 consensus_queue.cc:260] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:16.606308 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e51b0b04bb044b36be347afe90845b14 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.606571 14809 tablet_bootstrap.cc:492] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.606609 14811 ts_tablet_manager.cc:1428] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.606637 14809 ts_tablet_manager.cc:1397] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20250115 03:06:16.607012 14809 raft_consensus.cc:357] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.607101 14809 raft_consensus.cc:383] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.607131 14809 raft_consensus.cc:738] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.607298 14809 consensus_queue.cc:260] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.607554 14809 ts_tablet_manager.cc:1428] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.607833 14809 tablet_bootstrap.cc:492] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.608637 14809 tablet_bootstrap.cc:654] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.608659 14811 tablet_bootstrap.cc:492] T 6864f88bf6454ee980f624ec2f2d9fb6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.608918 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b169e386c2cc424da59a39503cec548f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.609201 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b169e386c2cc424da59a39503cec548f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.609508 14811 tablet_bootstrap.cc:654] T 6864f88bf6454ee980f624ec2f2d9fb6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.610472 14811 tablet_bootstrap.cc:492] T 6864f88bf6454ee980f624ec2f2d9fb6 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.610575 14811 ts_tablet_manager.cc:1397] T 6864f88bf6454ee980f624ec2f2d9fb6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20250115 03:06:16.611114 14811 raft_consensus.cc:357] T 6864f88bf6454ee980f624ec2f2d9fb6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.611227 14811 raft_consensus.cc:383] T 6864f88bf6454ee980f624ec2f2d9fb6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.611277 14811 raft_consensus.cc:738] T 6864f88bf6454ee980f624ec2f2d9fb6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.611426 14811 consensus_queue.cc:260] T 6864f88bf6454ee980f624ec2f2d9fb6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.611711 14811 ts_tablet_manager.cc:1428] T 6864f88bf6454ee980f624ec2f2d9fb6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.611897 14811 tablet_bootstrap.cc:492] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.612803 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 90277f7d279e4c7a83c43f40766cd11b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.612999 14823 tablet_replica.cc:331] T 90277f7d279e4c7a83c43f40766cd11b P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.613031 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_719936ebaae74a91821a75f9f8edc316" } new_table_name: "default.Table_341a3cd1c8c94ff4b05fc21960a4e9b4" modify_external_catalogs: true
I20250115 03:06:16.613106 14823 raft_consensus.cc:2238] T 90277f7d279e4c7a83c43f40766cd11b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.613185 14823 raft_consensus.cc:2267] T 90277f7d279e4c7a83c43f40766cd11b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.613615 14823 ts_tablet_manager.cc:1905] T 90277f7d279e4c7a83c43f40766cd11b P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.615743 14823 ts_tablet_manager.cc:1918] T 90277f7d279e4c7a83c43f40766cd11b P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.615803 14823 log.cc:1198] T 90277f7d279e4c7a83c43f40766cd11b P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/90277f7d279e4c7a83c43f40766cd11b
I20250115 03:06:16.615983 14823 ts_tablet_manager.cc:1939] T 90277f7d279e4c7a83c43f40766cd11b P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.616575 14166 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e8a58d3776bc440b90fe3d310db9b131" } modify_external_catalogs: true
I20250115 03:06:16.616657 14166 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e8a58d3776bc440b90fe3d310db9b131" } modify_external_catalogs: true
I20250115 03:06:16.612722 14811 tablet_bootstrap.cc:654] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.619691 14858 raft_consensus.cc:491] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.619778 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 90277f7d279e4c7a83c43f40766cd11b (table default.Table_7a132f92dbad4a868f47b81f0107d42e [id=a888fcc40089456fba95b6e634ae85f7]) successfully deleted
I20250115 03:06:16.619771 14858 raft_consensus.cc:513] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.620079 14811 tablet_bootstrap.cc:492] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.620157 14811 ts_tablet_manager.cc:1397] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20250115 03:06:16.620527 14811 raft_consensus.cc:357] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.620608 14811 raft_consensus.cc:383] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.620635 14811 raft_consensus.cc:738] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.620718 14811 consensus_queue.cc:260] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.621007 14811 ts_tablet_manager.cc:1428] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.621165 14811 tablet_bootstrap.cc:492] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.621973 14811 tablet_bootstrap.cc:654] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.622834 14853 raft_consensus.cc:491] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.622907 14853 raft_consensus.cc:513] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.623200 14853 leader_election.cc:290] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.620090 14858 leader_election.cc:290] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:16.624315 14493 leader_election.cc:343] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 8baf86b37e9942ba813b099a0da87ca5
I20250115 03:06:16.624446 14861 raft_consensus.cc:491] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.624529 14861 raft_consensus.cc:513] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.624836 14861 leader_election.cc:290] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.625226 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c3ff892f9b4dc484ffc80d7c5934c4" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:16.625346 14687 raft_consensus.cc:2463] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:16.625689 14493 leader_election.cc:304] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:16.625978 14858 raft_consensus.cc:2798] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:16.626034 14858 raft_consensus.cc:491] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:16.626075 14858 raft_consensus.cc:3054] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.626852 14858 raft_consensus.cc:513] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.627151 14858 leader_election.cc:290] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.627722 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4f688e4c40d4653929432b13508348a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:16.627827 14687 raft_consensus.cc:2463] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:16.627979 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c3ff892f9b4dc484ffc80d7c5934c4" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:16.628055 14691 raft_consensus.cc:3054] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.628551 14358 leader_election.cc:304] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.628710 14861 raft_consensus.cc:2798] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:16.628757 14861 raft_consensus.cc:491] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:16.628794 14861 raft_consensus.cc:3054] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.628841 14691 raft_consensus.cc:2463] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:16.629235 14493 leader_election.cc:304] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:16.629748 14861 raft_consensus.cc:513] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.630059 14861 leader_election.cc:290] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.630156 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c3ff892f9b4dc484ffc80d7c5934c4" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:16.630192 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet 8baf86b37e9942ba813b099a0da87ca5 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:16.630249 14422 raft_consensus.cc:3054] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.630509 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4f688e4c40d4653929432b13508348a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:16.630523 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4f688e4c40d4653929432b13508348a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:16.630602 14691 raft_consensus.cc:3054] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.630609 14555 raft_consensus.cc:3054] T e4f688e4c40d4653929432b13508348a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.631063 14422 raft_consensus.cc:2463] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:16.631413 14691 raft_consensus.cc:2463] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:16.631418 14555 raft_consensus.cc:2463] T e4f688e4c40d4653929432b13508348a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:16.631603 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4f688e4c40d4653929432b13508348a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.631700 14556 raft_consensus.cc:2371] T e4f688e4c40d4653929432b13508348a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1. Re-sending same reply.
I20250115 03:06:16.631734 14358 leader_election.cc:304] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
W20250115 03:06:16.631934 14493 leader_election.cc:343] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 8baf86b37e9942ba813b099a0da87ca5
I20250115 03:06:16.631994 14493 leader_election.cc:304] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.632193 14853 raft_consensus.cc:2798] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:16.632273 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 7043616bd6b34299862336924a6f16bf (DEFAULT_TABLE table=default.Table_426742b2ee4d41da8f060fb20309d184 [id=b947e0a4a93a42a589491ea823038dbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.632314 14853 raft_consensus.cc:695] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:16.632421 14853 consensus_queue.cc:237] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.632493 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7043616bd6b34299862336924a6f16bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.632644 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 7b86e66428504217af08f667f660d304 (DEFAULT_TABLE table=default.Table_719936ebaae74a91821a75f9f8edc316 [id=c096ce170874405ea740b9ea829f886e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.632731 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 998a515113d04034928c04dc5798a698 (DEFAULT_TABLE table=default.Table_1f20c003e8664c2381603de8f2055749 [id=fc1c81e2d2614487898512c6615c863b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.632860 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b86e66428504217af08f667f660d304. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.633080 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 7b86e66428504217af08f667f660d304 (DEFAULT_TABLE table=default.Table_719936ebaae74a91821a75f9f8edc316 [id=c096ce170874405ea740b9ea829f886e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.633107 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 12678da7ffee4a46b4690ac3e94cb19b (DEFAULT_TABLE table=default.Table_0337ad43f9b646a48c61c66c15f1844a [id=7f5506ea0a5e465cba8de1ba2f5e75c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.633117 14395 tablet_service.cc:1467] Processing CreateTablet for tablet 998a515113d04034928c04dc5798a698 (DEFAULT_TABLE table=default.Table_1f20c003e8664c2381603de8f2055749 [id=fc1c81e2d2614487898512c6615c863b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.633283 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b86e66428504217af08f667f660d304. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.633282 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12678da7ffee4a46b4690ac3e94cb19b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.633457 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 0f20bb66f36647e0811d0865b8f414f7 (DEFAULT_TABLE table=default.Table_f641c3009a9045c79adb4e38c38e8f1c [id=f48964597a90470299a755217c4dbea0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.633661 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f20bb66f36647e0811d0865b8f414f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.634610 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 7043616bd6b34299862336924a6f16bf (DEFAULT_TABLE table=default.Table_426742b2ee4d41da8f060fb20309d184 [id=b947e0a4a93a42a589491ea823038dbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.634822 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7043616bd6b34299862336924a6f16bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.635164 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 0792e52d171446b48a7c9d1f03d37a79 (DEFAULT_TABLE table=default.Table_72e82b3e776e4d398fa5d8240afde936 [id=17ae925a7c0942d59058b2fd619ccb45]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.635164 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 12678da7ffee4a46b4690ac3e94cb19b (DEFAULT_TABLE table=default.Table_0337ad43f9b646a48c61c66c15f1844a [id=7f5506ea0a5e465cba8de1ba2f5e75c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.635356 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12678da7ffee4a46b4690ac3e94cb19b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.635356 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0792e52d171446b48a7c9d1f03d37a79. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.635490 14395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 998a515113d04034928c04dc5798a698. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.635668 14672 tablet_service.cc:1467] Processing CreateTablet for tablet a4547465f3124e108dc2c4c54e2bc751 (DEFAULT_TABLE table=default.Table_58e0901d0ec94ef9859515f126825621 [id=50c73c26b4ed47a4961348db433b7dd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.635871 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4547465f3124e108dc2c4c54e2bc751. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.636629 14397 tablet_service.cc:1467] Processing CreateTablet for tablet a4547465f3124e108dc2c4c54e2bc751 (DEFAULT_TABLE table=default.Table_58e0901d0ec94ef9859515f126825621 [id=50c73c26b4ed47a4961348db433b7dd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.636860 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4547465f3124e108dc2c4c54e2bc751. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.637169 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 0f20bb66f36647e0811d0865b8f414f7 (DEFAULT_TABLE table=default.Table_f641c3009a9045c79adb4e38c38e8f1c [id=f48964597a90470299a755217c4dbea0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.637370 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f20bb66f36647e0811d0865b8f414f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.637705 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 0792e52d171446b48a7c9d1f03d37a79 (DEFAULT_TABLE table=default.Table_72e82b3e776e4d398fa5d8240afde936 [id=17ae925a7c0942d59058b2fd619ccb45]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.637879 14853 raft_consensus.cc:2743] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.637916 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0792e52d171446b48a7c9d1f03d37a79. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.638708 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 815ef85ce5cf4ae0a56c5734837e03ba (DEFAULT_TABLE table=default.Table_24cbe6b962b248a9b340506b63c41b20 [id=9866275e65ab4dfcbfee6bf209c3654e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.638942 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 815ef85ce5cf4ae0a56c5734837e03ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.639578 14394 tablet_service.cc:1467] Processing CreateTablet for tablet f2ca06c402cd46dc8311c8a47da92846 (DEFAULT_TABLE table=default.Table_a1547d1d923f4f4da05aaeb519f4cdf0 [id=e929ffaba567498c815393836b264a12]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.639775 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2ca06c402cd46dc8311c8a47da92846. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.639745 14663 tablet_service.cc:1467] Processing CreateTablet for tablet f2ca06c402cd46dc8311c8a47da92846 (DEFAULT_TABLE table=default.Table_a1547d1d923f4f4da05aaeb519f4cdf0 [id=e929ffaba567498c815393836b264a12]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.639925 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2ca06c402cd46dc8311c8a47da92846. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.640805 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 815ef85ce5cf4ae0a56c5734837e03ba (DEFAULT_TABLE table=default.Table_24cbe6b962b248a9b340506b63c41b20 [id=9866275e65ab4dfcbfee6bf209c3654e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.641016 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 815ef85ce5cf4ae0a56c5734837e03ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.641676 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 998a515113d04034928c04dc5798a698. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.642630 15401 raft_consensus.cc:2798] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:16.642711 15401 raft_consensus.cc:695] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:16.642818 15401 consensus_queue.cc:237] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.643289 14468 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:16.643362 15401 raft_consensus.cc:491] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.643424 15401 raft_consensus.cc:513] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.643687 15401 leader_election.cc:290] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.643950 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48aa039bf0364c02abb7f9677eae49e5" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:16.644311 14356 leader_election.cc:343] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.645614 15390 raft_consensus.cc:491] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.645689 15390 raft_consensus.cc:513] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.645898 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4c3ff892f9b4dc484ffc80d7c5934c4" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:16.645983 14421 raft_consensus.cc:2371] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1. Re-sending same reply.
I20250115 03:06:16.645983 15390 leader_election.cc:290] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.647408 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acb43d614111477abb10def48a1dfb3b" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:16.647656 14358 leader_election.cc:343] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: acb43d614111477abb10def48a1dfb3b
I20250115 03:06:16.648175 14603 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:16.648236 14534 tablet_service.cc:1467] Processing CreateTablet for tablet 7b86e66428504217af08f667f660d304 (DEFAULT_TABLE table=default.Table_719936ebaae74a91821a75f9f8edc316 [id=c096ce170874405ea740b9ea829f886e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.648607 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 12678da7ffee4a46b4690ac3e94cb19b (DEFAULT_TABLE table=default.Table_0337ad43f9b646a48c61c66c15f1844a [id=7f5506ea0a5e465cba8de1ba2f5e75c6]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:16.648895 14356 leader_election.cc:343] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.648975 14356 leader_election.cc:304] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:16.649106 14861 raft_consensus.cc:2743] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.649255 14531 tablet_service.cc:1467] Processing CreateTablet for tablet 7043616bd6b34299862336924a6f16bf (DEFAULT_TABLE table=default.Table_426742b2ee4d41da8f060fb20309d184 [id=b947e0a4a93a42a589491ea823038dbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.649503 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 998a515113d04034928c04dc5798a698 (DEFAULT_TABLE table=default.Table_1f20c003e8664c2381603de8f2055749 [id=fc1c81e2d2614487898512c6615c863b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.649742 14528 tablet_service.cc:1467] Processing CreateTablet for tablet 0792e52d171446b48a7c9d1f03d37a79 (DEFAULT_TABLE table=default.Table_72e82b3e776e4d398fa5d8240afde936 [id=17ae925a7c0942d59058b2fd619ccb45]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.650085 14535 tablet_service.cc:1467] Processing CreateTablet for tablet a4547465f3124e108dc2c4c54e2bc751 (DEFAULT_TABLE table=default.Table_58e0901d0ec94ef9859515f126825621 [id=50c73c26b4ed47a4961348db433b7dd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.650247 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b86e66428504217af08f667f660d304. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.650329 14529 tablet_service.cc:1467] Processing CreateTablet for tablet 815ef85ce5cf4ae0a56c5734837e03ba (DEFAULT_TABLE table=default.Table_24cbe6b962b248a9b340506b63c41b20 [id=9866275e65ab4dfcbfee6bf209c3654e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.650370 14532 tablet_service.cc:1467] Processing CreateTablet for tablet f2ca06c402cd46dc8311c8a47da92846 (DEFAULT_TABLE table=default.Table_a1547d1d923f4f4da05aaeb519f4cdf0 [id=e929ffaba567498c815393836b264a12]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.650597 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 0f20bb66f36647e0811d0865b8f414f7 (DEFAULT_TABLE table=default.Table_f641c3009a9045c79adb4e38c38e8f1c [id=f48964597a90470299a755217c4dbea0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.652817 14738 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:16.653245 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4547465f3124e108dc2c4c54e2bc751. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.653716 14812 tablet_bootstrap.cc:492] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.653792 14812 ts_tablet_manager.cc:1397] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.049s user 0.001s sys 0.000s
I20250115 03:06:16.654238 14812 raft_consensus.cc:357] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.654326 14812 raft_consensus.cc:383] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.654354 14812 raft_consensus.cc:738] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.654453 14812 consensus_queue.cc:260] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.654704 14812 ts_tablet_manager.cc:1428] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.654853 14812 tablet_bootstrap.cc:492] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.655375 14861 raft_consensus.cc:491] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.655448 14861 raft_consensus.cc:513] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:16.655557 14358 leader_election.cc:343] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 48aa039bf0364c02abb7f9677eae49e5
I20250115 03:06:16.655606 14358 leader_election.cc:304] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:16.655736 14812 tablet_bootstrap.cc:654] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.655752 14861 leader_election.cc:290] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.655762 15390 raft_consensus.cc:2743] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.656848 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2ca06c402cd46dc8311c8a47da92846. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.657301 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "872ad705e3e94c0eab4397df9f28f9ca" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:16.657402 14691 raft_consensus.cc:2463] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:16.657649 14358 leader_election.cc:304] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.657789 15390 raft_consensus.cc:2798] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:16.657830 15390 raft_consensus.cc:491] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:16.657863 15390 raft_consensus.cc:3054] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.658722 15390 raft_consensus.cc:513] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.658779 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f20bb66f36647e0811d0865b8f414f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.659039 15390 leader_election.cc:290] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.659507 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_51d58281d1844f48b14d851ff78eafe9" } modify_external_catalogs: true
I20250115 03:06:16.659608 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_51d58281d1844f48b14d851ff78eafe9" } modify_external_catalogs: true
I20250115 03:06:16.660316 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "872ad705e3e94c0eab4397df9f28f9ca" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.660419 14556 raft_consensus.cc:2463] T 872ad705e3e94c0eab4397df9f28f9ca P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:16.660594 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "872ad705e3e94c0eab4397df9f28f9ca" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:16.660681 14555 raft_consensus.cc:3054] T 872ad705e3e94c0eab4397df9f28f9ca P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.660975 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_9cc1022af4f14e7a8a8e11cc1803ca87" } new_table_name: "default.Table_a60b99b368f74c68873e8c89066cd3e0" modify_external_catalogs: true
I20250115 03:06:16.661547 14555 raft_consensus.cc:2463] T 872ad705e3e94c0eab4397df9f28f9ca P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:16.661934 14356 leader_election.cc:304] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.662077 15390 raft_consensus.cc:2798] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:16.662145 15390 raft_consensus.cc:695] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:16.662182 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 420b7c8d43a04bae972eed2014d5634c
I20250115 03:06:16.662251 15390 consensus_queue.cc:237] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.662431 14164 catalog_manager.cc:6596] ReplaceTablet: tablet 420b7c8d43a04bae972eed2014d5634c deleted and replaced by tablet b2629ae9d9854ae4b6e7ddddbb6f341b
I20250115 03:06:16.662564 14163 catalog_manager.cc:5526] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:16.662755 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 6864f88bf6454ee980f624ec2f2d9fb6 (table default.Table_2e68c93eaa1647df9b269879bd4520ab [id=18e0f06feeda4bca860e6f1bc6eab15b]). Expected version 1 got 0
W20250115 03:06:16.662802 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7b86e66428504217af08f667f660d304 consensus state has no leader
I20250115 03:06:16.662883 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 13f36a563f13403eab372deb10b67451 (table default.Table_70d0b700eac34892b4ef769d739481fd [id=61f3625fed2b4a70b5331d6c6ac73f1c]). Expected version 1 got 0
I20250115 03:06:16.662971 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 5986558350244e7aa4ddcd5cccb7d676 (table default.Table_e72d1ee69400410c9da9082d4bd8bc9a [id=6ab343bee1254fb19a3024ebafa44464]). Expected version 1 got 0
I20250115 03:06:16.663326 14166 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet f94657e230684757bf4bcc803b04e0db
I20250115 03:06:16.664021 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_aa81abdb583a417ca417be1d66854e8f" } modify_external_catalogs: true
I20250115 03:06:16.664103 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_aa81abdb583a417ca417be1d66854e8f" } modify_external_catalogs: true
I20250115 03:06:16.664788 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "872ad705e3e94c0eab4397df9f28f9ca" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:16.664885 14691 raft_consensus.cc:3054] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.665004 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 420b7c8d43a04bae972eed2014d5634c with delete_type TABLET_DATA_DELETED (replaced by tablet b2629ae9d9854ae4b6e7ddddbb6f341b) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.665035 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0792e52d171446b48a7c9d1f03d37a79. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.665144 14819 tablet_replica.cc:331] T 420b7c8d43a04bae972eed2014d5634c P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.665256 14819 raft_consensus.cc:2238] T 420b7c8d43a04bae972eed2014d5634c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.665336 14819 raft_consensus.cc:2267] T 420b7c8d43a04bae972eed2014d5634c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.665786 14691 raft_consensus.cc:2463] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:16.665843 14171 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet a1c93bfd3da44f9f8a8d585f2d21f1be (table default.Table_40b1953a4a364eb1ae58e3387fc65a91 [id=64d20b1792044c5e8add735aaf75a6d9]). Expected version 1 got 0
I20250115 03:06:16.665961 14171 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet f8b90f854a5e4221a24986f8c71ca300 (table default.Table_4e74d34a621f4b1898241f62a20ffc10 [id=d8084de64a264d28a3370971617a694c]). Expected version 2 got 0
I20250115 03:06:16.666031 14171 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 745344db22da4102bad39c2f121548a1 (table default.Table_365313892043408c92be8460c31d958e [id=f22f08e9af4a4496b100e7d4b5982121]). Expected version 1 got 0
I20250115 03:06:16.666887 14531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7043616bd6b34299862336924a6f16bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.667026 14819 ts_tablet_manager.cc:1905] T 420b7c8d43a04bae972eed2014d5634c P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.667351 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12678da7ffee4a46b4690ac3e94cb19b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.667331 14166 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8daa0abab4d1464488bec21bb7ad375b" } modify_external_catalogs: true
I20250115 03:06:16.667418 14166 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8daa0abab4d1464488bec21bb7ad375b" } modify_external_catalogs: true
I20250115 03:06:16.667742 14812 tablet_bootstrap.cc:492] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.667815 14812 ts_tablet_manager.cc:1397] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20250115 03:06:16.668264 14812 raft_consensus.cc:357] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.668358 14812 raft_consensus.cc:383] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.668388 14812 raft_consensus.cc:738] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.668782 14812 consensus_queue.cc:260] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.669070 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 815ef85ce5cf4ae0a56c5734837e03ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.669081 14819 ts_tablet_manager.cc:1918] T 420b7c8d43a04bae972eed2014d5634c P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.669132 14819 log.cc:1198] T 420b7c8d43a04bae972eed2014d5634c P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/420b7c8d43a04bae972eed2014d5634c
I20250115 03:06:16.669315 14819 ts_tablet_manager.cc:1939] T 420b7c8d43a04bae972eed2014d5634c P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.669728 14812 ts_tablet_manager.cc:1428] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:16.669885 14812 tablet_bootstrap.cc:492] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.670770 14812 tablet_bootstrap.cc:654] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.670892 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 998a515113d04034928c04dc5798a698. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.671686 14167 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet 5bf06fe685d8415e973535a05b683d54
I20250115 03:06:16.675400 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 420b7c8d43a04bae972eed2014d5634c with delete_type TABLET_DATA_DELETED (replaced by tablet b2629ae9d9854ae4b6e7ddddbb6f341b) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.675590 14823 tablet_replica.cc:331] T 420b7c8d43a04bae972eed2014d5634c P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.675712 14823 raft_consensus.cc:2238] T 420b7c8d43a04bae972eed2014d5634c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.675802 14823 raft_consensus.cc:2267] T 420b7c8d43a04bae972eed2014d5634c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.676162 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 420b7c8d43a04bae972eed2014d5634c (table default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]) successfully deleted
I20250115 03:06:16.676189 14823 ts_tablet_manager.cc:1905] T 420b7c8d43a04bae972eed2014d5634c P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.676730 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 48aa039bf0364c02abb7f9677eae49e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.676857 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 506198162e0b4e24924c408542707c4c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.676992 14170 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet eae13c6e85964d20b232f80711ad79eb
I20250115 03:06:16.676982 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 33380937a3b7428995b06e477a896c02 with delete_type TABLET_DATA_DELETED (replaced by tablet c251960c7c9a44429102a294f35f925c) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.677083 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d10731df02f4e9682b8de6962256635 with delete_type TABLET_DATA_DELETED (replaced by tablet 5780da67ce984297ba33d3c6901c9e3a) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.678112 14823 ts_tablet_manager.cc:1918] T 420b7c8d43a04bae972eed2014d5634c P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.678174 14823 log.cc:1198] T 420b7c8d43a04bae972eed2014d5634c P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/420b7c8d43a04bae972eed2014d5634c
I20250115 03:06:16.678339 14823 ts_tablet_manager.cc:1939] T 420b7c8d43a04bae972eed2014d5634c P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.678484 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet acb43d614111477abb10def48a1dfb3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.679878 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 5589b497b0e24740ad49d1392f5a737b
I20250115 03:06:16.680594 14819 tablet_replica.cc:331] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.680711 14819 raft_consensus.cc:2238] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.680799 14819 raft_consensus.cc:2267] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.681177 14170 catalog_manager.cc:5526] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:16.681371 14170 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet a4c3ff892f9b4dc484ffc80d7c5934c4 (table default.Table_76ef9277dc8142f1a86962bfc430f2b2 [id=f70558594b8348579a0ad5d4c8fd8885]). Expected version 1 got 0
I20250115 03:06:16.681480 14170 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 6864f88bf6454ee980f624ec2f2d9fb6 (table default.Table_2e68c93eaa1647df9b269879bd4520ab [id=18e0f06feeda4bca860e6f1bc6eab15b]). Expected version 1 got 0
I20250115 03:06:16.681663 14819 ts_tablet_manager.cc:1905] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.683420 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 420b7c8d43a04bae972eed2014d5634c with delete_type TABLET_DATA_DELETED (replaced by tablet b2629ae9d9854ae4b6e7ddddbb6f341b) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.683861 15617 tablet.cc:1721] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:16.685096 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet 1f2dc6928e2941ef9b5b5a7891376500 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:16.685735 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2af92748648345c0903a0f99da0d47c5" } modify_external_catalogs: true
I20250115 03:06:16.685827 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2af92748648345c0903a0f99da0d47c5" } modify_external_catalogs: true
I20250115 03:06:16.686159 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 5589b497b0e24740ad49d1392f5a737b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.686368 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 5589b497b0e24740ad49d1392f5a737b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.688117 15153 raft_consensus.cc:491] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.688186 15153 raft_consensus.cc:513] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:16.688381 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 420b7c8d43a04bae972eed2014d5634c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 420b7c8d43a04bae972eed2014d5634c already in progress: creating tablet
I20250115 03:06:16.688525 15153 leader_election.cc:290] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:16.688787 15274 tablet.cc:1721] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:16.688827 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcdb5282e41c43dbb235ae78465d19f9" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.688987 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcdb5282e41c43dbb235ae78465d19f9" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:16.689061 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 5589b497b0e24740ad49d1392f5a737b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:16.689591 14628 leader_election.cc:343] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.690775 14166 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet dcdb5282e41c43dbb235ae78465d19f9 (table default.Table_375bb4e2095c4675b734247fabacbec7 [id=b298dce7ab664327893633f821ed4d89]). Expected version 1 got 0
I20250115 03:06:16.691131 15834 sys_catalog.cc:263] Verifying existing consensus state
W20250115 03:06:16.691799 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 5589b497b0e24740ad49d1392f5a737b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5589b497b0e24740ad49d1392f5a737b already in progress: creating tablet
W20250115 03:06:16.692622 14626 leader_election.cc:343] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.692683 14626 leader_election.cc:304] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.693001 15153 raft_consensus.cc:2743] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.694238 14812 tablet_bootstrap.cc:492] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.694312 14812 ts_tablet_manager.cc:1397] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.001s
I20250115 03:06:16.695156 14812 raft_consensus.cc:357] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.695293 14812 raft_consensus.cc:383] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.695323 14812 raft_consensus.cc:738] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.695468 14812 consensus_queue.cc:260] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.696357 14812 ts_tablet_manager.cc:1428] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:16.696537 14812 tablet_bootstrap.cc:492] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.698149 15390 raft_consensus.cc:491] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.698231 15390 raft_consensus.cc:513] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.698567 15390 leader_election.cc:290] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.699043 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "745344db22da4102bad39c2f121548a1" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:16.699146 14691 raft_consensus.cc:2463] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:16.699358 14358 leader_election.cc:304] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.699713 15390 raft_consensus.cc:2798] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:16.699764 15390 raft_consensus.cc:491] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:16.699795 15390 raft_consensus.cc:3054] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.700400 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "745344db22da4102bad39c2f121548a1" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.700604 15426 tablet.cc:1721] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:16.700664 15390 raft_consensus.cc:513] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.700904 15390 leader_election.cc:290] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.701445 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet 20f86f93812849c4b7852a884f677c04 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:16.701495 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "745344db22da4102bad39c2f121548a1" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:16.701579 14691 raft_consensus.cc:3054] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.702154 14812 tablet_bootstrap.cc:654] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.702694 14691 raft_consensus.cc:2463] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:16.703022 14358 leader_election.cc:304] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.703197 14861 raft_consensus.cc:491] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.703263 14861 raft_consensus.cc:513] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.703554 14861 leader_election.cc:290] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.703629 14861 raft_consensus.cc:2798] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:16.703691 14861 raft_consensus.cc:695] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:16.703794 14861 consensus_queue.cc:237] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.704082 14809 tablet_bootstrap.cc:492] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.704150 14809 ts_tablet_manager.cc:1397] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.096s user 0.000s sys 0.001s
I20250115 03:06:16.704586 14809 raft_consensus.cc:357] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:16.704675 14358 leader_election.cc:343] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 33380937a3b7428995b06e477a896c02
I20250115 03:06:16.704684 14809 raft_consensus.cc:383] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.704787 14809 raft_consensus.cc:738] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.704885 14809 consensus_queue.cc:260] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.705171 14809 ts_tablet_manager.cc:1428] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20250115 03:06:16.705323 14356 leader_election.cc:343] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.705329 14809 tablet_bootstrap.cc:492] T eeff2cd0825c4419aacfc73ebe267a8e P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.706139 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b9e69a84b955444488aff2ca4794feca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.706467 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b9e69a84b955444488aff2ca4794feca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.706550 14809 tablet_bootstrap.cc:654] T eeff2cd0825c4419aacfc73ebe267a8e P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.706558 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33380937a3b7428995b06e477a896c02" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.706923 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "745344db22da4102bad39c2f121548a1" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:16.707154 15834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246: Bootstrap starting.
I20250115 03:06:16.708004 15153 raft_consensus.cc:491] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.708081 15153 raft_consensus.cc:513] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.708190 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_8f7e876bb60444429c30944284f9b10d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:16.708415 15153 leader_election.cc:290] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
W20250115 03:06:16.708491 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f7e876bb60444429c30944284f9b10d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.708648 14811 tablet_bootstrap.cc:492] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.708724 14811 ts_tablet_manager.cc:1397] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.088s user 0.001s sys 0.000s
I20250115 03:06:16.709069 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1c0f6a5611647eb884c77f7a6d1cd4e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:16.709165 14421 raft_consensus.cc:2463] T b1c0f6a5611647eb884c77f7a6d1cd4e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:16.709158 14811 raft_consensus.cc:357] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.709246 14811 raft_consensus.cc:383] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.709277 14811 raft_consensus.cc:738] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.709383 14811 consensus_queue.cc:260] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.709458 14628 leader_election.cc:304] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.709601 15153 raft_consensus.cc:2798] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:16.709645 15153 raft_consensus.cc:491] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:16.709653 14811 ts_tablet_manager.cc:1428] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.709681 15153 raft_consensus.cc:3054] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.709822 14811 tablet_bootstrap.cc:492] T 304fab45b38e4426ae6961ae786e5659 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.710562 15153 raft_consensus.cc:513] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.710692 14811 tablet_bootstrap.cc:654] T 304fab45b38e4426ae6961ae786e5659 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.710860 15153 leader_election.cc:290] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:16.710876 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet c251960c7c9a44429102a294f35f925c from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:16.711269 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1c0f6a5611647eb884c77f7a6d1cd4e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:16.711378 14421 raft_consensus.cc:3054] T b1c0f6a5611647eb884c77f7a6d1cd4e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.711457 14819 ts_tablet_manager.cc:1918] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.711504 14819 log.cc:1198] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/48aa039bf0364c02abb7f9677eae49e5
I20250115 03:06:16.711674 14819 ts_tablet_manager.cc:1939] T 48aa039bf0364c02abb7f9677eae49e5 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.711961 14819 tablet_replica.cc:331] T 506198162e0b4e24924c408542707c4c P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.711994 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1c0f6a5611647eb884c77f7a6d1cd4e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:16.712076 14819 raft_consensus.cc:2238] T 506198162e0b4e24924c408542707c4c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.712157 14819 raft_consensus.cc:2267] T 506198162e0b4e24924c408542707c4c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.712178 14421 raft_consensus.cc:2463] T b1c0f6a5611647eb884c77f7a6d1cd4e P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:16.712189 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1c0f6a5611647eb884c77f7a6d1cd4e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.712466 14628 leader_election.cc:304] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.712555 14819 ts_tablet_manager.cc:1905] T 506198162e0b4e24924c408542707c4c P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.712647 15153 raft_consensus.cc:2798] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:16.712711 15153 raft_consensus.cc:695] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
W20250115 03:06:16.712740 14626 leader_election.cc:343] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.712837 14823 tablet_replica.cc:331] T 6d10731df02f4e9682b8de6962256635 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.712816 15153 consensus_queue.cc:237] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:16.712915 14626 leader_election.cc:343] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.712970 14823 raft_consensus.cc:2238] T 6d10731df02f4e9682b8de6962256635 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.713053 14823 raft_consensus.cc:2267] T 6d10731df02f4e9682b8de6962256635 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:16.713251 14356 leader_election.cc:343] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.713308 14356 leader_election.cc:304] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:16.713500 15401 raft_consensus.cc:2743] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:16.713533 14356 leader_election.cc:343] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.714020 14823 ts_tablet_manager.cc:1905] T 6d10731df02f4e9682b8de6962256635 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.714818 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 420b7c8d43a04bae972eed2014d5634c (table default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]) successfully deleted
I20250115 03:06:16.714926 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 48aa039bf0364c02abb7f9677eae49e5 (table default.Table_35005559d2f34672bc3e70f6bbfb31d6 [id=d73a4eb552114e20887f9599f244e6ed]) successfully deleted
I20250115 03:06:16.716363 14809 tablet_bootstrap.cc:492] T eeff2cd0825c4419aacfc73ebe267a8e P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.716429 14809 ts_tablet_manager.cc:1397] T eeff2cd0825c4419aacfc73ebe267a8e P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.001s
I20250115 03:06:16.716835 14809 raft_consensus.cc:357] T eeff2cd0825c4419aacfc73ebe267a8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.716920 14809 raft_consensus.cc:383] T eeff2cd0825c4419aacfc73ebe267a8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.716989 14809 raft_consensus.cc:738] T eeff2cd0825c4419aacfc73ebe267a8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.717073 14809 consensus_queue.cc:260] T eeff2cd0825c4419aacfc73ebe267a8e P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.717326 14809 ts_tablet_manager.cc:1428] T eeff2cd0825c4419aacfc73ebe267a8e P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.717461 14809 tablet_bootstrap.cc:492] T b0a5d554764249f8a7eefdc943271090 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.718497 14809 tablet_bootstrap.cc:654] T b0a5d554764249f8a7eefdc943271090 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.719033 14823 ts_tablet_manager.cc:1918] T 6d10731df02f4e9682b8de6962256635 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.719094 14823 log.cc:1198] T 6d10731df02f4e9682b8de6962256635 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/6d10731df02f4e9682b8de6962256635
I20250115 03:06:16.719190 14858 raft_consensus.cc:491] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.719283 14823 ts_tablet_manager.cc:1939] T 6d10731df02f4e9682b8de6962256635 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.719254 14858 raft_consensus.cc:513] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.719380 14809 tablet_bootstrap.cc:492] T b0a5d554764249f8a7eefdc943271090 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.719467 14809 ts_tablet_manager.cc:1397] T b0a5d554764249f8a7eefdc943271090 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:16.719558 14858 leader_election.cc:290] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.719844 14809 raft_consensus.cc:357] T b0a5d554764249f8a7eefdc943271090 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.719928 14809 raft_consensus.cc:383] T b0a5d554764249f8a7eefdc943271090 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.719991 14809 raft_consensus.cc:738] T b0a5d554764249f8a7eefdc943271090 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.720093 14809 consensus_queue.cc:260] T b0a5d554764249f8a7eefdc943271090 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:16.720263 14493 leader_election.cc:343] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 5b7022db9edc4de8909e16769a3ea136
I20250115 03:06:16.720348 14809 ts_tablet_manager.cc:1428] T b0a5d554764249f8a7eefdc943271090 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.720489 14809 tablet_bootstrap.cc:492] T 3a0bb9b5093b454d921bf1150d631565 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
W20250115 03:06:16.720727 14493 leader_election.cc:343] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 5b7022db9edc4de8909e16769a3ea136
I20250115 03:06:16.720799 14493 leader_election.cc:304] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.720963 14858 raft_consensus.cc:2743] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.721346 14809 tablet_bootstrap.cc:654] T 3a0bb9b5093b454d921bf1150d631565 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.719921 14819 ts_tablet_manager.cc:1918] T 506198162e0b4e24924c408542707c4c P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.722066 14819 log.cc:1198] T 506198162e0b4e24924c408542707c4c P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/506198162e0b4e24924c408542707c4c
I20250115 03:06:16.722633 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet e368f87556944c46a6d8f160ead17ac9
I20250115 03:06:16.722692 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 1f2dc6928e2941ef9b5b5a7891376500
I20250115 03:06:16.722899 14163 catalog_manager.cc:6596] ReplaceTablet: tablet 1f2dc6928e2941ef9b5b5a7891376500 deleted and replaced by tablet 7f9818d0493547358fd5654a01d4f456
I20250115 03:06:16.724205 14819 ts_tablet_manager.cc:1939] T 506198162e0b4e24924c408542707c4c P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.724589 14166 catalog_manager.cc:5526] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:16.724803 14166 catalog_manager.cc:5526] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:16.725419 14811 tablet_bootstrap.cc:492] T 304fab45b38e4426ae6961ae786e5659 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.725502 14811 ts_tablet_manager.cc:1397] T 304fab45b38e4426ae6961ae786e5659 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20250115 03:06:16.725816 14819 tablet_replica.cc:331] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.725924 14819 raft_consensus.cc:2238] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.725957 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet e368f87556944c46a6d8f160ead17ac9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.726004 14819 raft_consensus.cc:2267] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.726089 14531 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f2dc6928e2941ef9b5b5a7891376500 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f9818d0493547358fd5654a01d4f456) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.726189 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 8baf86b37e9942ba813b099a0da87ca5 with delete_type TABLET_DATA_DELETED (replaced by tablet bfc34dc76ff94b8eaeb5c2822e6498da) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.726289 14529 tablet_service.cc:1514] Processing DeleteTablet for tablet 79579515fb7e45a8b16fd36dbc6a965a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.726361 14819 ts_tablet_manager.cc:1905] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.726402 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f67cbb929cd4ae589a14606aa168b46 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.726511 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet 3cbc91ad00b54ede804c3629bcfe7fe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.726606 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 84ce55f729224f78b420e57977419add with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.726691 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b7022db9edc4de8909e16769a3ea136 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.726805 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet ec9f14c1ab014f29843a568c460a28d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.727188 14820 tablet_replica.cc:331] T e368f87556944c46a6d8f160ead17ac9 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.727296 14820 raft_consensus.cc:2238] T e368f87556944c46a6d8f160ead17ac9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.727384 14820 raft_consensus.cc:2267] T e368f87556944c46a6d8f160ead17ac9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.728353 14167 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_a60b99b368f74c68873e8c89066cd3e0 [id=98f48acd89a6461bb4511141cab4d88b] alter complete (version 3)
I20250115 03:06:16.728442 14820 ts_tablet_manager.cc:1905] T e368f87556944c46a6d8f160ead17ac9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.730295 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f641c3009a9045c79adb4e38c38e8f1c" } new_table_name: "default.Table_0326e94494d0447585233c0c907113fd" modify_external_catalogs: true
I20250115 03:06:16.730561 14820 ts_tablet_manager.cc:1918] T e368f87556944c46a6d8f160ead17ac9 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:16.730618 14820 log.cc:1198] T e368f87556944c46a6d8f160ead17ac9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/e368f87556944c46a6d8f160ead17ac9
I20250115 03:06:16.730830 14820 ts_tablet_manager.cc:1939] T e368f87556944c46a6d8f160ead17ac9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.731940 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0c984dbf685d4dad9338caa1119c5e44"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:16.732064 14820 tablet_replica.cc:331] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.732173 14820 raft_consensus.cc:2238] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.732256 14820 raft_consensus.cc:2267] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:16.732240 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c984dbf685d4dad9338caa1119c5e44 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.732651 14811 raft_consensus.cc:357] T 304fab45b38e4426ae6961ae786e5659 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.732748 14811 raft_consensus.cc:383] T 304fab45b38e4426ae6961ae786e5659 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.732779 14811 raft_consensus.cc:738] T 304fab45b38e4426ae6961ae786e5659 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.732874 14811 consensus_queue.cc:260] T 304fab45b38e4426ae6961ae786e5659 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.733142 14811 ts_tablet_manager.cc:1428] T 304fab45b38e4426ae6961ae786e5659 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20250115 03:06:16.733296 14811 tablet_bootstrap.cc:492] T b1c0f6a5611647eb884c77f7a6d1cd4e P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.733534 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet e368f87556944c46a6d8f160ead17ac9 (table default.Table_2af92748648345c0903a0f99da0d47c5 [id=19efb3734d824699a088ad62e69095f3]) successfully deleted
W20250115 03:06:16.733682 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 1f2dc6928e2941ef9b5b5a7891376500 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f2dc6928e2941ef9b5b5a7891376500 already in progress: creating tablet
I20250115 03:06:16.734221 14811 tablet_bootstrap.cc:654] T b1c0f6a5611647eb884c77f7a6d1cd4e P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.734544 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet e368f87556944c46a6d8f160ead17ac9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.734712 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f2dc6928e2941ef9b5b5a7891376500 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f9818d0493547358fd5654a01d4f456) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.734797 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 6d10731df02f4e9682b8de6962256635 (table default.Table_551ca9fefcb74ed5aa29803f19de6ab6 [id=79eb8c9694b74a31a4253b33d064e230]) successfully deleted
W20250115 03:06:16.734923 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 5589b497b0e24740ad49d1392f5a737b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5589b497b0e24740ad49d1392f5a737b already in progress: creating tablet
I20250115 03:06:16.735057 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 506198162e0b4e24924c408542707c4c (table default.Table_75c071e03ba44140bce71e920f31715d [id=13ea98de708e409ba7b66e29bb49416d]) successfully deleted
I20250115 03:06:16.735160 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet f70a69acbdc049beb9d98eada09d9eb8 (table default.Table_db8a2673977247e0b14b7995f7e09661 [id=f7e4534c1f6f48f09e10b84c1d5c458c]). Expected version 1 got 0
I20250115 03:06:16.735997 14861 raft_consensus.cc:491] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.736065 14861 raft_consensus.cc:513] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.736209 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f2dc6928e2941ef9b5b5a7891376500 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f9818d0493547358fd5654a01d4f456) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.736318 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet e368f87556944c46a6d8f160ead17ac9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.736361 14861 leader_election.cc:290] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.736580 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4fcf8d20f3984b37910ea7164b26cca8" } new_table_name: "default.Table_dfcccd3390fa4814a0a04bb1ff5e43fa" modify_external_catalogs: true
I20250115 03:06:16.736671 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8b90f854a5e4221a24986f8c71ca300" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:16.736774 14687 raft_consensus.cc:2463] T f8b90f854a5e4221a24986f8c71ca300 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:16.737040 14358 leader_election.cc:304] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.737226 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8b90f854a5e4221a24986f8c71ca300" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:16.737761 14356 leader_election.cc:343] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.737998 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0c35f2414d284756b9c93ed61e75d451"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:16.738126 14820 ts_tablet_manager.cc:1905] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.738390 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 20f86f93812849c4b7852a884f677c04
I20250115 03:06:16.739060 15390 raft_consensus.cc:2798] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:16.739114 15390 raft_consensus.cc:491] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:16.739151 15390 raft_consensus.cc:3054] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.739995 15390 raft_consensus.cc:513] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:16.740036 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0f20bb66f36647e0811d0865b8f414f7 consensus state has no leader
I20250115 03:06:16.740079 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet 20f86f93812849c4b7852a884f677c04 with delete_type TABLET_DATA_DELETED (replaced by tablet b08e5fc2110e476e897240f9cf3b874d) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.740792 14812 tablet_bootstrap.cc:492] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.740865 14812 ts_tablet_manager.cc:1397] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.044s user 0.000s sys 0.002s
I20250115 03:06:16.741724 14812 raft_consensus.cc:357] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.741816 14812 raft_consensus.cc:383] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.741847 14812 raft_consensus.cc:738] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.741950 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8b90f854a5e4221a24986f8c71ca300" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:16.741977 14812 consensus_queue.cc:260] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.742043 14687 raft_consensus.cc:3054] T f8b90f854a5e4221a24986f8c71ca300 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.742589 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8b90f854a5e4221a24986f8c71ca300" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:16.742733 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7c503b29cb2843338fb1870a89aee443 consensus state has no leader
W20250115 03:06:16.742939 14356 leader_election.cc:343] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.742949 14687 raft_consensus.cc:2463] T f8b90f854a5e4221a24986f8c71ca300 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
W20250115 03:06:16.743645 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c35f2414d284756b9c93ed61e75d451 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.744112 15390 leader_election.cc:290] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.744338 14171 catalog_manager.cc:6596] ReplaceTablet: tablet 20f86f93812849c4b7852a884f677c04 deleted and replaced by tablet b08e5fc2110e476e897240f9cf3b874d
I20250115 03:06:16.744640 14812 ts_tablet_manager.cc:1428] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20250115 03:06:16.744796 14812 tablet_bootstrap.cc:492] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.745381 14358 leader_election.cc:304] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:16.745635 14885 raft_consensus.cc:2798] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:16.745698 14885 raft_consensus.cc:695] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:16.745774 14812 tablet_bootstrap.cc:654] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.745790 14885 consensus_queue.cc:237] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.746474 14820 ts_tablet_manager.cc:1918] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.746532 14820 log.cc:1198] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/8baf86b37e9942ba813b099a0da87ca5
I20250115 03:06:16.746716 14820 ts_tablet_manager.cc:1939] T 8baf86b37e9942ba813b099a0da87ca5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.746938 14811 tablet_bootstrap.cc:492] T b1c0f6a5611647eb884c77f7a6d1cd4e P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.747014 14811 ts_tablet_manager.cc:1397] T b1c0f6a5611647eb884c77f7a6d1cd4e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20250115 03:06:16.747414 14811 raft_consensus.cc:357] T b1c0f6a5611647eb884c77f7a6d1cd4e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.747524 14811 raft_consensus.cc:383] T b1c0f6a5611647eb884c77f7a6d1cd4e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.747557 14811 raft_consensus.cc:738] T b1c0f6a5611647eb884c77f7a6d1cd4e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.747664 14811 consensus_queue.cc:260] T b1c0f6a5611647eb884c77f7a6d1cd4e P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.747910 14811 ts_tablet_manager.cc:1428] T b1c0f6a5611647eb884c77f7a6d1cd4e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:16.748062 14811 tablet_bootstrap.cc:492] T 82be772e05d443749914004aea8af300 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.748961 14811 tablet_bootstrap.cc:654] T 82be772e05d443749914004aea8af300 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.750325 15453 raft_consensus.cc:491] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.750397 15453 raft_consensus.cc:513] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.750687 15453 leader_election.cc:290] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:16.751199 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeff2cd0825c4419aacfc73ebe267a8e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:16.751562 14626 leader_election.cc:343] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.751719 14163 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 304fab45b38e4426ae6961ae786e5659 (table default.Table_d407111658854f02aa8387a067f33f37 [id=526370a40a0d4f7aa96c8348412d8b26]). Expected version 1 got 0
I20250115 03:06:16.752048 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeff2cd0825c4419aacfc73ebe267a8e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:16.752144 14421 raft_consensus.cc:2463] T eeff2cd0825c4419aacfc73ebe267a8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:16.752480 14167 catalog_manager.cc:5526] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:16.753261 14628 leader_election.cc:304] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:16.753660 15453 raft_consensus.cc:2798] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:16.753710 15453 raft_consensus.cc:491] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:16.753754 15453 raft_consensus.cc:3054] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.754566 15453 raft_consensus.cc:513] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.754799 15857 raft_consensus.cc:491] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.754884 15453 leader_election.cc:290] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:16.754874 15857 raft_consensus.cc:513] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.755148 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeff2cd0825c4419aacfc73ebe267a8e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:16.755345 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 20f86f93812849c4b7852a884f677c04 with delete_type TABLET_DATA_DELETED (replaced by tablet b08e5fc2110e476e897240f9cf3b874d) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.755522 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeff2cd0825c4419aacfc73ebe267a8e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:16.755613 14422 raft_consensus.cc:3054] T eeff2cd0825c4419aacfc73ebe267a8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.755633 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 4122c97f5ba445ee8c79b7d056438229 (DEFAULT_TABLE table=default.Table_b169e386c2cc424da59a39503cec548f [id=2b97c2f5b5284cd8a6311247712cc447]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.755633 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 4122c97f5ba445ee8c79b7d056438229 (DEFAULT_TABLE table=default.Table_b169e386c2cc424da59a39503cec548f [id=2b97c2f5b5284cd8a6311247712cc447]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.755853 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4122c97f5ba445ee8c79b7d056438229. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.755853 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4122c97f5ba445ee8c79b7d056438229. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.756474 14422 raft_consensus.cc:2463] T eeff2cd0825c4419aacfc73ebe267a8e P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:16.756807 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 20f86f93812849c4b7852a884f677c04 with delete_type TABLET_DATA_DELETED (replaced by tablet b08e5fc2110e476e897240f9cf3b874d) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:16.757078 14626 leader_election.cc:343] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.757737 15857 leader_election.cc:290] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.757756 14398 tablet_service.cc:1467] Processing CreateTablet for tablet ad0e9216bd154f4a8a37874d3d8a08c2 (DEFAULT_TABLE table=default.Table_10e94765088044309cd6f627ab7a293b [id=0be45947ea87476db5c30b5c03f7605a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.757838 14663 tablet_service.cc:1467] Processing CreateTablet for tablet bfeafbf380df41738ebbcc11c19a7f85 (DEFAULT_TABLE table=default.Table_9194fcb0d915415ea9220cc06cb2a37d [id=03ee00da1e024ebab51b3a366d2c0cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.757969 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad0e9216bd154f4a8a37874d3d8a08c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.757990 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa08518530ae4d5487af386a650c4672" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:16.758040 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfeafbf380df41738ebbcc11c19a7f85. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.758095 14687 raft_consensus.cc:2388] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in current term 1: Already voted for candidate e8df7247ee9c43e0aa18216da2661f75 in this term.
I20250115 03:06:16.758523 14665 tablet_service.cc:1467] Processing CreateTablet for tablet d6cf1a0b9c81413b8f72de0cafc76963 (DEFAULT_TABLE table=default.Table_e51b0b04bb044b36be347afe90845b14 [id=17ea0c263011486f82895eb86b21fe78]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.758720 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6cf1a0b9c81413b8f72de0cafc76963. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.759950 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 38f5554b10a44c249e4c3d3825cd7952 (DEFAULT_TABLE table=default.Table_4f083ad1bef14359a5e1713d9dabbdca [id=85888c016e3d4dd1915bbef8b2d8fd3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.760052 14662 tablet_service.cc:1467] Processing CreateTablet for tablet bb68869eaf55477090cf12f40f76a89c (DEFAULT_TABLE table=default.Table_0964201dda974eaabef3d5694c40389c [id=4a6fefd2006a4b11b121bfad61cf95ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.760150 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f5554b10a44c249e4c3d3825cd7952. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.760246 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb68869eaf55477090cf12f40f76a89c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.760892 14664 tablet_service.cc:1467] Processing CreateTablet for tablet ad0e9216bd154f4a8a37874d3d8a08c2 (DEFAULT_TABLE table=default.Table_10e94765088044309cd6f627ab7a293b [id=0be45947ea87476db5c30b5c03f7605a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.761114 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad0e9216bd154f4a8a37874d3d8a08c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.762075 14397 tablet_service.cc:1467] Processing CreateTablet for tablet bfeafbf380df41738ebbcc11c19a7f85 (DEFAULT_TABLE table=default.Table_9194fcb0d915415ea9220cc06cb2a37d [id=03ee00da1e024ebab51b3a366d2c0cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.762331 14628 leader_election.cc:304] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:16.762567 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa08518530ae4d5487af386a650c4672" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:16.763423 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfeafbf380df41738ebbcc11c19a7f85. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.763876 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet df44c5f5a9ae4d1aa7647dcbe2303a39 (table default.Table_098c9884c0494d4592f142b4a197f38c [id=8bf1da8d633940258affd0e11c334786]). Expected version 1 got 0
I20250115 03:06:16.764019 14163 catalog_manager.cc:5526] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:16.764128 15390 raft_consensus.cc:491] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.764168 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 745344db22da4102bad39c2f121548a1 (table default.Table_365313892043408c92be8460c31d958e [id=f22f08e9af4a4496b100e7d4b5982121]). Expected version 1 got 0
I20250115 03:06:16.764194 15390 raft_consensus.cc:513] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.764277 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet eeff2cd0825c4419aacfc73ebe267a8e (table default.Table_590b26746c6a4006bb430ba9dcbacafc [id=021f5685630741a8880d2471f7d2d1d2]). Expected version 1 got 0
I20250115 03:06:16.764786 15390 leader_election.cc:290] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.765524 14493 leader_election.cc:304] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.765576 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 4122c97f5ba445ee8c79b7d056438229 (DEFAULT_TABLE table=default.Table_b169e386c2cc424da59a39503cec548f [id=2b97c2f5b5284cd8a6311247712cc447]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.765709 14534 tablet_service.cc:1467] Processing CreateTablet for tablet d6cf1a0b9c81413b8f72de0cafc76963 (DEFAULT_TABLE table=default.Table_e51b0b04bb044b36be347afe90845b14 [id=17ea0c263011486f82895eb86b21fe78]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.765784 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4122c97f5ba445ee8c79b7d056438229. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.765965 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6cf1a0b9c81413b8f72de0cafc76963. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.766261 14532 tablet_service.cc:1467] Processing CreateTablet for tablet ad0e9216bd154f4a8a37874d3d8a08c2 (DEFAULT_TABLE table=default.Table_10e94765088044309cd6f627ab7a293b [id=0be45947ea87476db5c30b5c03f7605a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.766446 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad0e9216bd154f4a8a37874d3d8a08c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.766846 15453 raft_consensus.cc:2798] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:16.766923 15453 raft_consensus.cc:695] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:16.767030 15453 consensus_queue.cc:237] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.767596 14528 tablet_service.cc:1467] Processing CreateTablet for tablet 38f5554b10a44c249e4c3d3825cd7952 (DEFAULT_TABLE table=default.Table_4f083ad1bef14359a5e1713d9dabbdca [id=85888c016e3d4dd1915bbef8b2d8fd3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.767799 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f5554b10a44c249e4c3d3825cd7952. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.768241 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 38f5554b10a44c249e4c3d3825cd7952 (DEFAULT_TABLE table=default.Table_4f083ad1bef14359a5e1713d9dabbdca [id=85888c016e3d4dd1915bbef8b2d8fd3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.768311 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_cd69692684cd401a9c6f9d24e374582c" } new_table_name: "default.Table_aea3e4039c704eaba082b3b109362b93" modify_external_catalogs: true
I20250115 03:06:16.768432 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38f5554b10a44c249e4c3d3825cd7952. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.768993 15857 raft_consensus.cc:3054] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.769685 14529 tablet_service.cc:1467] Processing CreateTablet for tablet bb68869eaf55477090cf12f40f76a89c (DEFAULT_TABLE table=default.Table_0964201dda974eaabef3d5694c40389c [id=4a6fefd2006a4b11b121bfad61cf95ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.769799 15857 raft_consensus.cc:2743] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.769878 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb68869eaf55477090cf12f40f76a89c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.770128 15325 raft_consensus.cc:491] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.770195 15325 raft_consensus.cc:513] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.770468 15325 leader_election.cc:290] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:16.772238 14820 tablet_replica.cc:331] T 79579515fb7e45a8b16fd36dbc6a965a P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.772346 14820 raft_consensus.cc:2238] T 79579515fb7e45a8b16fd36dbc6a965a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.772511 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 8baf86b37e9942ba813b099a0da87ca5 (table default.Table_a1547d1d923f4f4da05aaeb519f4cdf0 [id=e929ffaba567498c815393836b264a12]) successfully deleted
I20250115 03:06:16.772521 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "304fab45b38e4426ae6961ae786e5659" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.772634 14555 raft_consensus.cc:2463] T 304fab45b38e4426ae6961ae786e5659 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:16.772687 14396 tablet_service.cc:1467] Processing CreateTablet for tablet bb68869eaf55477090cf12f40f76a89c (DEFAULT_TABLE table=default.Table_0964201dda974eaabef3d5694c40389c [id=4a6fefd2006a4b11b121bfad61cf95ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.772899 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb68869eaf55477090cf12f40f76a89c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.772914 14397 tablet_service.cc:1467] Processing CreateTablet for tablet d6cf1a0b9c81413b8f72de0cafc76963 (DEFAULT_TABLE table=default.Table_e51b0b04bb044b36be347afe90845b14 [id=17ea0c263011486f82895eb86b21fe78]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.773986 14356 leader_election.cc:304] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.774233 14861 raft_consensus.cc:2798] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:16.774284 14861 raft_consensus.cc:491] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:16.774320 14861 raft_consensus.cc:3054] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.774775 14399 tablet_service.cc:1467] Processing CreateTablet for tablet b2629ae9d9854ae4b6e7ddddbb6f341b (DEFAULT_TABLE table=default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.774946 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df44c5f5a9ae4d1aa7647dcbe2303a39" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.775036 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_478e448d8f6d494d90d421e2f0cd21fc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:16.775115 14861 raft_consensus.cc:513] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.775089 14167 catalog_manager.cc:5526] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:16.775256 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet eeff2cd0825c4419aacfc73ebe267a8e (table default.Table_590b26746c6a4006bb430ba9dcbacafc [id=021f5685630741a8880d2471f7d2d1d2]). Expected version 1 got 0
W20250115 03:06:16.775334 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_478e448d8f6d494d90d421e2f0cd21fc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.775334 14537 tablet_service.cc:1467] Processing CreateTablet for tablet bfeafbf380df41738ebbcc11c19a7f85 (DEFAULT_TABLE table=default.Table_9194fcb0d915415ea9220cc06cb2a37d [id=03ee00da1e024ebab51b3a366d2c0cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.775399 14861 leader_election.cc:290] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.775533 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfeafbf380df41738ebbcc11c19a7f85. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:16.775750 14626 leader_election.cc:343] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.776077 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "304fab45b38e4426ae6961ae786e5659" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:16.776160 14556 raft_consensus.cc:3054] T 304fab45b38e4426ae6961ae786e5659 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.776154 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "304fab45b38e4426ae6961ae786e5659" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:16.776247 14689 raft_consensus.cc:3054] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.776957 14556 raft_consensus.cc:2463] T 304fab45b38e4426ae6961ae786e5659 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:16.777123 14689 raft_consensus.cc:2463] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:16.777225 14356 leader_election.cc:304] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.777467 15390 raft_consensus.cc:2798] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:16.777556 15390 raft_consensus.cc:695] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:16.777663 15390 consensus_queue.cc:237] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.777786 15401 consensus_queue.cc:1035] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.772424 14820 raft_consensus.cc:2267] T 79579515fb7e45a8b16fd36dbc6a965a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.778843 14820 ts_tablet_manager.cc:1905] T 79579515fb7e45a8b16fd36dbc6a965a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.779372 14823 tablet_replica.cc:331] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.779747 14823 raft_consensus.cc:2238] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:16.779919 14823 raft_consensus.cc:2267] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.780865 14823 ts_tablet_manager.cc:1905] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.782162 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df44c5f5a9ae4d1aa7647dcbe2303a39" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:16.782265 14421 raft_consensus.cc:2463] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:16.782296 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6cf1a0b9c81413b8f72de0cafc76963. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.783808 14532 tablet_service.cc:1467] Processing CreateTablet for tablet b2629ae9d9854ae4b6e7ddddbb6f341b (DEFAULT_TABLE table=default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.784018 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2629ae9d9854ae4b6e7ddddbb6f341b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.784376 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2629ae9d9854ae4b6e7ddddbb6f341b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.785859 14535 tablet_service.cc:1467] Processing CreateTablet for tablet dda418ed197e4af2aa511d3f1851663e (DEFAULT_TABLE table=default.Table_204aaf913d224b93adf8ce86999572c7 [id=cbab87c958a346199acdad65ab75be3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.786609 14628 leader_election.cc:304] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:16.786978 15153 raft_consensus.cc:2798] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:16.787038 15153 raft_consensus.cc:491] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:16.787078 15153 raft_consensus.cc:3054] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.787565 15390 consensus_queue.cc:1035] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.787883 15153 raft_consensus.cc:513] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.788180 15153 leader_election.cc:290] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:16.788806 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df44c5f5a9ae4d1aa7647dcbe2303a39" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:16.789775 14820 ts_tablet_manager.cc:1918] T 79579515fb7e45a8b16fd36dbc6a965a P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.789839 14820 log.cc:1198] T 79579515fb7e45a8b16fd36dbc6a965a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/79579515fb7e45a8b16fd36dbc6a965a
I20250115 03:06:16.789978 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df44c5f5a9ae4d1aa7647dcbe2303a39" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:16.790030 14820 ts_tablet_manager.cc:1939] T 79579515fb7e45a8b16fd36dbc6a965a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.790062 14421 raft_consensus.cc:3054] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:16.790396 14626 leader_election.cc:343] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.790833 14421 raft_consensus.cc:2463] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:16.791028 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dda418ed197e4af2aa511d3f1851663e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.792959 14603 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:16.793078 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "304fab45b38e4426ae6961ae786e5659" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:16.793171 14691 raft_consensus.cc:2371] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1. Re-sending same reply.
I20250115 03:06:16.793287 14858 raft_consensus.cc:491] T 6864f88bf6454ee980f624ec2f2d9fb6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.793354 14858 raft_consensus.cc:513] T 6864f88bf6454ee980f624ec2f2d9fb6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.793648 14858 leader_election.cc:290] T 6864f88bf6454ee980f624ec2f2d9fb6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.794401 14662 tablet_service.cc:1467] Processing CreateTablet for tablet dda418ed197e4af2aa511d3f1851663e (DEFAULT_TABLE table=default.Table_204aaf913d224b93adf8ce86999572c7 [id=cbab87c958a346199acdad65ab75be3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.794615 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dda418ed197e4af2aa511d3f1851663e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.794728 14861 consensus_queue.cc:1035] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.794972 14665 tablet_service.cc:1467] Processing CreateTablet for tablet b2629ae9d9854ae4b6e7ddddbb6f341b (DEFAULT_TABLE table=default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.795157 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2629ae9d9854ae4b6e7ddddbb6f341b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.795400 15325 raft_consensus.cc:491] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.795468 15325 raft_consensus.cc:513] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.795768 15325 leader_election.cc:290] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:16.796838 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 341180618b4748eabc568888e2e6a7df with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.797175 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6864f88bf6454ee980f624ec2f2d9fb6" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:16.797293 14421 raft_consensus.cc:2388] T 6864f88bf6454ee980f624ec2f2d9fb6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in current term 1: Already voted for candidate 5c0dae7aff4b44c699275b95ba639770 in this term.
I20250115 03:06:16.797307 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed8538c4a44d18279371b826c22a5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:16.798417 14626 leader_election.cc:343] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.798434 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed8538c4a44d18279371b826c22a5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:16.798619 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6864f88bf6454ee980f624ec2f2d9fb6" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:16.798831 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet b0a5d554764249f8a7eefdc943271090 with delete_type TABLET_DATA_DELETED (replaced by tablet d895f75911b7456a88e7a5159db7d60b) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.798987 14628 leader_election.cc:304] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
W20250115 03:06:16.799196 14628 leader_election.cc:343] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.799242 14628 leader_election.cc:304] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.799378 15325 raft_consensus.cc:491] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.799420 15401 consensus_queue.cc:1035] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.799433 15325 raft_consensus.cc:513] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.799716 15325 leader_election.cc:290] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:16.799800 15325 raft_consensus.cc:2798] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:16.799863 15325 raft_consensus.cc:695] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:16.799968 15325 consensus_queue.cc:237] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.800213 14397 tablet_service.cc:1467] Processing CreateTablet for tablet dda418ed197e4af2aa511d3f1851663e (DEFAULT_TABLE table=default.Table_204aaf913d224b93adf8ce86999572c7 [id=cbab87c958a346199acdad65ab75be3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.800423 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dda418ed197e4af2aa511d3f1851663e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.800521 15325 raft_consensus.cc:2743] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.800753 14557 raft_consensus.cc:1270] T e4f688e4c40d4653929432b13508348a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:16.801976 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "341180618b4748eabc568888e2e6a7df" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:16.802122 14688 raft_consensus.cc:1270] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250115 03:06:16.802284 14628 leader_election.cc:343] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.802924 14861 consensus_queue.cc:1035] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.803334 14167 catalog_manager.cc:5526] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:16.803963 14167 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet f8b90f854a5e4221a24986f8c71ca300 (table default.Table_4e74d34a621f4b1898241f62a20ffc10 [id=d8084de64a264d28a3370971617a694c]). Expected version 2 got 0
I20250115 03:06:16.804139 14167 catalog_manager.cc:5526] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:16.804366 14167 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 304fab45b38e4426ae6961ae786e5659 (table default.Table_d407111658854f02aa8387a067f33f37 [id=526370a40a0d4f7aa96c8348412d8b26]). Expected version 1 got 0
I20250115 03:06:16.805042 14819 ts_tablet_manager.cc:1918] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.805105 14819 log.cc:1198] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/33380937a3b7428995b06e477a896c02
I20250115 03:06:16.805289 14819 ts_tablet_manager.cc:1939] T 33380937a3b7428995b06e477a896c02 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.805379 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "341180618b4748eabc568888e2e6a7df" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.806295 14823 ts_tablet_manager.cc:1918] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:16.806366 14823 log.cc:1198] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/e368f87556944c46a6d8f160ead17ac9
I20250115 03:06:16.806599 14823 ts_tablet_manager.cc:1939] T e368f87556944c46a6d8f160ead17ac9 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
W20250115 03:06:16.806746 14626 leader_election.cc:343] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.806809 14626 leader_election.cc:304] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.806922 15453 raft_consensus.cc:2743] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.808125 14493 leader_election.cc:304] T 6864f88bf6454ee980f624ec2f2d9fb6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.808313 14858 raft_consensus.cc:3054] T 6864f88bf6454ee980f624ec2f2d9fb6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.809175 14858 raft_consensus.cc:2743] T 6864f88bf6454ee980f624ec2f2d9fb6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.809965 14823 tablet_replica.cc:331] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.810096 14823 raft_consensus.cc:2238] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.810672 14823 raft_consensus.cc:2267] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.811390 14858 raft_consensus.cc:491] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.811396 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet e368f87556944c46a6d8f160ead17ac9 (table default.Table_2af92748648345c0903a0f99da0d47c5 [id=19efb3734d824699a088ad62e69095f3]) successfully deleted
I20250115 03:06:16.811462 14858 raft_consensus.cc:513] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.811528 14823 ts_tablet_manager.cc:1905] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:16.811565 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 1f2dc6928e2941ef9b5b5a7891376500 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f2dc6928e2941ef9b5b5a7891376500 already in progress: creating tablet
I20250115 03:06:16.811546 14166 catalog_manager.cc:5526] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:16.811671 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 33380937a3b7428995b06e477a896c02 (table default.Table_90aedfcdb1274e1497cf36620b980768 [id=9e4f5de45eb24beabef550d15dcad8bf]) successfully deleted
I20250115 03:06:16.811710 14166 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet df44c5f5a9ae4d1aa7647dcbe2303a39 (table default.Table_098c9884c0494d4592f142b4a197f38c [id=8bf1da8d633940258affd0e11c334786]). Expected version 1 got 0
I20250115 03:06:16.806607 14809 tablet_bootstrap.cc:492] T 3a0bb9b5093b454d921bf1150d631565 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.811766 14858 leader_election.cc:290] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.811811 14166 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet e4f688e4c40d4653929432b13508348a (table default.Table_aea3e4039c704eaba082b3b109362b93 [id=5beb8c6fadc742c4bd0b5171ed51d0e2]). Expected version 1 got 0
I20250115 03:06:16.811844 14809 ts_tablet_manager.cc:1397] T 3a0bb9b5093b454d921bf1150d631565 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.091s user 0.000s sys 0.002s
I20250115 03:06:16.812268 14809 raft_consensus.cc:357] T 3a0bb9b5093b454d921bf1150d631565 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.812371 14809 raft_consensus.cc:383] T 3a0bb9b5093b454d921bf1150d631565 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.812407 14809 raft_consensus.cc:738] T 3a0bb9b5093b454d921bf1150d631565 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.812518 14809 consensus_queue.cc:260] T 3a0bb9b5093b454d921bf1150d631565 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.812783 14809 ts_tablet_manager.cc:1428] T 3a0bb9b5093b454d921bf1150d631565 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.812961 14809 tablet_bootstrap.cc:492] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
W20250115 03:06:16.813692 14493 leader_election.cc:343] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 84ce55f729224f78b420e57977419add
I20250115 03:06:16.813977 14809 tablet_bootstrap.cc:654] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:16.814208 14493 leader_election.cc:343] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 84ce55f729224f78b420e57977419add
I20250115 03:06:16.814266 14493 leader_election.cc:304] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.814479 14858 raft_consensus.cc:2743] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.807337 14861 consensus_queue.cc:1035] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20250115 03:06:16.806281 14819 tablet_replica.cc:331] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.815008 15390 consensus_queue.cc:1035] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.815315 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 79579515fb7e45a8b16fd36dbc6a965a (table default.Table_b1775fbfc0eb4cf4a59a0f9a853a081e [id=ed57b2cba1db497fa0af4bba35580a35]) successfully deleted
I20250115 03:06:16.815807 15617 tablet.cc:1721] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:16.818511 15274 tablet.cc:1721] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:16.819804 14812 tablet_bootstrap.cc:492] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.819893 14812 ts_tablet_manager.cc:1397] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.075s user 0.001s sys 0.000s
I20250115 03:06:16.820312 14812 raft_consensus.cc:357] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.820407 14812 raft_consensus.cc:383] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.820438 14812 raft_consensus.cc:738] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.820556 14812 consensus_queue.cc:260] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.820688 14820 tablet_replica.cc:331] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.820816 14820 raft_consensus.cc:2238] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.820825 14812 ts_tablet_manager.cc:1428] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.821002 14812 tablet_bootstrap.cc:492] T d0c3536533e94174bb1e3888a3458395 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.820998 14820 raft_consensus.cc:2267] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.821501 14820 ts_tablet_manager.cc:1905] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.821882 14812 tablet_bootstrap.cc:654] T d0c3536533e94174bb1e3888a3458395 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.822211 14819 raft_consensus.cc:2238] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.822316 14819 raft_consensus.cc:2267] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.824503 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet e4f688e4c40d4653929432b13508348a (table default.Table_aea3e4039c704eaba082b3b109362b93 [id=5beb8c6fadc742c4bd0b5171ed51d0e2]). Expected version 1 got 0
I20250115 03:06:16.826634 14811 tablet_bootstrap.cc:492] T 82be772e05d443749914004aea8af300 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.826718 14811 ts_tablet_manager.cc:1397] T 82be772e05d443749914004aea8af300 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.079s user 0.002s sys 0.000s
I20250115 03:06:16.827136 14811 raft_consensus.cc:357] T 82be772e05d443749914004aea8af300 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.827234 14811 raft_consensus.cc:383] T 82be772e05d443749914004aea8af300 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.827263 14811 raft_consensus.cc:738] T 82be772e05d443749914004aea8af300 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.827373 14811 consensus_queue.cc:260] T 82be772e05d443749914004aea8af300 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.827627 14811 ts_tablet_manager.cc:1428] T 82be772e05d443749914004aea8af300 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.827790 14811 tablet_bootstrap.cc:492] T a1c93bfd3da44f9f8a8d585f2d21f1be P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.829433 14861 consensus_queue.cc:1035] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.831250 14738 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:16.831782 14811 tablet_bootstrap.cc:654] T a1c93bfd3da44f9f8a8d585f2d21f1be P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.832546 15153 raft_consensus.cc:491] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.832618 15153 raft_consensus.cc:513] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.832767 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet 82be772e05d443749914004aea8af300 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.833675 15153 leader_election.cc:290] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:16.833998 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6f61dc685a48bdb367278773bf1d90" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.834582 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6f61dc685a48bdb367278773bf1d90" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:16.834997 14626 leader_election.cc:343] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:16.835227 14628 leader_election.cc:343] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.835286 14628 leader_election.cc:304] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.835573 15153 raft_consensus.cc:2743] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.836498 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_aea3e4039c704eaba082b3b109362b93 [id=5beb8c6fadc742c4bd0b5171ed51d0e2] alter complete (version 1)
I20250115 03:06:16.840273 14809 tablet_bootstrap.cc:492] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.840358 14809 ts_tablet_manager.cc:1397] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.000s
I20250115 03:06:16.840793 14809 raft_consensus.cc:357] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.840888 14809 raft_consensus.cc:383] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.840917 14809 raft_consensus.cc:738] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.841034 14809 consensus_queue.cc:260] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.841274 14820 ts_tablet_manager.cc:1918] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.841284 14809 ts_tablet_manager.cc:1428] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.841452 14809 tablet_bootstrap.cc:492] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.842387 14809 tablet_bootstrap.cc:654] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.842409 15453 raft_consensus.cc:491] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.842442 14823 ts_tablet_manager.cc:1918] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:16.842478 15453 raft_consensus.cc:513] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.842775 15453 leader_election.cc:290] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:16.844095 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58161a322a3d469c8444d353689bed65" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:16.845759 14628 leader_election.cc:343] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.847407 14819 ts_tablet_manager.cc:1905] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.847471 14823 log.cc:1198] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/20f86f93812849c4b7852a884f677c04
I20250115 03:06:16.847733 14823 ts_tablet_manager.cc:1939] T 20f86f93812849c4b7852a884f677c04 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.843125 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58161a322a3d469c8444d353689bed65" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.848170 14820 log.cc:1198] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/7f67cbb929cd4ae589a14606aa168b46
W20250115 03:06:16.848233 14626 leader_election.cc:343] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.848304 14626 leader_election.cc:304] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.848296 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 20f86f93812849c4b7852a884f677c04 (table default.Table_50c659a42a154d46a26879df0101bf23 [id=f8f2a7ff6fdc40468de6ae9ba28c1427]) successfully deleted
I20250115 03:06:16.848426 15453 raft_consensus.cc:2743] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.849133 14820 ts_tablet_manager.cc:1939] T 7f67cbb929cd4ae589a14606aa168b46 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.849691 14820 tablet_replica.cc:331] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.849937 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 7f67cbb929cd4ae589a14606aa168b46 (table default.Table_a7ef00d45deb49578042744e5f66f14e [id=29daeed94f414e5abdb17e05a26b7516]) successfully deleted
I20250115 03:06:16.850142 14820 raft_consensus.cc:2238] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.850239 14812 tablet_bootstrap.cc:492] T d0c3536533e94174bb1e3888a3458395 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.850320 14812 ts_tablet_manager.cc:1397] T d0c3536533e94174bb1e3888a3458395 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20250115 03:06:16.850721 14820 raft_consensus.cc:2267] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.850967 14812 raft_consensus.cc:357] T d0c3536533e94174bb1e3888a3458395 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.851069 14812 raft_consensus.cc:383] T d0c3536533e94174bb1e3888a3458395 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.851104 14812 raft_consensus.cc:738] T d0c3536533e94174bb1e3888a3458395 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.851215 14812 consensus_queue.cc:260] T d0c3536533e94174bb1e3888a3458395 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.851538 14812 ts_tablet_manager.cc:1428] T d0c3536533e94174bb1e3888a3458395 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.851588 14820 ts_tablet_manager.cc:1905] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.851698 14812 tablet_bootstrap.cc:492] T 6110b6c4911d4932b4cac4d86598a5c6 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.852653 14812 tablet_bootstrap.cc:654] T 6110b6c4911d4932b4cac4d86598a5c6 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.854641 14820 ts_tablet_manager.cc:1918] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.855168 14809 tablet_bootstrap.cc:492] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.855243 14809 ts_tablet_manager.cc:1397] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.002s
I20250115 03:06:16.855647 14809 raft_consensus.cc:357] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.855736 14809 raft_consensus.cc:383] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.855764 14809 raft_consensus.cc:738] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.855778 14819 ts_tablet_manager.cc:1918] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.855827 14819 log.cc:1198] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/acb43d614111477abb10def48a1dfb3b
I20250115 03:06:16.855872 14809 consensus_queue.cc:260] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.856124 14809 ts_tablet_manager.cc:1428] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.856284 14809 tablet_bootstrap.cc:492] T a75b9401cb4b4f8b92a4779c3704eb46 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.857185 14809 tablet_bootstrap.cc:654] T a75b9401cb4b4f8b92a4779c3704eb46 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.858006 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a0bb9b5093b454d921bf1150d631565 with delete_type TABLET_DATA_DELETED (replaced by tablet 5589b497b0e24740ad49d1392f5a737b) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.858142 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_5577048ebf95456d9cbe74d11b84f131"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:16.858237 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_73f837db71d74dae9606eb6995ad7270"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.858497 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5577048ebf95456d9cbe74d11b84f131 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:16.858677 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_73f837db71d74dae9606eb6995ad7270 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.859051 14823 tablet_replica.cc:331] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.859181 14823 raft_consensus.cc:2238] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.859273 14823 raft_consensus.cc:2267] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.859490 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_268ae5920054489fa49b7ce5b1569532"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.859771 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_268ae5920054489fa49b7ce5b1569532 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.860633 14819 ts_tablet_manager.cc:1939] T acb43d614111477abb10def48a1dfb3b P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.861316 14823 ts_tablet_manager.cc:1905] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.861598 15453 raft_consensus.cc:491] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.861668 15453 raft_consensus.cc:513] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.861966 15453 leader_election.cc:290] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:16.860783 14811 tablet_bootstrap.cc:492] T a1c93bfd3da44f9f8a8d585f2d21f1be P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.863677 14811 ts_tablet_manager.cc:1397] T a1c93bfd3da44f9f8a8d585f2d21f1be P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.036s user 0.002s sys 0.000s
I20250115 03:06:16.864092 14811 raft_consensus.cc:357] T a1c93bfd3da44f9f8a8d585f2d21f1be P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.864180 14811 raft_consensus.cc:383] T a1c93bfd3da44f9f8a8d585f2d21f1be P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.864205 14811 raft_consensus.cc:738] T a1c93bfd3da44f9f8a8d585f2d21f1be P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.864295 14811 consensus_queue.cc:260] T a1c93bfd3da44f9f8a8d585f2d21f1be P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.864542 14811 ts_tablet_manager.cc:1428] T a1c93bfd3da44f9f8a8d585f2d21f1be P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.864699 14811 tablet_bootstrap.cc:492] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.865494 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2e3e4b87e2e04feaab40ea415807cd06"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:16.865645 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_4f5d0494ec384c479419c10b7ff36dc0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.865789 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e3e4b87e2e04feaab40ea415807cd06 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:16.865919 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f5d0494ec384c479419c10b7ff36dc0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.866350 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b9e69a84b955444488aff2ca4794feca" } new_table_name: "default.Table_e68b21fe987c4ea49523f24440585fce" modify_external_catalogs: true
I20250115 03:06:16.866456 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_afc5c7e955fb4b3ba43151cb8823a76b" } modify_external_catalogs: true
I20250115 03:06:16.866545 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_afc5c7e955fb4b3ba43151cb8823a76b" } modify_external_catalogs: true
I20250115 03:06:16.866775 14823 ts_tablet_manager.cc:1918] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.866829 14823 log.cc:1198] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/341180618b4748eabc568888e2e6a7df
I20250115 03:06:16.862955 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bffef32339a4aa6b2ef7fb5661d035c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.867007 14823 ts_tablet_manager.cc:1939] T 341180618b4748eabc568888e2e6a7df P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.863140 15426 tablet.cc:1721] T e4f688e4c40d4653929432b13508348a P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:16.868031 14811 tablet_bootstrap.cc:654] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.868553 14820 log.cc:1198] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/3cbc91ad00b54ede804c3629bcfe7fe6
I20250115 03:06:16.868743 14820 ts_tablet_manager.cc:1939] T 3cbc91ad00b54ede804c3629bcfe7fe6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.869026 14820 tablet_replica.cc:331] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
W20250115 03:06:16.869019 14626 leader_election.cc:343] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.869133 14820 raft_consensus.cc:2238] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.869220 14820 raft_consensus.cc:2267] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.869237 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 3cbc91ad00b54ede804c3629bcfe7fe6 (table default.Table_271f46df55f14ffd9b7bb48352620543 [id=44488be7335b488a9ff7cf913ce4273a]) successfully deleted
I20250115 03:06:16.869256 14169 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet e4f688e4c40d4653929432b13508348a (table default.Table_aea3e4039c704eaba082b3b109362b93 [id=5beb8c6fadc742c4bd0b5171ed51d0e2]). Expected version 1 got 0
I20250115 03:06:16.872702 14820 ts_tablet_manager.cc:1905] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.873448 14812 tablet_bootstrap.cc:492] T 6110b6c4911d4932b4cac4d86598a5c6 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.873539 14812 ts_tablet_manager.cc:1397] T 6110b6c4911d4932b4cac4d86598a5c6 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.001s
I20250115 03:06:16.873955 14812 raft_consensus.cc:357] T 6110b6c4911d4932b4cac4d86598a5c6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.874058 14812 raft_consensus.cc:383] T 6110b6c4911d4932b4cac4d86598a5c6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.874095 14812 raft_consensus.cc:738] T 6110b6c4911d4932b4cac4d86598a5c6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.874209 14812 consensus_queue.cc:260] T 6110b6c4911d4932b4cac4d86598a5c6 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.874487 14812 ts_tablet_manager.cc:1428] T 6110b6c4911d4932b4cac4d86598a5c6 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.874655 14812 tablet_bootstrap.cc:492] T b6175f691fec45bf9853ee90959a1c98 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.876785 14812 tablet_bootstrap.cc:654] T b6175f691fec45bf9853ee90959a1c98 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.876747 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4464f446819641ebab4c3f23f0632251" } new_table_name: "default.Table_57fbe77148284b4a9666630570f69321" modify_external_catalogs: true
I20250115 03:06:16.877830 14820 ts_tablet_manager.cc:1918] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.877938 14820 log.cc:1198] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/84ce55f729224f78b420e57977419add
I20250115 03:06:16.878162 14820 ts_tablet_manager.cc:1939] T 84ce55f729224f78b420e57977419add P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.879278 14812 tablet_bootstrap.cc:492] T b6175f691fec45bf9853ee90959a1c98 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:16.879360 14812 ts_tablet_manager.cc:1397] T b6175f691fec45bf9853ee90959a1c98 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20250115 03:06:16.862326 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bffef32339a4aa6b2ef7fb5661d035c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:16.868492 14468 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:16.879765 14812 raft_consensus.cc:357] T b6175f691fec45bf9853ee90959a1c98 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.879865 14812 raft_consensus.cc:383] T b6175f691fec45bf9853ee90959a1c98 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.879900 14812 raft_consensus.cc:738] T b6175f691fec45bf9853ee90959a1c98 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.879969 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 84ce55f729224f78b420e57977419add (table default.Table_ff21d7bc463d43cfa3330f7288ba7a55 [id=5de066e9f2174fff9303cba23eeec988]) successfully deleted
I20250115 03:06:16.879995 14812 consensus_queue.cc:260] T b6175f691fec45bf9853ee90959a1c98 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.880229 14812 ts_tablet_manager.cc:1428] T b6175f691fec45bf9853ee90959a1c98 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.880371 14812 tablet_bootstrap.cc:492] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:16.881228 14812 tablet_bootstrap.cc:654] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.882004 15390 raft_consensus.cc:491] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.882043 14820 tablet_replica.cc:331] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.882084 15390 raft_consensus.cc:513] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.882162 14820 raft_consensus.cc:2238] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.882247 14820 raft_consensus.cc:2267] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.882404 15390 leader_election.cc:290] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:16.882476 14628 leader_election.cc:343] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.882532 14628 leader_election.cc:304] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.882869 15453 raft_consensus.cc:2743] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.882874 14820 ts_tablet_manager.cc:1905] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.883087 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "341b24d9b63f4761b2013035023fcd98" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.883224 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "341b24d9b63f4761b2013035023fcd98" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:16.883318 14689 raft_consensus.cc:2463] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:16.883514 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet 304fab45b38e4426ae6961ae786e5659 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:16.884071 14358 leader_election.cc:304] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.884351 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a60b99b368f74c68873e8c89066cd3e0" } modify_external_catalogs: true
I20250115 03:06:16.884436 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a60b99b368f74c68873e8c89066cd3e0" } modify_external_catalogs: true
W20250115 03:06:16.885198 14356 leader_election.cc:343] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.886636 15390 raft_consensus.cc:2798] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:16.886691 15390 raft_consensus.cc:491] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:16.886729 15390 raft_consensus.cc:3054] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.887640 15390 raft_consensus.cc:513] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.887836 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 872ad705e3e94c0eab4397df9f28f9ca
W20250115 03:06:16.887903 14162 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5db9d5f6cecb454a90ad931d8d165bcc consensus state has no leader
I20250115 03:06:16.887954 15390 leader_election.cc:290] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.888077 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2db28e1f2ea5450092ed00f593c4939f" } modify_external_catalogs: true
I20250115 03:06:16.888154 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2db28e1f2ea5450092ed00f593c4939f" } modify_external_catalogs: true
I20250115 03:06:16.888221 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_75cd7b01fc534652975eb593912b88bc" } modify_external_catalogs: true
I20250115 03:06:16.888795 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "341b24d9b63f4761b2013035023fcd98" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:16.888808 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "341b24d9b63f4761b2013035023fcd98" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:16.888895 14689 raft_consensus.cc:3054] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:16.888918 14165 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9febec144bb343a8a21817e6d3655ad7 consensus state has no leader
I20250115 03:06:16.888305 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_75cd7b01fc534652975eb593912b88bc" } modify_external_catalogs: true
I20250115 03:06:16.889798 14689 raft_consensus.cc:2463] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
W20250115 03:06:16.889942 14356 leader_election.cc:343] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.890573 14169 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet a1c93bfd3da44f9f8a8d585f2d21f1be (table default.Table_40b1953a4a364eb1ae58e3387fc65a91 [id=64d20b1792044c5e8add735aaf75a6d9]). Expected version 1 got 0
I20250115 03:06:16.890579 14358 leader_election.cc:304] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:16.890900 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet 872ad705e3e94c0eab4397df9f28f9ca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.891206 15390 raft_consensus.cc:2798] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:16.891304 15390 raft_consensus.cc:695] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:16.891413 15390 consensus_queue.cc:237] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.891476 15857 raft_consensus.cc:491] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.891542 15857 raft_consensus.cc:513] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.891731 14820 ts_tablet_manager.cc:1918] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.891789 14820 log.cc:1198] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/5b7022db9edc4de8909e16769a3ea136
I20250115 03:06:16.891880 15857 leader_election.cc:290] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.891968 14820 ts_tablet_manager.cc:1939] T 5b7022db9edc4de8909e16769a3ea136 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.892318 14535 tablet_service.cc:1467] Processing CreateTablet for tablet b08e5fc2110e476e897240f9cf3b874d (DEFAULT_TABLE table=default.Table_50c659a42a154d46a26879df0101bf23 [id=f8f2a7ff6fdc40468de6ae9ba28c1427]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.892406 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f70a69acbdc049beb9d98eada09d9eb8" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:16.892748 14532 tablet_service.cc:1467] Processing CreateTablet for tablet 8efa998709034fe9bb37c7c608c99e76 (DEFAULT_TABLE table=default.Table_8f7e876bb60444429c30944284f9b10d [id=4c458a80b1a2490da1cf689843cae584]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.893468 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f70a69acbdc049beb9d98eada09d9eb8" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:16.893805 14493 leader_election.cc:304] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.893996 15857 raft_consensus.cc:2743] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.895306 14819 tablet_replica.cc:331] T e368f87556944c46a6d8f160ead17ac9 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.895434 14819 raft_consensus.cc:2238] T e368f87556944c46a6d8f160ead17ac9 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.895524 14819 raft_consensus.cc:2267] T e368f87556944c46a6d8f160ead17ac9 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.896091 14809 tablet_bootstrap.cc:492] T a75b9401cb4b4f8b92a4779c3704eb46 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.896179 14809 ts_tablet_manager.cc:1397] T a75b9401cb4b4f8b92a4779c3704eb46 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.040s user 0.001s sys 0.000s
I20250115 03:06:16.896580 14809 raft_consensus.cc:357] T a75b9401cb4b4f8b92a4779c3704eb46 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.896675 14809 raft_consensus.cc:383] T a75b9401cb4b4f8b92a4779c3704eb46 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.896708 14809 raft_consensus.cc:738] T a75b9401cb4b4f8b92a4779c3704eb46 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.896812 14809 consensus_queue.cc:260] T a75b9401cb4b4f8b92a4779c3704eb46 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.897439 14809 ts_tablet_manager.cc:1428] T a75b9401cb4b4f8b92a4779c3704eb46 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.897604 14809 tablet_bootstrap.cc:492] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.898371 14809 tablet_bootstrap.cc:654] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.898764 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet 872ad705e3e94c0eab4397df9f28f9ca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.898974 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet d0c3536533e94174bb1e3888a3458395 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.899120 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 6110b6c4911d4932b4cac4d86598a5c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.899313 14823 tablet_replica.cc:331] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.899335 15401 raft_consensus.cc:491] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.899402 15401 raft_consensus.cc:513] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.899438 14823 raft_consensus.cc:2238] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.899535 14823 raft_consensus.cc:2267] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.899660 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 872ad705e3e94c0eab4397df9f28f9ca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.899725 15401 leader_election.cc:290] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.899909 14398 tablet_service.cc:1467] Processing CreateTablet for tablet b08e5fc2110e476e897240f9cf3b874d (DEFAULT_TABLE table=default.Table_50c659a42a154d46a26879df0101bf23 [id=f8f2a7ff6fdc40468de6ae9ba28c1427]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.900123 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b08e5fc2110e476e897240f9cf3b874d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.900674 14823 ts_tablet_manager.cc:1905] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.900874 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 4ff51d514fb84809aa53fa9fc1010681
I20250115 03:06:16.901515 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c93bfd3da44f9f8a8d585f2d21f1be" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.901621 14557 raft_consensus.cc:2463] T a1c93bfd3da44f9f8a8d585f2d21f1be P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:16.902084 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b08e5fc2110e476e897240f9cf3b874d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.902082 14356 leader_election.cc:304] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.902223 15401 raft_consensus.cc:2798] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:16.902274 15401 raft_consensus.cc:491] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:16.902309 15401 raft_consensus.cc:3054] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.903088 15401 raft_consensus.cc:513] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.903381 15401 leader_election.cc:290] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.904392 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8efa998709034fe9bb37c7c608c99e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.904421 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c93bfd3da44f9f8a8d585f2d21f1be" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:16.904546 14557 raft_consensus.cc:3054] T a1c93bfd3da44f9f8a8d585f2d21f1be P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.905202 14819 ts_tablet_manager.cc:1905] T e368f87556944c46a6d8f160ead17ac9 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.905544 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 054251151c294e06905092f2a9306849 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.906142 14557 raft_consensus.cc:2463] T a1c93bfd3da44f9f8a8d585f2d21f1be P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:16.906222 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c93bfd3da44f9f8a8d585f2d21f1be" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:16.906314 14687 raft_consensus.cc:3054] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.906699 14356 leader_election.cc:304] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:16.906850 15401 raft_consensus.cc:2798] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:16.906920 15401 raft_consensus.cc:695] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:16.907025 15401 consensus_queue.cc:237] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.907164 14687 raft_consensus.cc:2463] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:16.907905 14820 tablet_replica.cc:331] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.908061 14820 raft_consensus.cc:2238] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.908151 14820 raft_consensus.cc:2267] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.908281 14664 tablet_service.cc:1467] Processing CreateTablet for tablet b08e5fc2110e476e897240f9cf3b874d (DEFAULT_TABLE table=default.Table_50c659a42a154d46a26879df0101bf23 [id=f8f2a7ff6fdc40468de6ae9ba28c1427]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.908579 14820 ts_tablet_manager.cc:1905] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.900113 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1c93bfd3da44f9f8a8d585f2d21f1be" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:16.909016 14689 raft_consensus.cc:2371] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1. Re-sending same reply.
I20250115 03:06:16.901785 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet a4484651d964411e90ed93c6caab0a13
I20250115 03:06:16.911545 14823 ts_tablet_manager.cc:1918] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.902866 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet 4122c97f5ba445ee8c79b7d056438229 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:16.911605 14820 ts_tablet_manager.cc:1918] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.911665 14820 log.cc:1198] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/ec9f14c1ab014f29843a568c460a28d6
I20250115 03:06:16.912082 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b08e5fc2110e476e897240f9cf3b874d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.912305 14823 log.cc:1198] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/872ad705e3e94c0eab4397df9f28f9ca
I20250115 03:06:16.912518 14823 ts_tablet_manager.cc:1939] T 872ad705e3e94c0eab4397df9f28f9ca P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.913267 14823 tablet_replica.cc:331] T d0c3536533e94174bb1e3888a3458395 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.913388 14823 raft_consensus.cc:2238] T d0c3536533e94174bb1e3888a3458395 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.913477 14823 raft_consensus.cc:2267] T d0c3536533e94174bb1e3888a3458395 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.915817 14809 tablet_bootstrap.cc:492] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.915915 14809 ts_tablet_manager.cc:1397] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20250115 03:06:16.916370 14809 raft_consensus.cc:357] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.916473 14809 raft_consensus.cc:383] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.916517 14809 raft_consensus.cc:738] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.916641 14809 consensus_queue.cc:260] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.916911 14809 ts_tablet_manager.cc:1428] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.916715 14811 tablet_bootstrap.cc:492] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.917069 14811 ts_tablet_manager.cc:1397] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.052s user 0.002s sys 0.000s
I20250115 03:06:16.917099 14809 tablet_bootstrap.cc:492] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.917479 14811 raft_consensus.cc:357] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.917577 14811 raft_consensus.cc:383] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.917605 14811 raft_consensus.cc:738] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.917713 14811 consensus_queue.cc:260] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.917975 14811 ts_tablet_manager.cc:1428] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.918011 14809 tablet_bootstrap.cc:654] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.918162 14811 tablet_bootstrap.cc:492] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.919052 14811 tablet_bootstrap.cc:654] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.920233 14820 ts_tablet_manager.cc:1939] T ec9f14c1ab014f29843a568c460a28d6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.920336 14819 ts_tablet_manager.cc:1918] T e368f87556944c46a6d8f160ead17ac9 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:16.920428 14819 log.cc:1198] T e368f87556944c46a6d8f160ead17ac9 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/e368f87556944c46a6d8f160ead17ac9
I20250115 03:06:16.920679 14819 ts_tablet_manager.cc:1939] T e368f87556944c46a6d8f160ead17ac9 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.921236 14820 tablet_replica.cc:331] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.903672 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet acb43d614111477abb10def48a1dfb3b (table default.Table_85ea53c1d8b245cc94e6e81e157242aa [id=4d8e724e03dc4e2ab9a6e0281159aca8]) successfully deleted
I20250115 03:06:16.905575 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 5b7022db9edc4de8909e16769a3ea136 (table default.Table_a5cd96954b1c48c68a3bef815c47d8b4 [id=f94088b0776f4be990ddcec04afbca43]) successfully deleted
I20250115 03:06:16.908742 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 80ede4e9696d491aa385b375fcb5cf3b
I20250115 03:06:16.909264 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 304fab45b38e4426ae6961ae786e5659
I20250115 03:06:16.922593 14163 catalog_manager.cc:6596] ReplaceTablet: tablet 304fab45b38e4426ae6961ae786e5659 deleted and replaced by tablet 0b5fd91a0ea64b3b85b7571e5daa08b2
I20250115 03:06:16.922679 14819 tablet_replica.cc:331] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:16.922766 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet ec9f14c1ab014f29843a568c460a28d6 (table default.Table_60943a26c80945ae84effaed75c0cc42 [id=d94e2b655f424f549af052bc8dee0430]) successfully deleted
I20250115 03:06:16.922801 14819 raft_consensus.cc:2238] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.922886 14819 raft_consensus.cc:2267] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.923161 14819 ts_tablet_manager.cc:1905] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.923471 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ff51d514fb84809aa53fa9fc1010681 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.923671 14820 raft_consensus.cc:2238] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:16.923938 14820 raft_consensus.cc:2267] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.924062 14532 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1ce43146ed4e4d82a7c134216e2872 (DEFAULT_TABLE table=default.Table_0c35f2414d284756b9c93ed61e75d451 [id=5775da4995774af39bfc049e48eaf2f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.924235 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet 506198162e0b4e24924c408542707c4c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.924284 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e1ce43146ed4e4d82a7c134216e2872. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:16.924288 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 5589b497b0e24740ad49d1392f5a737b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5589b497b0e24740ad49d1392f5a737b already in progress: creating tablet
W20250115 03:06:16.924391 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 1f2dc6928e2941ef9b5b5a7891376500 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1f2dc6928e2941ef9b5b5a7891376500 already in progress: creating tablet
I20250115 03:06:16.924613 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet e368f87556944c46a6d8f160ead17ac9 (table default.Table_2af92748648345c0903a0f99da0d47c5 [id=19efb3734d824699a088ad62e69095f3]) successfully deleted
I20250115 03:06:16.924710 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 341180618b4748eabc568888e2e6a7df (table default.Table_65f0fd5511f64bfeb6539f9019a4873c [id=7191c3890ae642558fe99561455d01e7]) successfully deleted
I20250115 03:06:16.924786 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 872ad705e3e94c0eab4397df9f28f9ca (table default.Table_afc5c7e955fb4b3ba43151cb8823a76b [id=bac33f20b4524a05bde7452db44d1104]) successfully deleted
I20250115 03:06:16.925292 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 8efa998709034fe9bb37c7c608c99e76 (DEFAULT_TABLE table=default.Table_8f7e876bb60444429c30944284f9b10d [id=4c458a80b1a2490da1cf689843cae584]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.925509 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8efa998709034fe9bb37c7c608c99e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.926345 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 72abf6acd9d54f3b983b7098aaf2691d (DEFAULT_TABLE table=default.Table_0c984dbf685d4dad9338caa1119c5e44 [id=5ca0a4e7179649d3870fdadc05eaa427]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.926564 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72abf6acd9d54f3b983b7098aaf2691d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.927184 14811 tablet_bootstrap.cc:492] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.927250 14811 ts_tablet_manager.cc:1397] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20250115 03:06:16.927491 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 8efa998709034fe9bb37c7c608c99e76 (DEFAULT_TABLE table=default.Table_8f7e876bb60444429c30944284f9b10d [id=4c458a80b1a2490da1cf689843cae584]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.927716 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8efa998709034fe9bb37c7c608c99e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.927747 14811 raft_consensus.cc:357] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.927827 14811 raft_consensus.cc:383] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.927853 14811 raft_consensus.cc:738] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.927958 14811 consensus_queue.cc:260] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.928190 14811 ts_tablet_manager.cc:1428] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.928332 14811 tablet_bootstrap.cc:492] T 745344db22da4102bad39c2f121548a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.928481 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 5db9d5f6cecb454a90ad931d8d165bcc (DEFAULT_TABLE table=default.Table_e68b21fe987c4ea49523f24440585fce [id=07019baad4a74fd0b468bbd9b60d54c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.928685 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5db9d5f6cecb454a90ad931d8d165bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.929224 14811 tablet_bootstrap.cc:654] T 745344db22da4102bad39c2f121548a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.929839 15153 raft_consensus.cc:491] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.929910 15153 raft_consensus.cc:513] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.930177 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 4122c97f5ba445ee8c79b7d056438229
I20250115 03:06:16.930246 15153 leader_election.cc:290] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:16.930420 14171 catalog_manager.cc:6596] ReplaceTablet: tablet 4122c97f5ba445ee8c79b7d056438229 deleted and replaced by tablet fca14a7d574842ee94a93afb85ca1afd
I20250115 03:06:16.930601 14529 tablet_service.cc:1467] Processing CreateTablet for tablet 7f9818d0493547358fd5654a01d4f456 (DEFAULT_TABLE table=default.Table_652f8e06e005491381404a11759aefb1 [id=6fb6e311a0af4f51b190a880f84373d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.930752 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13f36a563f13403eab372deb10b67451" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:16.930797 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f9818d0493547358fd5654a01d4f456. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.930802 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1ce43146ed4e4d82a7c134216e2872 (DEFAULT_TABLE table=default.Table_0c35f2414d284756b9c93ed61e75d451 [id=5775da4995774af39bfc049e48eaf2f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.930987 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e1ce43146ed4e4d82a7c134216e2872. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.930984 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13f36a563f13403eab372deb10b67451" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:16.931360 14628 leader_election.cc:343] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:16.931517 14626 leader_election.cc:343] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.931569 14626 leader_election.cc:304] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.932196 15153 raft_consensus.cc:2743] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.932582 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet a4484651d964411e90ed93c6caab0a13 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.932706 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 80ede4e9696d491aa385b375fcb5cf3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.932796 14531 tablet_service.cc:1514] Processing DeleteTablet for tablet 304fab45b38e4426ae6961ae786e5659 with delete_type TABLET_DATA_DELETED (replaced by tablet 0b5fd91a0ea64b3b85b7571e5daa08b2) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.932869 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_a82405d576ec40228fe5ad55b98753e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:16.933027 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1ce43146ed4e4d82a7c134216e2872 (DEFAULT_TABLE table=default.Table_0c35f2414d284756b9c93ed61e75d451 [id=5775da4995774af39bfc049e48eaf2f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.933231 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e1ce43146ed4e4d82a7c134216e2872. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:16.933246 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a82405d576ec40228fe5ad55b98753e8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.935053 15153 raft_consensus.cc:491] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.935148 15153 raft_consensus.cc:513] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.935470 15153 leader_election.cc:290] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:16.935995 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5986558350244e7aa4ddcd5cccb7d676" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:16.936331 14628 leader_election.cc:343] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.936592 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5986558350244e7aa4ddcd5cccb7d676" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:16.936875 14626 leader_election.cc:343] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.924242 14820 ts_tablet_manager.cc:1905] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.936954 14626 leader_election.cc:304] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:16.937127 15153 raft_consensus.cc:2743] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.937928 14557 raft_consensus.cc:3054] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:16.939551 15426 tablet.cc:1721] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:16.939882 14809 tablet_bootstrap.cc:492] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.939958 14809 ts_tablet_manager.cc:1397] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.001s
I20250115 03:06:16.939915 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_bd6a79400bc244d69676d8df1069cbb2" } modify_external_catalogs: true
I20250115 03:06:16.940013 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_bd6a79400bc244d69676d8df1069cbb2" } modify_external_catalogs: true
I20250115 03:06:16.940373 14809 raft_consensus.cc:357] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.940461 14809 raft_consensus.cc:383] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.940492 14809 raft_consensus.cc:738] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.940599 14809 consensus_queue.cc:260] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.940868 14809 ts_tablet_manager.cc:1428] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.940908 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 4122c97f5ba445ee8c79b7d056438229 with delete_type TABLET_DATA_DELETED (replaced by tablet fca14a7d574842ee94a93afb85ca1afd) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.941056 14809 tablet_bootstrap.cc:492] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.942132 14809 tablet_bootstrap.cc:654] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.942075 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f634ab8b7b7c437f90035122e8f974b9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.942376 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f634ab8b7b7c437f90035122e8f974b9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.943037 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_c0bde98045a84e568d03c9608c7731fb" } new_table_name: "default.Table_23165257844448ce93a97233b487ddc7" modify_external_catalogs: true
I20250115 03:06:16.943639 14819 ts_tablet_manager.cc:1918] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:16.943691 14819 log.cc:1198] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/20f86f93812849c4b7852a884f677c04
I20250115 03:06:16.943899 14819 ts_tablet_manager.cc:1939] T 20f86f93812849c4b7852a884f677c04 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:16.944097 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_6ba55a3b38784f449b0e6bd030983567"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.944375 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6ba55a3b38784f449b0e6bd030983567 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.945037 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_93108a53124a492599eda22f3633780a" } new_table_name: "default.Table_87c206116f514479ab9de8cd4a05817f" modify_external_catalogs: true
I20250115 03:06:16.946012 14820 ts_tablet_manager.cc:1918] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:16.946072 14820 log.cc:1198] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/20f86f93812849c4b7852a884f677c04
I20250115 03:06:16.946280 14820 ts_tablet_manager.cc:1939] T 20f86f93812849c4b7852a884f677c04 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.946415 14162 master_service.cc:946] ReplaceTablet: received request to replace tablet b2629ae9d9854ae4b6e7ddddbb6f341b from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:16.946619 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet acb43d614111477abb10def48a1dfb3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:14 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.947911 14811 tablet_bootstrap.cc:492] T 745344db22da4102bad39c2f121548a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.947989 14811 ts_tablet_manager.cc:1397] T 745344db22da4102bad39c2f121548a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20250115 03:06:16.948288 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 20f86f93812849c4b7852a884f677c04 (table default.Table_50c659a42a154d46a26879df0101bf23 [id=f8f2a7ff6fdc40468de6ae9ba28c1427]) successfully deleted
I20250115 03:06:16.948398 14811 raft_consensus.cc:357] T 745344db22da4102bad39c2f121548a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.948475 14820 tablet_replica.cc:331] T 82be772e05d443749914004aea8af300 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:16.948493 14811 raft_consensus.cc:383] T 745344db22da4102bad39c2f121548a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.948522 14811 raft_consensus.cc:738] T 745344db22da4102bad39c2f121548a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.948580 14820 raft_consensus.cc:2238] T 82be772e05d443749914004aea8af300 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.948653 14820 raft_consensus.cc:2267] T 82be772e05d443749914004aea8af300 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.948621 14811 consensus_queue.cc:260] T 745344db22da4102bad39c2f121548a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.948870 14811 ts_tablet_manager.cc:1428] T 745344db22da4102bad39c2f121548a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.949301 14811 tablet_bootstrap.cc:492] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.950186 14811 tablet_bootstrap.cc:654] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.951635 14820 ts_tablet_manager.cc:1905] T 82be772e05d443749914004aea8af300 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.952630 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_727f8ca159af4f75b43618f937add0b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.953133 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_727f8ca159af4f75b43618f937add0b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.954654 14170 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 745344db22da4102bad39c2f121548a1 (table default.Table_365313892043408c92be8460c31d958e [id=f22f08e9af4a4496b100e7d4b5982121]). Expected version 1 got 0
I20250115 03:06:16.954782 14820 ts_tablet_manager.cc:1918] T 82be772e05d443749914004aea8af300 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.954841 14820 log.cc:1198] T 82be772e05d443749914004aea8af300 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/82be772e05d443749914004aea8af300
I20250115 03:06:16.955021 14820 ts_tablet_manager.cc:1939] T 82be772e05d443749914004aea8af300 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:16.958642 14823 ts_tablet_manager.cc:1905] T d0c3536533e94174bb1e3888a3458395 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.958796 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 72abf6acd9d54f3b983b7098aaf2691d (DEFAULT_TABLE table=default.Table_0c984dbf685d4dad9338caa1119c5e44 [id=5ca0a4e7179649d3870fdadc05eaa427]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.959014 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72abf6acd9d54f3b983b7098aaf2691d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.959678 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 7f9818d0493547358fd5654a01d4f456 (DEFAULT_TABLE table=default.Table_652f8e06e005491381404a11759aefb1 [id=6fb6e311a0af4f51b190a880f84373d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.959906 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f9818d0493547358fd5654a01d4f456. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.960083 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 72abf6acd9d54f3b983b7098aaf2691d (DEFAULT_TABLE table=default.Table_0c984dbf685d4dad9338caa1119c5e44 [id=5ca0a4e7179649d3870fdadc05eaa427]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.960271 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72abf6acd9d54f3b983b7098aaf2691d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.961282 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 5db9d5f6cecb454a90ad931d8d165bcc (DEFAULT_TABLE table=default.Table_e68b21fe987c4ea49523f24440585fce [id=07019baad4a74fd0b468bbd9b60d54c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.961540 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5db9d5f6cecb454a90ad931d8d165bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.962815 14823 ts_tablet_manager.cc:1918] T d0c3536533e94174bb1e3888a3458395 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.962873 14823 log.cc:1198] T d0c3536533e94174bb1e3888a3458395 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/d0c3536533e94174bb1e3888a3458395
I20250115 03:06:16.963054 14823 ts_tablet_manager.cc:1939] T d0c3536533e94174bb1e3888a3458395 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.963620 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 5db9d5f6cecb454a90ad931d8d165bcc (DEFAULT_TABLE table=default.Table_e68b21fe987c4ea49523f24440585fce [id=07019baad4a74fd0b468bbd9b60d54c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.963860 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5db9d5f6cecb454a90ad931d8d165bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.963938 14809 tablet_bootstrap.cc:492] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.964013 14809 ts_tablet_manager.cc:1397] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20250115 03:06:16.963630 14823 tablet_replica.cc:331] T 6110b6c4911d4932b4cac4d86598a5c6 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.964133 14823 raft_consensus.cc:2238] T 6110b6c4911d4932b4cac4d86598a5c6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.964223 14823 raft_consensus.cc:2267] T 6110b6c4911d4932b4cac4d86598a5c6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.964224 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet a4484651d964411e90ed93c6caab0a13 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.964454 14809 raft_consensus.cc:357] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.964555 14809 raft_consensus.cc:383] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.964586 14809 raft_consensus.cc:738] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.964763 14823 ts_tablet_manager.cc:1905] T 6110b6c4911d4932b4cac4d86598a5c6 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.964804 14809 consensus_queue.cc:260] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.965142 14809 ts_tablet_manager.cc:1428] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.965262 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet 80ede4e9696d491aa385b375fcb5cf3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.965305 14809 tablet_bootstrap.cc:492] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.965579 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 304fab45b38e4426ae6961ae786e5659 with delete_type TABLET_DATA_DELETED (replaced by tablet 0b5fd91a0ea64b3b85b7571e5daa08b2) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.965922 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet 304fab45b38e4426ae6961ae786e5659 with delete_type TABLET_DATA_DELETED (replaced by tablet 0b5fd91a0ea64b3b85b7571e5daa08b2) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.966113 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 7f9818d0493547358fd5654a01d4f456 (DEFAULT_TABLE table=default.Table_652f8e06e005491381404a11759aefb1 [id=6fb6e311a0af4f51b190a880f84373d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.966185 14809 tablet_bootstrap.cc:654] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.966315 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f9818d0493547358fd5654a01d4f456. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.966495 14823 ts_tablet_manager.cc:1918] T 6110b6c4911d4932b4cac4d86598a5c6 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.966547 14823 log.cc:1198] T 6110b6c4911d4932b4cac4d86598a5c6 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/6110b6c4911d4932b4cac4d86598a5c6
I20250115 03:06:16.966702 14823 ts_tablet_manager.cc:1939] T 6110b6c4911d4932b4cac4d86598a5c6 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.966853 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ff51d514fb84809aa53fa9fc1010681 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.967310 14823 tablet_replica.cc:331] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.967414 14823 raft_consensus.cc:2238] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.967504 14823 raft_consensus.cc:2267] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.968307 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 80ede4e9696d491aa385b375fcb5cf3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.968860 14823 ts_tablet_manager.cc:1905] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.969455 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ff51d514fb84809aa53fa9fc1010681 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.969610 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet a4484651d964411e90ed93c6caab0a13 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.970608 14823 ts_tablet_manager.cc:1918] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:16.970680 14823 log.cc:1198] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/a4484651d964411e90ed93c6caab0a13
I20250115 03:06:16.970906 14823 ts_tablet_manager.cc:1939] T a4484651d964411e90ed93c6caab0a13 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.972239 14811 tablet_bootstrap.cc:492] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:16.972322 14811 ts_tablet_manager.cc:1397] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s
I20250115 03:06:16.972388 14809 tablet_bootstrap.cc:492] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.972462 14809 ts_tablet_manager.cc:1397] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20250115 03:06:16.972767 14811 raft_consensus.cc:357] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.972862 14811 raft_consensus.cc:383] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.972838 14809 raft_consensus.cc:357] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.972892 14811 raft_consensus.cc:738] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.972918 14809 raft_consensus.cc:383] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.973006 14809 raft_consensus.cc:738] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.973027 14811 consensus_queue.cc:260] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.973105 14809 consensus_queue.cc:260] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.973279 14811 ts_tablet_manager.cc:1428] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:16.973346 14809 ts_tablet_manager.cc:1428] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.973469 14811 tablet_bootstrap.cc:492] T eeff2cd0825c4419aacfc73ebe267a8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:16.973485 14809 tablet_bootstrap.cc:492] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.974342 14809 tablet_bootstrap.cc:654] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.974339 14823 tablet_replica.cc:331] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.974447 14823 raft_consensus.cc:2238] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.974545 14823 raft_consensus.cc:2267] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:16.974833 14823 ts_tablet_manager.cc:1905] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:16.976353 14811 tablet_bootstrap.cc:654] T eeff2cd0825c4419aacfc73ebe267a8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.976426 14823 ts_tablet_manager.cc:1918] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:16.976488 14823 log.cc:1198] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/304fab45b38e4426ae6961ae786e5659
I20250115 03:06:16.976648 14823 ts_tablet_manager.cc:1939] T 304fab45b38e4426ae6961ae786e5659 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:16.978634 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet 054251151c294e06905092f2a9306849 from {username='slave'} at 127.0.0.1:60432
W20250115 03:06:16.979089 14167 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b56279bb16054869a97006d88574e381 consensus state has no leader
I20250115 03:06:16.979446 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet e594ea5cff81475886cdceb8400645a6
I20250115 03:06:16.980248 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet e594ea5cff81475886cdceb8400645a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.980465 14809 tablet_bootstrap.cc:492] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.980543 14809 ts_tablet_manager.cc:1397] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20250115 03:06:16.980962 14809 raft_consensus.cc:357] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.981056 14809 raft_consensus.cc:383] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.981088 14809 raft_consensus.cc:738] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.981191 14809 consensus_queue.cc:260] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.981449 14809 ts_tablet_manager.cc:1428] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:16.981597 14809 tablet_bootstrap.cc:492] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.981889 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 48aa039bf0364c02abb7f9677eae49e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:16.982323 14823 tablet_replica.cc:331] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:16.982492 14823 raft_consensus.cc:2238] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:16.982621 14809 tablet_bootstrap.cc:654] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.983168 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6ba55a3b38784f449b0e6bd030983567" } new_table_name: "default.Table_df79ed42f26f4b68b197877c445b9665" modify_external_catalogs: true
I20250115 03:06:16.984802 14809 tablet_bootstrap.cc:492] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:16.984876 14809 ts_tablet_manager.cc:1397] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20250115 03:06:16.985298 14809 raft_consensus.cc:357] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.985384 14809 raft_consensus.cc:383] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:16.985414 14809 raft_consensus.cc:738] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:16.985527 14809 consensus_queue.cc:260] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.985774 14809 ts_tablet_manager.cc:1428] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:16.985992 14809 tablet_bootstrap.cc:492] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:16.986786 14809 tablet_bootstrap.cc:654] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:16.987350 15401 raft_consensus.cc:491] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:16.987429 15401 raft_consensus.cc:513] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:16.987761 15401 leader_election.cc:290] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:16.987988 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 4b39608e19bf44b3a79a8616ccdad30a (DEFAULT_TABLE table=default.Table_268ae5920054489fa49b7ce5b1569532 [id=17c3c2e3dc554605b5417f84a0b26637]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.987998 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90277f7d279e4c7a83c43f40766cd11b" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:16.988233 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b39608e19bf44b3a79a8616ccdad30a. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:16.988493 14358 leader_election.cc:343] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 90277f7d279e4c7a83c43f40766cd11b
W20250115 03:06:16.988561 14356 leader_election.cc:343] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:16.988627 14356 leader_election.cc:304] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:16.988744 15401 raft_consensus.cc:2743] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:16.990072 14171 catalog_manager.cc:5526] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:16.991037 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 4122c97f5ba445ee8c79b7d056438229 with delete_type TABLET_DATA_DELETED (replaced by tablet fca14a7d574842ee94a93afb85ca1afd) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:16.991853 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 4122c97f5ba445ee8c79b7d056438229 with delete_type TABLET_DATA_DELETED (replaced by tablet fca14a7d574842ee94a93afb85ca1afd) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:16.992493 14531 tablet_service.cc:1467] Processing CreateTablet for tablet d891020ed0bb4fc49155bd9f2f6ed35c (DEFAULT_TABLE table=default.Table_73f837db71d74dae9606eb6995ad7270 [id=ff302d72ca26434f9a86f478f3346318]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.992718 14531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d891020ed0bb4fc49155bd9f2f6ed35c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.993104 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 6ed47b26cdf744acb28fb657a1b41087 (DEFAULT_TABLE table=default.Table_4f5d0494ec384c479419c10b7ff36dc0 [id=b9026c7e60a14240b2f5c596d31ba517]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:16.993403 14626 consensus_peers.cc:487] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:16.993386 14528 tablet_service.cc:1467] Processing CreateTablet for tablet 6bd4e4200f9b489ea5dd2154aa6f382d (DEFAULT_TABLE table=default.Table_2e3e4b87e2e04feaab40ea415807cd06 [id=299ac71cfb98480f873eb659419e522c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.993443 14421 raft_consensus.cc:1270] T eeff2cd0825c4419aacfc73ebe267a8e P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 5c0dae7aff4b44c699275b95ba639770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:16.993676 14529 tablet_service.cc:1467] Processing CreateTablet for tablet c2afa048b3fd4362b50c8a6da306c25a (DEFAULT_TABLE table=default.Table_478e448d8f6d494d90d421e2f0cd21fc [id=74853e82e7024da6a9e4e1dd23bd043d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:16.993763 15153 consensus_queue.cc:1035] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:16.994637 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2afa048b3fd4362b50c8a6da306c25a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.995117 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bd4e4200f9b489ea5dd2154aa6f382d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.995429 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ed47b26cdf744acb28fb657a1b41087. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:16.997169 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_59afdaba8b114e13b37772ac27949580"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:16.997475 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_59afdaba8b114e13b37772ac27949580 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:16.999101 15834 log.cc:826] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246: Log is configured to *not* fsync() on all Append() calls
I20250115 03:06:16.999506 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet e594ea5cff81475886cdceb8400645a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.000037 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet e594ea5cff81475886cdceb8400645a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.000232 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 4b39608e19bf44b3a79a8616ccdad30a (DEFAULT_TABLE table=default.Table_268ae5920054489fa49b7ce5b1569532 [id=17c3c2e3dc554605b5417f84a0b26637]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.000442 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b39608e19bf44b3a79a8616ccdad30a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.002812 14398 tablet_service.cc:1467] Processing CreateTablet for tablet a8e41e31ffae4bb092cb08d83eda16b5 (DEFAULT_TABLE table=default.Table_5577048ebf95456d9cbe74d11b84f131 [id=a530442772dc4b12b23992cee12fea85]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.003038 14662 tablet_service.cc:1467] Processing CreateTablet for tablet a8e41e31ffae4bb092cb08d83eda16b5 (DEFAULT_TABLE table=default.Table_5577048ebf95456d9cbe74d11b84f131 [id=a530442772dc4b12b23992cee12fea85]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.003062 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 4b39608e19bf44b3a79a8616ccdad30a (DEFAULT_TABLE table=default.Table_268ae5920054489fa49b7ce5b1569532 [id=17c3c2e3dc554605b5417f84a0b26637]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.003300 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8e41e31ffae4bb092cb08d83eda16b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.003700 14537 tablet_service.cc:1467] Processing CreateTablet for tablet a8e41e31ffae4bb092cb08d83eda16b5 (DEFAULT_TABLE table=default.Table_5577048ebf95456d9cbe74d11b84f131 [id=a530442772dc4b12b23992cee12fea85]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.003908 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8e41e31ffae4bb092cb08d83eda16b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.004268 14809 tablet_bootstrap.cc:492] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.004330 14809 ts_tablet_manager.cc:1397] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20250115 03:06:17.004701 14809 raft_consensus.cc:357] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.004788 14809 raft_consensus.cc:383] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.004817 14809 raft_consensus.cc:738] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.004915 14809 consensus_queue.cc:260] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.005162 14809 ts_tablet_manager.cc:1428] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:17.006336 15274 tablet.cc:1721] T eeff2cd0825c4419aacfc73ebe267a8e P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.007117 14823 raft_consensus.cc:2267] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.009119 15857 raft_consensus.cc:491] T 745344db22da4102bad39c2f121548a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.009192 15857 raft_consensus.cc:513] T 745344db22da4102bad39c2f121548a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.009487 15857 leader_election.cc:290] T 745344db22da4102bad39c2f121548a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.007676 14823 ts_tablet_manager.cc:1905] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.009979 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "745344db22da4102bad39c2f121548a1" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:17.012342 14823 ts_tablet_manager.cc:1918] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:17.012408 14823 log.cc:1198] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/4ff51d514fb84809aa53fa9fc1010681
I20250115 03:06:17.012611 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet eeff2cd0825c4419aacfc73ebe267a8e (table default.Table_590b26746c6a4006bb430ba9dcbacafc [id=021f5685630741a8880d2471f7d2d1d2]). Expected version 1 got 0
I20250115 03:06:17.012637 14823 ts_tablet_manager.cc:1939] T 4ff51d514fb84809aa53fa9fc1010681 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:17.013248 14823 tablet_replica.cc:331] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:17.013370 14823 raft_consensus.cc:2238] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.013473 14823 raft_consensus.cc:2267] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.013645 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "745344db22da4102bad39c2f121548a1" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:17.013778 14688 raft_consensus.cc:2388] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in current term 1: Already voted for candidate e8df7247ee9c43e0aa18216da2661f75 in this term.
I20250115 03:06:17.013840 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8e41e31ffae4bb092cb08d83eda16b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.014751 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b39608e19bf44b3a79a8616ccdad30a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.014782 14809 tablet_bootstrap.cc:492] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.015038 14493 leader_election.cc:304] T 745344db22da4102bad39c2f121548a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.015268 15857 raft_consensus.cc:3054] T 745344db22da4102bad39c2f121548a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.015748 14809 tablet_bootstrap.cc:654] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.016068 15857 raft_consensus.cc:2743] T 745344db22da4102bad39c2f121548a1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.020671 15617 tablet.cc:1721] T eeff2cd0825c4419aacfc73ebe267a8e P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.021734 14809 tablet_bootstrap.cc:492] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.021806 14809 ts_tablet_manager.cc:1397] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20250115 03:06:17.022091 14811 tablet_bootstrap.cc:492] T eeff2cd0825c4419aacfc73ebe267a8e P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:17.022163 14811 ts_tablet_manager.cc:1397] T eeff2cd0825c4419aacfc73ebe267a8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.049s user 0.001s sys 0.000s
I20250115 03:06:17.022190 14809 raft_consensus.cc:357] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.022267 14809 raft_consensus.cc:383] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.022291 14809 raft_consensus.cc:738] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.022382 14809 consensus_queue.cc:260] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.022616 14809 ts_tablet_manager.cc:1428] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:17.022588 14811 raft_consensus.cc:357] T eeff2cd0825c4419aacfc73ebe267a8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.022688 14811 raft_consensus.cc:383] T eeff2cd0825c4419aacfc73ebe267a8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.022719 14811 raft_consensus.cc:738] T eeff2cd0825c4419aacfc73ebe267a8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.022745 14809 tablet_bootstrap.cc:492] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.022827 14811 consensus_queue.cc:260] T eeff2cd0825c4419aacfc73ebe267a8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.023084 14811 ts_tablet_manager.cc:1428] T eeff2cd0825c4419aacfc73ebe267a8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.023265 14811 tablet_bootstrap.cc:492] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:17.023742 14809 tablet_bootstrap.cc:654] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.024158 14811 tablet_bootstrap.cc:654] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.025811 14163 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet eeff2cd0825c4419aacfc73ebe267a8e (table default.Table_590b26746c6a4006bb430ba9dcbacafc [id=021f5685630741a8880d2471f7d2d1d2]). Expected version 1 got 0
I20250115 03:06:17.027281 15401 raft_consensus.cc:491] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.027357 15401 raft_consensus.cc:513] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.027680 15401 leader_election.cc:290] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:17.028430 14358 leader_election.cc:343] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 6d10731df02f4e9682b8de6962256635
I20250115 03:06:17.028669 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d10731df02f4e9682b8de6962256635" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:17.029042 14356 leader_election.cc:343] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.029109 14356 leader_election.cc:304] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.029304 14823 ts_tablet_manager.cc:1905] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.030331 15401 raft_consensus.cc:2743] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.030522 15401 raft_consensus.cc:491] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.030603 15401 raft_consensus.cc:513] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.030896 15401 leader_election.cc:290] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.031122 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13f36a563f13403eab372deb10b67451" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:17.031515 14356 leader_election.cc:343] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.032037 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13f36a563f13403eab372deb10b67451" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:17.032151 14688 raft_consensus.cc:2463] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:17.032680 14811 tablet_bootstrap.cc:492] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:17.032660 14358 leader_election.cc:304] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.032753 14811 ts_tablet_manager.cc:1397] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20250115 03:06:17.032954 15401 raft_consensus.cc:2798] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:17.033004 15401 raft_consensus.cc:491] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:17.033039 15401 raft_consensus.cc:3054] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.033205 14811 raft_consensus.cc:357] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.033282 14811 raft_consensus.cc:383] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.033306 14811 raft_consensus.cc:738] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.033402 14811 consensus_queue.cc:260] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.033646 14811 ts_tablet_manager.cc:1428] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.033812 14811 tablet_bootstrap.cc:492] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:17.033840 15401 raft_consensus.cc:513] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.034133 15401 leader_election.cc:290] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.034518 14809 tablet_bootstrap.cc:492] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.034588 14809 ts_tablet_manager.cc:1397] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20250115 03:06:17.034698 14811 tablet_bootstrap.cc:654] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.034751 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13f36a563f13403eab372deb10b67451" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:17.034843 14688 raft_consensus.cc:3054] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.034965 14809 raft_consensus.cc:357] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.035048 14809 raft_consensus.cc:383] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.035073 14809 raft_consensus.cc:738] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.035154 14809 consensus_queue.cc:260] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.035396 14809 ts_tablet_manager.cc:1428] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.035532 14809 tablet_bootstrap.cc:492] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.035688 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13f36a563f13403eab372deb10b67451" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:17.035710 14688 raft_consensus.cc:2463] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
W20250115 03:06:17.035964 14356 leader_election.cc:343] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.036047 14358 leader_election.cc:304] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.036288 15390 raft_consensus.cc:2798] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:17.036353 14809 tablet_bootstrap.cc:654] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.036899 14861 raft_consensus.cc:491] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.037003 14861 raft_consensus.cc:513] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.037267 14861 leader_election.cc:290] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.036358 15390 raft_consensus.cc:695] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:17.037531 15390 consensus_queue.cc:237] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.037544 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "054251151c294e06905092f2a9306849" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:17.037833 14356 leader_election.cc:343] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.037943 14819 tablet_replica.cc:331] T b0a5d554764249f8a7eefdc943271090 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.038065 14819 raft_consensus.cc:2238] T b0a5d554764249f8a7eefdc943271090 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.038142 14819 raft_consensus.cc:2267] T b0a5d554764249f8a7eefdc943271090 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:17.038745 14358 leader_election.cc:343] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 054251151c294e06905092f2a9306849
I20250115 03:06:17.038808 14358 leader_election.cc:304] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.039021 15401 raft_consensus.cc:2743] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.042351 14398 tablet_service.cc:1467] Processing CreateTablet for tablet d891020ed0bb4fc49155bd9f2f6ed35c (DEFAULT_TABLE table=default.Table_73f837db71d74dae9606eb6995ad7270 [id=ff302d72ca26434f9a86f478f3346318]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.042582 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d891020ed0bb4fc49155bd9f2f6ed35c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.042635 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 6ed47b26cdf744acb28fb657a1b41087 (DEFAULT_TABLE table=default.Table_4f5d0494ec384c479419c10b7ff36dc0 [id=b9026c7e60a14240b2f5c596d31ba517]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.042865 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ed47b26cdf744acb28fb657a1b41087. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.043238 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet d0c3536533e94174bb1e3888a3458395 (table default.Table_1c52ca26880a48289f5916fabdb05ac4 [id=44c03f21942545319762494029cebc02]) successfully deleted
I20250115 03:06:17.043381 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 6110b6c4911d4932b4cac4d86598a5c6 (table default.Table_3e911fd952934f79bee759f184cbcce0 [id=0cbd9b74bdc84d9291bd5abee773d650]) successfully deleted
I20250115 03:06:17.043457 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet a4484651d964411e90ed93c6caab0a13 (table default.Table_2db28e1f2ea5450092ed00f593c4939f [id=6fe98239ee3346df8b25712aaec67b82]) successfully deleted
W20250115 03:06:17.043545 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 80ede4e9696d491aa385b375fcb5cf3b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 80ede4e9696d491aa385b375fcb5cf3b already in progress: creating tablet
I20250115 03:06:17.043619 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 304fab45b38e4426ae6961ae786e5659 (table default.Table_d407111658854f02aa8387a067f33f37 [id=526370a40a0d4f7aa96c8348412d8b26]) successfully deleted
I20250115 03:06:17.043856 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 4ff51d514fb84809aa53fa9fc1010681 (table default.Table_a60b99b368f74c68873e8c89066cd3e0 [id=98f48acd89a6461bb4511141cab4d88b]) successfully deleted
W20250115 03:06:17.043963 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 4122c97f5ba445ee8c79b7d056438229 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4122c97f5ba445ee8c79b7d056438229 already in progress: creating tablet
I20250115 03:06:17.044320 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 20f86f93812849c4b7852a884f677c04 (table default.Table_50c659a42a154d46a26879df0101bf23 [id=f8f2a7ff6fdc40468de6ae9ba28c1427]) successfully deleted
I20250115 03:06:17.044479 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 6bd4e4200f9b489ea5dd2154aa6f382d (DEFAULT_TABLE table=default.Table_2e3e4b87e2e04feaab40ea415807cd06 [id=299ac71cfb98480f873eb659419e522c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.044585 14662 tablet_service.cc:1467] Processing CreateTablet for tablet d891020ed0bb4fc49155bd9f2f6ed35c (DEFAULT_TABLE table=default.Table_73f837db71d74dae9606eb6995ad7270 [id=ff302d72ca26434f9a86f478f3346318]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.044678 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bd4e4200f9b489ea5dd2154aa6f382d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.044888 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d891020ed0bb4fc49155bd9f2f6ed35c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.046490 14669 tablet_service.cc:1467] Processing CreateTablet for tablet c2afa048b3fd4362b50c8a6da306c25a (DEFAULT_TABLE table=default.Table_478e448d8f6d494d90d421e2f0cd21fc [id=74853e82e7024da6a9e4e1dd23bd043d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.046947 14820 tablet_replica.cc:331] T 872ad705e3e94c0eab4397df9f28f9ca P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:17.046974 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 6bd4e4200f9b489ea5dd2154aa6f382d (DEFAULT_TABLE table=default.Table_2e3e4b87e2e04feaab40ea415807cd06 [id=299ac71cfb98480f873eb659419e522c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.047027 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 82be772e05d443749914004aea8af300 (table default.Table_3863dfaf57694ee7b4a9ba1e400d7c9d [id=831e8e5e33ae41368950b0b8fce37e9b]) successfully deleted
I20250115 03:06:17.047067 14820 raft_consensus.cc:2238] T 872ad705e3e94c0eab4397df9f28f9ca P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.047179 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bd4e4200f9b489ea5dd2154aa6f382d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.047175 14820 raft_consensus.cc:2267] T 872ad705e3e94c0eab4397df9f28f9ca P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.047243 14823 ts_tablet_manager.cc:1918] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:17.047494 14820 ts_tablet_manager.cc:1905] T 872ad705e3e94c0eab4397df9f28f9ca P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.047500 14823 log.cc:1198] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/e594ea5cff81475886cdceb8400645a6
I20250115 03:06:17.048789 14811 tablet_bootstrap.cc:492] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:17.048856 14811 ts_tablet_manager.cc:1397] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20250115 03:06:17.049106 14396 tablet_service.cc:1467] Processing CreateTablet for tablet c2afa048b3fd4362b50c8a6da306c25a (DEFAULT_TABLE table=default.Table_478e448d8f6d494d90d421e2f0cd21fc [id=74853e82e7024da6a9e4e1dd23bd043d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.049307 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2afa048b3fd4362b50c8a6da306c25a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.049273 14811 raft_consensus.cc:357] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.049364 14811 raft_consensus.cc:383] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.049399 14811 raft_consensus.cc:738] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.049520 14811 consensus_queue.cc:260] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.049566 14820 ts_tablet_manager.cc:1918] T 872ad705e3e94c0eab4397df9f28f9ca P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.049993 14823 ts_tablet_manager.cc:1939] T e594ea5cff81475886cdceb8400645a6 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:17.054673 14819 ts_tablet_manager.cc:1905] T b0a5d554764249f8a7eefdc943271090 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.056713 14812 tablet_bootstrap.cc:492] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.056788 14812 ts_tablet_manager.cc:1397] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.176s user 0.000s sys 0.001s
I20250115 03:06:17.057230 14812 raft_consensus.cc:357] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.057319 14819 ts_tablet_manager.cc:1918] T b0a5d554764249f8a7eefdc943271090 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.057329 14812 raft_consensus.cc:383] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.057358 14812 raft_consensus.cc:738] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.057374 14819 log.cc:1198] T b0a5d554764249f8a7eefdc943271090 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/b0a5d554764249f8a7eefdc943271090
I20250115 03:06:17.057487 14812 consensus_queue.cc:260] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.057554 14819 ts_tablet_manager.cc:1939] T b0a5d554764249f8a7eefdc943271090 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.057786 14812 ts_tablet_manager.cc:1428] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.049778 14811 ts_tablet_manager.cc:1428] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.058432 14811 tablet_bootstrap.cc:492] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:17.059412 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet b2629ae9d9854ae4b6e7ddddbb6f341b from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:17.059581 14811 tablet_bootstrap.cc:654] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.059765 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 6ed47b26cdf744acb28fb657a1b41087 (DEFAULT_TABLE table=default.Table_4f5d0494ec384c479419c10b7ff36dc0 [id=b9026c7e60a14240b2f5c596d31ba517]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.059967 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ed47b26cdf744acb28fb657a1b41087. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.061758 14812 tablet_bootstrap.cc:492] T 858bbbb62fc249d5a915697c12362d0f P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:17.061764 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2afa048b3fd4362b50c8a6da306c25a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.062822 14812 tablet_bootstrap.cc:654] T 858bbbb62fc249d5a915697c12362d0f P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.064174 15857 consensus_queue.cc:1035] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:17.065202 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 074fdc09e4e24078ad0bb588b32ff24f consensus state has no leader
I20250115 03:06:17.049614 14820 log.cc:1198] T 872ad705e3e94c0eab4397df9f28f9ca P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/872ad705e3e94c0eab4397df9f28f9ca
W20250115 03:06:17.068809 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5a2894e4736b4191963a61d908bc673c has no consensus state
I20250115 03:06:17.069051 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet b2629ae9d9854ae4b6e7ddddbb6f341b
I20250115 03:06:17.069270 14162 catalog_manager.cc:6596] ReplaceTablet: tablet b2629ae9d9854ae4b6e7ddddbb6f341b deleted and replaced by tablet 103f78d0d9fc42308e1e23316895be33
I20250115 03:06:17.069381 14167 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_590b26746c6a4006bb430ba9dcbacafc [id=021f5685630741a8880d2471f7d2d1d2] alter complete (version 1)
I20250115 03:06:17.069411 14820 ts_tablet_manager.cc:1939] T 872ad705e3e94c0eab4397df9f28f9ca P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:17.069914 15401 raft_consensus.cc:491] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.069984 15401 raft_consensus.cc:513] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.070259 15401 leader_election.cc:290] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.070353 15390 raft_consensus.cc:491] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.070430 15390 raft_consensus.cc:513] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.070746 15390 leader_election.cc:290] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.070904 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed8538c4a44d18279371b826c22a5" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:17.071002 14688 raft_consensus.cc:2463] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:17.071246 14358 leader_election.cc:304] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:17.071349 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 872ad705e3e94c0eab4397df9f28f9ca (table default.Table_afc5c7e955fb4b3ba43151cb8823a76b [id=bac33f20b4524a05bde7452db44d1104]) successfully deleted
I20250115 03:06:17.071664 15401 raft_consensus.cc:2798] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:17.071723 15401 raft_consensus.cc:491] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:17.071759 15401 raft_consensus.cc:3054] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.072515 15401 raft_consensus.cc:513] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.072813 15401 leader_election.cc:290] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.072856 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet b2629ae9d9854ae4b6e7ddddbb6f341b with delete_type TABLET_DATA_DELETED (replaced by tablet 103f78d0d9fc42308e1e23316895be33) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:17.074155 14358 leader_election.cc:343] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 6110b6c4911d4932b4cac4d86598a5c6
I20250115 03:06:17.074152 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed8538c4a44d18279371b826c22a5" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:17.074247 14688 raft_consensus.cc:3054] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.074608 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed8538c4a44d18279371b826c22a5" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:17.074810 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6110b6c4911d4932b4cac4d86598a5c6" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:17.074983 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d28ed8538c4a44d18279371b826c22a5" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:17.075098 14688 raft_consensus.cc:2463] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:17.075388 14358 leader_election.cc:304] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:17.075444 14819 tablet_replica.cc:331] T 3a0bb9b5093b454d921bf1150d631565 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.075551 15390 raft_consensus.cc:2798] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:17.075582 14819 raft_consensus.cc:2238] T 3a0bb9b5093b454d921bf1150d631565 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.075618 15390 raft_consensus.cc:695] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:17.075670 14819 raft_consensus.cc:2267] T 3a0bb9b5093b454d921bf1150d631565 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.075739 15390 consensus_queue.cc:237] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.076083 14820 tablet_replica.cc:331] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:17.076200 14820 raft_consensus.cc:2238] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
W20250115 03:06:17.076308 14356 leader_election.cc:343] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:17.076470 14356 leader_election.cc:343] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.076526 14356 leader_election.cc:304] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.076685 14819 ts_tablet_manager.cc:1905] T 3a0bb9b5093b454d921bf1150d631565 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.076663 15401 raft_consensus.cc:2743] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.077481 15857 raft_consensus.cc:491] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.077555 15857 raft_consensus.cc:513] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.077836 15857 leader_election.cc:290] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.078002 14163 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet df44c5f5a9ae4d1aa7647dcbe2303a39 (table default.Table_098c9884c0494d4592f142b4a197f38c [id=8bf1da8d633940258affd0e11c334786]). Expected version 1 got 0
W20250115 03:06:17.078409 14493 leader_election.cc:343] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: acb43d614111477abb10def48a1dfb3b
I20250115 03:06:17.079329 14669 tablet_service.cc:1514] Processing DeleteTablet for tablet b2629ae9d9854ae4b6e7ddddbb6f341b with delete_type TABLET_DATA_DELETED (replaced by tablet 103f78d0d9fc42308e1e23316895be33) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.079478 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a0bb9b5093b454d921bf1150d631565 with delete_type TABLET_DATA_DELETED (replaced by tablet 5589b497b0e24740ad49d1392f5a737b) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.079527 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet b2629ae9d9854ae4b6e7ddddbb6f341b with delete_type TABLET_DATA_DELETED (replaced by tablet 103f78d0d9fc42308e1e23316895be33) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:17.076704 14356 leader_election.cc:343] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.079643 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet a75b9401cb4b4f8b92a4779c3704eb46 with delete_type TABLET_DATA_DELETED (replaced by tablet 21524aef80f74a6c899ea66b9b022389) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.081439 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet b0a5d554764249f8a7eefdc943271090 (table default.Table_39e0d37b84fc42c0807a4cbffb7bae0a [id=4b6ce1381c024498a78d91e346f488f2]) successfully deleted
I20250115 03:06:17.082577 14819 ts_tablet_manager.cc:1918] T 3a0bb9b5093b454d921bf1150d631565 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.082633 14819 log.cc:1198] T 3a0bb9b5093b454d921bf1150d631565 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/3a0bb9b5093b454d921bf1150d631565
I20250115 03:06:17.082811 14819 ts_tablet_manager.cc:1939] T 3a0bb9b5093b454d921bf1150d631565 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
W20250115 03:06:17.083316 14493 leader_election.cc:343] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: acb43d614111477abb10def48a1dfb3b
I20250115 03:06:17.083381 14493 leader_election.cc:304] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.083575 14809 tablet_bootstrap.cc:492] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.083573 14858 raft_consensus.cc:2743] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.083647 14809 ts_tablet_manager.cc:1397] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.048s user 0.001s sys 0.000s
I20250115 03:06:17.083586 15857 raft_consensus.cc:491] T eeff2cd0825c4419aacfc73ebe267a8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.083752 15857 raft_consensus.cc:513] T eeff2cd0825c4419aacfc73ebe267a8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.084055 15857 leader_election.cc:290] T eeff2cd0825c4419aacfc73ebe267a8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.084097 14809 raft_consensus.cc:357] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.084185 14809 raft_consensus.cc:383] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.084218 14809 raft_consensus.cc:738] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.084319 14809 consensus_queue.cc:260] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.084575 14809 ts_tablet_manager.cc:1428] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.084748 14809 tablet_bootstrap.cc:492] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.084762 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeff2cd0825c4419aacfc73ebe267a8e" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:17.085757 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeff2cd0825c4419aacfc73ebe267a8e" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:17.085937 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_62e6b3005d334927ae377d56649bda6d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:17.086099 14493 leader_election.cc:304] T eeff2cd0825c4419aacfc73ebe267a8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
W20250115 03:06:17.086278 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_62e6b3005d334927ae377d56649bda6d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.086280 15857 raft_consensus.cc:2743] T eeff2cd0825c4419aacfc73ebe267a8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.086800 14819 tablet_replica.cc:331] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.086910 14819 raft_consensus.cc:2238] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:17.087029 14819 pending_rounds.cc:62] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75: Trying to abort 1 pending ops.
I20250115 03:06:17.087075 14809 tablet_bootstrap.cc:654] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.087756 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_77d4ac27a2c44250a2dcc62121f202c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.088052 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_77d4ac27a2c44250a2dcc62121f202c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.088681 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_39b71235b2df40ec88411bb02d370d7b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:17.088809 14812 tablet_bootstrap.cc:492] T 858bbbb62fc249d5a915697c12362d0f P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.088883 14812 ts_tablet_manager.cc:1397] T 858bbbb62fc249d5a915697c12362d0f P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
W20250115 03:06:17.088995 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_39b71235b2df40ec88411bb02d370d7b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.089345 14812 raft_consensus.cc:357] T 858bbbb62fc249d5a915697c12362d0f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.089434 14812 raft_consensus.cc:383] T 858bbbb62fc249d5a915697c12362d0f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.089471 14812 raft_consensus.cc:738] T 858bbbb62fc249d5a915697c12362d0f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.089574 14812 consensus_queue.cc:260] T 858bbbb62fc249d5a915697c12362d0f P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.089815 14812 ts_tablet_manager.cc:1428] T 858bbbb62fc249d5a915697c12362d0f P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.089965 14812 tablet_bootstrap.cc:492] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:17.090888 14812 tablet_bootstrap.cc:654] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.087057 14819 pending_rounds.cc:69] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384902809436160 op_type: NO_OP noop_request { }
I20250115 03:06:17.091212 14819 raft_consensus.cc:2883] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:17.091279 14819 raft_consensus.cc:2267] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.093729 14819 ts_tablet_manager.cc:1905] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.093902 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet e594ea5cff81475886cdceb8400645a6 (table default.Table_bd6a79400bc244d69676d8df1069cbb2 [id=e8ef3a6c99c14ba585efea22568617a9]) successfully deleted
W20250115 03:06:17.094069 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet b2629ae9d9854ae4b6e7ddddbb6f341b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b2629ae9d9854ae4b6e7ddddbb6f341b already in progress: creating tablet
I20250115 03:06:17.094182 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 3a0bb9b5093b454d921bf1150d631565 (table default.Table_8daa0abab4d1464488bec21bb7ad375b [id=37d965d8a5eb408ea3f33313bd22c152]) successfully deleted
I20250115 03:06:17.095428 15401 consensus_queue.cc:1035] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.096969 14819 ts_tablet_manager.cc:1918] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:17.097035 14819 log.cc:1198] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/872ad705e3e94c0eab4397df9f28f9ca
I20250115 03:06:17.097239 14819 ts_tablet_manager.cc:1939] T 872ad705e3e94c0eab4397df9f28f9ca P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.098531 14537 tablet_service.cc:1467] Processing CreateTablet for tablet fca14a7d574842ee94a93afb85ca1afd (DEFAULT_TABLE table=default.Table_b169e386c2cc424da59a39503cec548f [id=2b97c2f5b5284cd8a6311247712cc447]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.098778 14529 tablet_service.cc:1467] Processing CreateTablet for tablet 5a2894e4736b4191963a61d908bc673c (DEFAULT_TABLE table=default.Table_df79ed42f26f4b68b197877c445b9665 [id=bb6453f2ab97404cbd3ba32a87976e52]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.099012 14528 tablet_service.cc:1467] Processing CreateTablet for tablet ca117aca95f640be880b640c85e18cd4 (DEFAULT_TABLE table=default.Table_a82405d576ec40228fe5ad55b98753e8 [id=9ce40fc0fe094c2da26bf1941aeaa657]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.099216 14531 tablet_service.cc:1467] Processing CreateTablet for tablet b61cc4b11251486a8089ca9bf44ac91e (DEFAULT_TABLE table=default.Table_f634ab8b7b7c437f90035122e8f974b9 [id=555a4a4508a1409fb8febc47add30a1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.099843 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 0b5fd91a0ea64b3b85b7571e5daa08b2 (DEFAULT_TABLE table=default.Table_d407111658854f02aa8387a067f33f37 [id=526370a40a0d4f7aa96c8348412d8b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.100538 15401 consensus_queue.cc:1035] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.101078 14166 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0964201dda974eaabef3d5694c40389c" } modify_external_catalogs: true
I20250115 03:06:17.101171 14166 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0964201dda974eaabef3d5694c40389c" } modify_external_catalogs: true
I20250115 03:06:17.101502 14809 tablet_bootstrap.cc:492] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.101581 14809 ts_tablet_manager.cc:1397] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20250115 03:06:17.101910 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_35de20a1cb6a488bbc86997188d7ce82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:17.101979 14809 raft_consensus.cc:357] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.102066 14809 raft_consensus.cc:383] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.102095 14809 raft_consensus.cc:738] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
W20250115 03:06:17.102197 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_35de20a1cb6a488bbc86997188d7ce82 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.102195 14809 consensus_queue.cc:260] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.102408 14168 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 745344db22da4102bad39c2f121548a1 (table default.Table_365313892043408c92be8460c31d958e [id=f22f08e9af4a4496b100e7d4b5982121]). Expected version 1 got 0
I20250115 03:06:17.102491 14809 ts_tablet_manager.cc:1428] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.102691 14809 tablet_bootstrap.cc:492] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.103142 14163 catalog_manager.cc:5526] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:17.103320 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet a1c93bfd3da44f9f8a8d585f2d21f1be (table default.Table_40b1953a4a364eb1ae58e3387fc65a91 [id=64d20b1792044c5e8add735aaf75a6d9]). Expected version 1 got 0
I20250115 03:06:17.103441 14163 catalog_manager.cc:5526] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:17.103581 14809 tablet_bootstrap.cc:654] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.103647 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet a4c3ff892f9b4dc484ffc80d7c5934c4 (table default.Table_76ef9277dc8142f1a86962bfc430f2b2 [id=f70558594b8348579a0ad5d4c8fd8885]). Expected version 1 got 0
I20250115 03:06:17.103767 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet dcdb5282e41c43dbb235ae78465d19f9 (table default.Table_375bb4e2095c4675b734247fabacbec7 [id=b298dce7ab664327893633f821ed4d89]). Expected version 1 got 0
I20250115 03:06:17.103827 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5986558350244e7aa4ddcd5cccb7d676 (table default.Table_e72d1ee69400410c9da9082d4bd8bc9a [id=6ab343bee1254fb19a3024ebafa44464]). Expected version 1 got 0
I20250115 03:06:17.103899 14163 catalog_manager.cc:5526] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:17.104039 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 13f36a563f13403eab372deb10b67451 (table default.Table_70d0b700eac34892b4ef769d739481fd [id=61f3625fed2b4a70b5331d6c6ac73f1c]). Expected version 1 got 0
I20250115 03:06:17.104696 14819 tablet_replica.cc:331] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.105130 14819 raft_consensus.cc:2238] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.105222 14819 raft_consensus.cc:2267] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.105614 14819 ts_tablet_manager.cc:1905] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.107165 14396 tablet_service.cc:1467] Processing CreateTablet for tablet fca14a7d574842ee94a93afb85ca1afd (DEFAULT_TABLE table=default.Table_b169e386c2cc424da59a39503cec548f [id=2b97c2f5b5284cd8a6311247712cc447]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.107363 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fca14a7d574842ee94a93afb85ca1afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.108700 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2299c208a52a4d99ba20b1f3b5783fbd" } new_table_name: "default.Table_655e9d02aacf4b88ba521f0f57a5f522" modify_external_catalogs: true
I20250115 03:06:17.109329 14819 ts_tablet_manager.cc:1918] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.109505 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_77127dee522147cf8e0f3ec2f5539301"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.109782 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_77127dee522147cf8e0f3ec2f5539301 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.110678 14171 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ff993b0824114ebdb66d40758eb3b827" } new_table_name: "default.Table_b165e5d12ca845b9984c99dde5e2e06e" modify_external_catalogs: true
I20250115 03:06:17.111323 14811 tablet_bootstrap.cc:492] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:17.111394 14811 ts_tablet_manager.cc:1397] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.053s user 0.002s sys 0.000s
I20250115 03:06:17.111806 14811 raft_consensus.cc:357] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.111899 14811 raft_consensus.cc:383] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.111928 14811 raft_consensus.cc:738] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.112020 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 745344db22da4102bad39c2f121548a1 (table default.Table_365313892043408c92be8460c31d958e [id=f22f08e9af4a4496b100e7d4b5982121]). Expected version 1 got 0
I20250115 03:06:17.112031 14811 consensus_queue.cc:260] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.112288 14811 ts_tablet_manager.cc:1428] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.113062 14669 tablet_service.cc:1467] Processing CreateTablet for tablet fca14a7d574842ee94a93afb85ca1afd (DEFAULT_TABLE table=default.Table_b169e386c2cc424da59a39503cec548f [id=2b97c2f5b5284cd8a6311247712cc447]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.113307 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fca14a7d574842ee94a93afb85ca1afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.114665 14819 log.cc:1198] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/054251151c294e06905092f2a9306849
I20250115 03:06:17.114866 14819 ts_tablet_manager.cc:1939] T 054251151c294e06905092f2a9306849 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.115103 15453 raft_consensus.cc:491] T b6175f691fec45bf9853ee90959a1c98 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.115192 15453 raft_consensus.cc:513] T b6175f691fec45bf9853ee90959a1c98 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.115517 14819 tablet_replica.cc:331] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.115530 15453 leader_election.cc:290] T b6175f691fec45bf9853ee90959a1c98 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:17.115630 14819 raft_consensus.cc:2238] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:17.115748 14819 pending_rounds.cc:62] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75: Trying to abort 1 pending ops.
I20250115 03:06:17.115772 14819 pending_rounds.cc:69] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384903282507776 op_type: NO_OP noop_request { }
I20250115 03:06:17.115816 14819 raft_consensus.cc:2883] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:17.115857 14819 raft_consensus.cc:2267] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.115911 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6175f691fec45bf9853ee90959a1c98" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:17.116057 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6175f691fec45bf9853ee90959a1c98" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:17.116494 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 5a2894e4736b4191963a61d908bc673c (DEFAULT_TABLE table=default.Table_df79ed42f26f4b68b197877c445b9665 [id=bb6453f2ab97404cbd3ba32a87976e52]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:17.116549 14628 leader_election.cc:343] T b6175f691fec45bf9853ee90959a1c98 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:17.116655 14626 leader_election.cc:343] T b6175f691fec45bf9853ee90959a1c98 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.116698 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a2894e4736b4191963a61d908bc673c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.116721 14626 leader_election.cc:304] T b6175f691fec45bf9853ee90959a1c98 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.116905 15453 raft_consensus.cc:2743] T b6175f691fec45bf9853ee90959a1c98 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.118237 14819 ts_tablet_manager.cc:1905] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.118814 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 5a2894e4736b4191963a61d908bc673c (DEFAULT_TABLE table=default.Table_df79ed42f26f4b68b197877c445b9665 [id=bb6453f2ab97404cbd3ba32a87976e52]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.119031 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a2894e4736b4191963a61d908bc673c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.119961 14819 ts_tablet_manager.cc:1918] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:17.120026 14819 log.cc:1198] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/304fab45b38e4426ae6961ae786e5659
I20250115 03:06:17.120254 14819 ts_tablet_manager.cc:1939] T 304fab45b38e4426ae6961ae786e5659 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.120492 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet b0a5d554764249f8a7eefdc943271090 with delete_type TABLET_DATA_DELETED (replaced by tablet d895f75911b7456a88e7a5159db7d60b) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.120837 14819 tablet_replica.cc:331] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.120970 14819 raft_consensus.cc:2238] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:17.121127 14819 raft_consensus.cc:2267] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.122090 14669 tablet_service.cc:1467] Processing CreateTablet for tablet ca117aca95f640be880b640c85e18cd4 (DEFAULT_TABLE table=default.Table_a82405d576ec40228fe5ad55b98753e8 [id=9ce40fc0fe094c2da26bf1941aeaa657]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.122340 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca117aca95f640be880b640c85e18cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.123854 15857 raft_consensus.cc:491] T b1c0f6a5611647eb884c77f7a6d1cd4e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.123955 15857 raft_consensus.cc:513] T b1c0f6a5611647eb884c77f7a6d1cd4e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.124233 15857 leader_election.cc:290] T b1c0f6a5611647eb884c77f7a6d1cd4e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.124884 14819 ts_tablet_manager.cc:1905] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.125147 14396 tablet_service.cc:1467] Processing CreateTablet for tablet ca117aca95f640be880b640c85e18cd4 (DEFAULT_TABLE table=default.Table_a82405d576ec40228fe5ad55b98753e8 [id=9ce40fc0fe094c2da26bf1941aeaa657]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.125339 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca117aca95f640be880b640c85e18cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.125722 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1c0f6a5611647eb884c77f7a6d1cd4e" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:17.126598 14819 ts_tablet_manager.cc:1918] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:17.126657 14819 log.cc:1198] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/4ff51d514fb84809aa53fa9fc1010681
I20250115 03:06:17.126863 14819 ts_tablet_manager.cc:1939] T 4ff51d514fb84809aa53fa9fc1010681 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.128325 14812 tablet_bootstrap.cc:492] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.128407 14812 ts_tablet_manager.cc:1397] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.000s
I20250115 03:06:17.128837 14812 raft_consensus.cc:357] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.128943 14812 raft_consensus.cc:383] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.128914 15857 raft_consensus.cc:491] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.128976 14812 raft_consensus.cc:738] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.129004 15857 raft_consensus.cc:513] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.129076 14812 consensus_queue.cc:260] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.129289 15857 leader_election.cc:290] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.129320 14812 ts_tablet_manager.cc:1428] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.129493 14812 tablet_bootstrap.cc:492] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:17.129976 14493 leader_election.cc:343] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 506198162e0b4e24924c408542707c4c
I20250115 03:06:17.130357 14812 tablet_bootstrap.cc:654] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.130949 15857 consensus_queue.cc:1035] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:17.131865 14493 leader_election.cc:343] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 506198162e0b4e24924c408542707c4c
I20250115 03:06:17.131925 14493 leader_election.cc:304] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.132086 15857 raft_consensus.cc:2743] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.132701 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a82405d576ec40228fe5ad55b98753e8" } modify_external_catalogs: true
I20250115 03:06:17.132743 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_76ef9277dc8142f1a86962bfc430f2b2" } modify_external_catalogs: true
I20250115 03:06:17.132822 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a82405d576ec40228fe5ad55b98753e8" } modify_external_catalogs: true
I20250115 03:06:17.132961 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_24b2f1f4bbb14d2fa7cba35e089f6707" } modify_external_catalogs: true
I20250115 03:06:17.133028 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_24b2f1f4bbb14d2fa7cba35e089f6707" } modify_external_catalogs: true
I20250115 03:06:17.132833 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_76ef9277dc8142f1a86962bfc430f2b2" } modify_external_catalogs: true
I20250115 03:06:17.134354 14820 raft_consensus.cc:2267] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.134907 14820 ts_tablet_manager.cc:1905] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.135650 14669 tablet_service.cc:1467] Processing CreateTablet for tablet b61cc4b11251486a8089ca9bf44ac91e (DEFAULT_TABLE table=default.Table_f634ab8b7b7c437f90035122e8f974b9 [id=555a4a4508a1409fb8febc47add30a1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.136034 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b61cc4b11251486a8089ca9bf44ac91e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.137022 14738 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:17.131090 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1c0f6a5611647eb884c77f7a6d1cd4e" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:17.138834 14422 raft_consensus.cc:2388] T b1c0f6a5611647eb884c77f7a6d1cd4e P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in current term 1: Already voted for candidate 5c0dae7aff4b44c699275b95ba639770 in this term.
I20250115 03:06:17.139106 14493 leader_election.cc:304] T b1c0f6a5611647eb884c77f7a6d1cd4e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.139310 14819 tablet_replica.cc:331] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.139375 15857 raft_consensus.cc:3054] T b1c0f6a5611647eb884c77f7a6d1cd4e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.139497 14819 raft_consensus.cc:2238] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:17.139668 14819 raft_consensus.cc:2267] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.139815 15401 raft_consensus.cc:491] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.139873 15401 raft_consensus.cc:513] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.140141 15401 leader_election.cc:290] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.140251 15857 raft_consensus.cc:2743] T b1c0f6a5611647eb884c77f7a6d1cd4e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.140476 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58161a322a3d469c8444d353689bed65" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:17.140578 14687 raft_consensus.cc:2463] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:17.140744 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58161a322a3d469c8444d353689bed65" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:17.140842 14358 leader_election.cc:304] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:17.141063 15401 raft_consensus.cc:2798] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:17.141108 15401 raft_consensus.cc:491] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:17.141139 15401 raft_consensus.cc:3054] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:17.141149 14356 leader_election.cc:343] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.141911 15401 raft_consensus.cc:513] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.142122 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet bb68869eaf55477090cf12f40f76a89c
I20250115 03:06:17.142208 15401 leader_election.cc:290] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.142683 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58161a322a3d469c8444d353689bed65" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:17.142791 14688 raft_consensus.cc:3054] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.143553 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58161a322a3d469c8444d353689bed65" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:17.143597 14819 ts_tablet_manager.cc:1905] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.143649 14688 raft_consensus.cc:2463] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:17.143911 14358 leader_election.cc:304] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
W20250115 03:06:17.143999 14356 leader_election.cc:343] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.144052 14885 raft_consensus.cc:2798] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:17.144057 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet bb68869eaf55477090cf12f40f76a89c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.144112 14885 raft_consensus.cc:695] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:17.144212 14885 consensus_queue.cc:237] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.145128 15453 consensus_queue.cc:1035] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.140642 14396 tablet_service.cc:1467] Processing CreateTablet for tablet b61cc4b11251486a8089ca9bf44ac91e (DEFAULT_TABLE table=default.Table_f634ab8b7b7c437f90035122e8f974b9 [id=555a4a4508a1409fb8febc47add30a1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.145526 14819 ts_tablet_manager.cc:1918] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:17.145582 14819 log.cc:1198] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/a4484651d964411e90ed93c6caab0a13
I20250115 03:06:17.145565 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b61cc4b11251486a8089ca9bf44ac91e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.145797 14819 ts_tablet_manager.cc:1939] T a4484651d964411e90ed93c6caab0a13 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.146533 14819 tablet_replica.cc:331] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.146646 14819 raft_consensus.cc:2238] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:17.146795 14819 raft_consensus.cc:2267] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.152998 14819 ts_tablet_manager.cc:1905] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.153537 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 745344db22da4102bad39c2f121548a1 (table default.Table_365313892043408c92be8460c31d958e [id=f22f08e9af4a4496b100e7d4b5982121]). Expected version 1 got 0
I20250115 03:06:17.154016 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_1fd65820535a4dc7a4fc884c4184bba4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:17.154242 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 0b5fd91a0ea64b3b85b7571e5daa08b2 (DEFAULT_TABLE table=default.Table_d407111658854f02aa8387a067f33f37 [id=526370a40a0d4f7aa96c8348412d8b26]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:17.154369 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1fd65820535a4dc7a4fc884c4184bba4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.154466 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b5fd91a0ea64b3b85b7571e5daa08b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.155233 14819 ts_tablet_manager.cc:1918] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:17.155359 14819 log.cc:1198] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/e594ea5cff81475886cdceb8400645a6
I20250115 03:06:17.155639 14819 ts_tablet_manager.cc:1939] T e594ea5cff81475886cdceb8400645a6 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.156147 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 872ad705e3e94c0eab4397df9f28f9ca (table default.Table_afc5c7e955fb4b3ba43151cb8823a76b [id=bac33f20b4524a05bde7452db44d1104]) successfully deleted
I20250115 03:06:17.156589 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 054251151c294e06905092f2a9306849 (table default.Table_92b7f10774fe4db0ac9872d33fceb7a7 [id=54fd71a4b30d41b7b59b8dfc261a616d]) successfully deleted
I20250115 03:06:17.156827 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 304fab45b38e4426ae6961ae786e5659 (table default.Table_d407111658854f02aa8387a067f33f37 [id=526370a40a0d4f7aa96c8348412d8b26]) successfully deleted
W20250115 03:06:17.157063 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 80ede4e9696d491aa385b375fcb5cf3b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 80ede4e9696d491aa385b375fcb5cf3b already in progress: creating tablet
I20250115 03:06:17.157490 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 4ff51d514fb84809aa53fa9fc1010681 (table default.Table_a60b99b368f74c68873e8c89066cd3e0 [id=98f48acd89a6461bb4511141cab4d88b]) successfully deleted
I20250115 03:06:17.157698 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet a4484651d964411e90ed93c6caab0a13 (table default.Table_2db28e1f2ea5450092ed00f593c4939f [id=6fe98239ee3346df8b25712aaec67b82]) successfully deleted
I20250115 03:06:17.157747 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet 858bbbb62fc249d5a915697c12362d0f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.157900 14823 tablet_replica.cc:331] T 858bbbb62fc249d5a915697c12362d0f P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
W20250115 03:06:17.157963 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 4122c97f5ba445ee8c79b7d056438229 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4122c97f5ba445ee8c79b7d056438229 already in progress: creating tablet
I20250115 03:06:17.158026 14823 raft_consensus.cc:2238] T 858bbbb62fc249d5a915697c12362d0f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.158105 14823 raft_consensus.cc:2267] T 858bbbb62fc249d5a915697c12362d0f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.158532 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet bb68869eaf55477090cf12f40f76a89c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.158978 14823 ts_tablet_manager.cc:1905] T 858bbbb62fc249d5a915697c12362d0f P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.159868 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet bb68869eaf55477090cf12f40f76a89c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.160504 14819 tablet_replica.cc:331] T a75b9401cb4b4f8b92a4779c3704eb46 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.160665 14819 raft_consensus.cc:2238] T a75b9401cb4b4f8b92a4779c3704eb46 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.160800 14819 raft_consensus.cc:2267] T a75b9401cb4b4f8b92a4779c3704eb46 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.161016 14823 ts_tablet_manager.cc:1918] T 858bbbb62fc249d5a915697c12362d0f P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.161083 14823 log.cc:1198] T 858bbbb62fc249d5a915697c12362d0f P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/858bbbb62fc249d5a915697c12362d0f
I20250115 03:06:17.161258 14823 ts_tablet_manager.cc:1939] T 858bbbb62fc249d5a915697c12362d0f P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:17.161429 14819 ts_tablet_manager.cc:1905] T a75b9401cb4b4f8b92a4779c3704eb46 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.162835 14820 ts_tablet_manager.cc:1918] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:17.163187 14819 ts_tablet_manager.cc:1918] T a75b9401cb4b4f8b92a4779c3704eb46 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.164024 14819 log.cc:1198] T a75b9401cb4b4f8b92a4779c3704eb46 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/a75b9401cb4b4f8b92a4779c3704eb46
I20250115 03:06:17.164279 14819 ts_tablet_manager.cc:1939] T a75b9401cb4b4f8b92a4779c3704eb46 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.170308 14170 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 6864f88bf6454ee980f624ec2f2d9fb6 (table default.Table_2e68c93eaa1647df9b269879bd4520ab [id=18e0f06feeda4bca860e6f1bc6eab15b]). Expected version 1 got 0
I20250115 03:06:17.170688 14170 master_service.cc:946] ReplaceTablet: received request to replace tablet 312dcdda8cf042648db9fdf6228062d9 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:17.171506 15453 consensus_queue.cc:1035] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.178622 15453 consensus_queue.cc:1035] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.188629 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 0b5fd91a0ea64b3b85b7571e5daa08b2 (DEFAULT_TABLE table=default.Table_d407111658854f02aa8387a067f33f37 [id=526370a40a0d4f7aa96c8348412d8b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.189319 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b5fd91a0ea64b3b85b7571e5daa08b2. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:17.189932 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 15fbf8d5de904fc4b284af664f3bd60f consensus state has no leader
I20250115 03:06:17.190104 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet 7d894471f7b94aec8a19df043906d7e7 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:17.192128 14861 raft_consensus.cc:491] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.192199 14861 raft_consensus.cc:513] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.192490 14861 leader_election.cc:290] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.193171 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6f61dc685a48bdb367278773bf1d90" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:17.193285 14689 raft_consensus.cc:2463] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:17.194129 14358 leader_election.cc:304] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:17.194351 14861 raft_consensus.cc:2798] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:17.194392 14861 raft_consensus.cc:491] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:17.194422 14861 raft_consensus.cc:3054] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.195187 14861 raft_consensus.cc:513] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.195497 14861 leader_election.cc:290] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.196334 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6f61dc685a48bdb367278773bf1d90" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:17.196432 14689 raft_consensus.cc:3054] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:17.196436 14356 consensus_peers.cc:487] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:17.197338 14689 raft_consensus.cc:2463] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:17.197842 14358 leader_election.cc:304] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:17.197996 15390 raft_consensus.cc:2798] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:17.198065 15390 raft_consensus.cc:695] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:17.198174 15390 consensus_queue.cc:237] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.199627 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f3b64ddeafb3400583229c6698c2a5e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.199961 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f3b64ddeafb3400583229c6698c2a5e5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.205739 14820 log.cc:1198] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/4ff51d514fb84809aa53fa9fc1010681
I20250115 03:06:17.183804 14811 tablet_bootstrap.cc:492] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:17.192763 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6f61dc685a48bdb367278773bf1d90" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:17.195747 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6f61dc685a48bdb367278773bf1d90" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:17.207051 14356 leader_election.cc:343] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:17.207218 14356 leader_election.cc:343] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.183292 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fca14a7d574842ee94a93afb85ca1afd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.207466 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b5fd91a0ea64b3b85b7571e5daa08b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.208288 14811 tablet_bootstrap.cc:654] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.209609 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a2894e4736b4191963a61d908bc673c. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:17.210582 14171 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3bc1f0f72a3b4c79847c0a91fe98ccd1 consensus state has no leader
I20250115 03:06:17.211536 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca117aca95f640be880b640c85e18cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.212541 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_5ddb4f54525d4439b54c2c898471fa7e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.212867 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5ddb4f54525d4439b54c2c898471fa7e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.213021 14531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b61cc4b11251486a8089ca9bf44ac91e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.214227 14820 ts_tablet_manager.cc:1939] T 4ff51d514fb84809aa53fa9fc1010681 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:17.214627 14820 tablet_replica.cc:331] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:17.214761 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 4ff51d514fb84809aa53fa9fc1010681 (table default.Table_a60b99b368f74c68873e8c89066cd3e0 [id=98f48acd89a6461bb4511141cab4d88b]) successfully deleted
I20250115 03:06:17.215557 15453 raft_consensus.cc:491] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.215636 15453 raft_consensus.cc:513] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.216468 15453 leader_election.cc:290] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:17.216835 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d895f75911b7456a88e7a5159db7d60b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:17.217149 14628 leader_election.cc:343] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.219182 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 858bbbb62fc249d5a915697c12362d0f (table default.Table_79d7aa1c665d4695a2a17501157b3807 [id=5cd92dd0835f4c9d88d917d8ff277bbb]) successfully deleted
W20250115 03:06:17.219321 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet bb68869eaf55477090cf12f40f76a89c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb68869eaf55477090cf12f40f76a89c already in progress: creating tablet
I20250115 03:06:17.219426 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet e594ea5cff81475886cdceb8400645a6 (table default.Table_bd6a79400bc244d69676d8df1069cbb2 [id=e8ef3a6c99c14ba585efea22568617a9]) successfully deleted
W20250115 03:06:17.219563 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet b2629ae9d9854ae4b6e7ddddbb6f341b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b2629ae9d9854ae4b6e7ddddbb6f341b already in progress: creating tablet
I20250115 03:06:17.219748 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet a75b9401cb4b4f8b92a4779c3704eb46 (table default.Table_a0c249d69b904390952a780b10e1369b [id=22d05297fb9545aeab39b5b62248d0d4]) successfully deleted
W20250115 03:06:17.219871 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet bb68869eaf55477090cf12f40f76a89c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb68869eaf55477090cf12f40f76a89c already in progress: creating tablet
I20250115 03:06:17.217497 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d895f75911b7456a88e7a5159db7d60b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:17.221290 14626 leader_election.cc:343] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.221362 14626 leader_election.cc:304] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.221716 15153 raft_consensus.cc:2743] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.222160 14809 tablet_bootstrap.cc:492] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.222160 14812 tablet_bootstrap.cc:492] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.222241 14809 ts_tablet_manager.cc:1397] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.120s user 0.002s sys 0.000s
I20250115 03:06:17.222241 14812 ts_tablet_manager.cc:1397] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.093s user 0.002s sys 0.000s
I20250115 03:06:17.222661 14809 raft_consensus.cc:357] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.222671 14812 raft_consensus.cc:357] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.222754 14809 raft_consensus.cc:383] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.222759 14812 raft_consensus.cc:383] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.222786 14809 raft_consensus.cc:738] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.222788 14812 raft_consensus.cc:738] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.222893 14809 consensus_queue.cc:260] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.222893 14812 consensus_queue.cc:260] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.223153 14809 ts_tablet_manager.cc:1428] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:17.223165 14812 ts_tablet_manager.cc:1428] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.223306 14809 tablet_bootstrap.cc:492] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.223356 14812 tablet_bootstrap.cc:492] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:17.224069 14557 raft_consensus.cc:3054] T eeff2cd0825c4419aacfc73ebe267a8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.224143 14809 tablet_bootstrap.cc:654] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.224269 14812 tablet_bootstrap.cc:654] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.225781 15426 tablet.cc:1721] T eeff2cd0825c4419aacfc73ebe267a8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.227165 14555 raft_consensus.cc:3054] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.230245 15453 consensus_queue.cc:1035] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.230713 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet ca117aca95f640be880b640c85e18cd4
I20250115 03:06:17.231107 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4e74d34a621f4b1898241f62a20ffc10" } modify_external_catalogs: true
I20250115 03:06:17.231137 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet ca117aca95f640be880b640c85e18cd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.231209 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4e74d34a621f4b1898241f62a20ffc10" } modify_external_catalogs: true
I20250115 03:06:17.231418 14853 raft_consensus.cc:491] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.231490 14853 raft_consensus.cc:513] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.231707 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet ca117aca95f640be880b640c85e18cd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.231806 14853 leader_election.cc:290] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.232196 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df44c5f5a9ae4d1aa7647dcbe2303a39" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:17.232255 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet ca117aca95f640be880b640c85e18cd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.232476 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df44c5f5a9ae4d1aa7647dcbe2303a39" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:17.232944 14493 leader_election.cc:304] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.233075 14853 raft_consensus.cc:2743] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250115 03:06:17.232093 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet ca117aca95f640be880b640c85e18cd4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ca117aca95f640be880b640c85e18cd4 already in progress: creating tablet
W20250115 03:06:17.233359 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet ca117aca95f640be880b640c85e18cd4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ca117aca95f640be880b640c85e18cd4 already in progress: creating tablet
I20250115 03:06:17.236912 14820 raft_consensus.cc:2238] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.237076 14820 raft_consensus.cc:2267] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.237973 14820 ts_tablet_manager.cc:1905] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.239511 14603 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:17.240631 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 2c69daff62144976ae9082c89cc58eca
I20250115 03:06:17.241261 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c69daff62144976ae9082c89cc58eca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.241415 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c69daff62144976ae9082c89cc58eca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.242544 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c69daff62144976ae9082c89cc58eca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:17.242549 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 2c69daff62144976ae9082c89cc58eca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c69daff62144976ae9082c89cc58eca already in progress: creating tablet
W20250115 03:06:17.242951 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 2c69daff62144976ae9082c89cc58eca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c69daff62144976ae9082c89cc58eca already in progress: creating tablet
I20250115 03:06:17.243577 14809 tablet_bootstrap.cc:492] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.243659 14809 ts_tablet_manager.cc:1397] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.000s
I20250115 03:06:17.244129 14809 raft_consensus.cc:357] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.244223 14809 raft_consensus.cc:383] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.244253 14809 raft_consensus.cc:738] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.244364 14809 consensus_queue.cc:260] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.244652 14809 ts_tablet_manager.cc:1428] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.244823 14809 tablet_bootstrap.cc:492] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.245966 14809 tablet_bootstrap.cc:654] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.246229 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_bac546b42308426c9fd8a4ac15b547b3" } new_table_name: "default.Table_d94cabfa959d4cbbbd0d5552a6b1829b" modify_external_catalogs: true
I20250115 03:06:17.247610 14853 raft_consensus.cc:491] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.247687 14853 raft_consensus.cc:513] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.247993 14853 leader_election.cc:290] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:17.248535 14493 leader_election.cc:343] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: b0a5d554764249f8a7eefdc943271090
W20250115 03:06:17.247917 14158 catalog_manager.cc:4945] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): alter failed for tablet 304fab45b38e4426ae6961ae786e5659 (table default.Table_d407111658854f02aa8387a067f33f37 [id=526370a40a0d4f7aa96c8348412d8b26]),no further retry: Not found: Tablet not found: 304fab45b38e4426ae6961ae786e5659
W20250115 03:06:17.248687 14493 leader_election.cc:343] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: b0a5d554764249f8a7eefdc943271090
I20250115 03:06:17.248745 14493 leader_election.cc:304] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.248860 14853 raft_consensus.cc:2743] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.249413 14809 tablet_bootstrap.cc:492] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.249501 14809 ts_tablet_manager.cc:1397] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20250115 03:06:17.249890 14809 raft_consensus.cc:357] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.249980 14809 raft_consensus.cc:383] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.250011 14809 raft_consensus.cc:738] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.250121 14809 consensus_queue.cc:260] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.250389 14809 ts_tablet_manager.cc:1428] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:17.250552 14809 tablet_bootstrap.cc:492] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.250528 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_99e4573f330b4bf5b33dac953542aca5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.250841 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_99e4573f330b4bf5b33dac953542aca5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.251574 14468 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:17.251657 14809 tablet_bootstrap.cc:654] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.256482 14809 tablet_bootstrap.cc:492] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.256560 14809 ts_tablet_manager.cc:1397] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250115 03:06:17.257036 14809 raft_consensus.cc:357] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.257175 14809 raft_consensus.cc:383] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.257248 14809 raft_consensus.cc:738] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.257421 14809 consensus_queue.cc:260] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.257741 14809 ts_tablet_manager.cc:1428] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.257930 14809 tablet_bootstrap.cc:492] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.258592 14861 raft_consensus.cc:491] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.258662 14861 raft_consensus.cc:513] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.258960 14861 leader_election.cc:290] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.259402 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet a4c3ff892f9b4dc484ffc80d7c5934c4
I20250115 03:06:17.259490 14809 tablet_bootstrap.cc:654] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.259972 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet a4c3ff892f9b4dc484ffc80d7c5934c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.260228 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet a4c3ff892f9b4dc484ffc80d7c5934c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.260375 14819 tablet_replica.cc:331] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.260502 14819 raft_consensus.cc:2238] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.260589 14819 pending_rounds.cc:62] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75: Trying to abort 1 pending ops.
I20250115 03:06:17.260613 14819 pending_rounds.cc:69] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7114384905246056448 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "a4c3ff892f9b4dc484ffc80d7c5934c4" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_76ef9277dc8142f1a86962bfc430f2b2" dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" new_extra_config { } }
I20250115 03:06:17.260797 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 90277f7d279e4c7a83c43f40766cd11b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.260914 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d10731df02f4e9682b8de6962256635 with delete_type TABLET_DATA_DELETED (replaced by tablet 5780da67ce984297ba33d3c6901c9e3a) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:17.260785 14358 leader_election.cc:343] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: d0c3536533e94174bb1e3888a3458395
I20250115 03:06:17.261041 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 6110b6c4911d4932b4cac4d86598a5c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.261684 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0c3536533e94174bb1e3888a3458395" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:17.260784 14819 raft_consensus.cc:2267] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.265094 14809 tablet_bootstrap.cc:492] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.265192 14809 ts_tablet_manager.cc:1397] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20250115 03:06:17.265599 14809 raft_consensus.cc:357] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.265744 14809 raft_consensus.cc:383] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.265808 14809 raft_consensus.cc:738] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
W20250115 03:06:17.265947 14356 leader_election.cc:343] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.266010 14356 leader_election.cc:304] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.266209 15274 tablet.cc:1721] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.266494 14861 raft_consensus.cc:2743] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.266654 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet a4c3ff892f9b4dc484ffc80d7c5934c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.266830 14823 tablet_replica.cc:331] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:17.266955 14823 raft_consensus.cc:2238] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.267038 14823 raft_consensus.cc:2267] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.265939 14809 consensus_queue.cc:260] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.267493 14809 ts_tablet_manager.cc:1428] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:17.267549 14162 master_service.cc:946] ReplaceTablet: received request to replace tablet c7e9a1d1ada941b68688e0ed6742aa99 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:17.267640 14809 tablet_bootstrap.cc:492] T e63e854c49b34c1ab286f1f97fe443ba P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.268740 14809 tablet_bootstrap.cc:654] T e63e854c49b34c1ab286f1f97fe443ba P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:17.269228 15919 consensus_queue.cc:1159] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e8df7247ee9c43e0aa18216da2661f75" status { last_received { term: 1 index: 2 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 2 } }
I20250115 03:06:17.269647 15941 tablet.cc:1721] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.269701 14819 ts_tablet_manager.cc:1905] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.271802 14819 ts_tablet_manager.cc:1918] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:17.271867 14819 log.cc:1198] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/a4c3ff892f9b4dc484ffc80d7c5934c4
I20250115 03:06:17.272076 14819 ts_tablet_manager.cc:1939] T a4c3ff892f9b4dc484ffc80d7c5934c4 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
W20250115 03:06:17.273370 14493 consensus_peers.cc:487] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 -> Peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Couldn't send request to peer e8df7247ee9c43e0aa18216da2661f75. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:17.274140 14819 tablet_replica.cc:331] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.274252 14819 raft_consensus.cc:2238] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.274331 14819 raft_consensus.cc:2267] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.274340 14823 ts_tablet_manager.cc:1905] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.274384 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet a4c3ff892f9b4dc484ffc80d7c5934c4 (table default.Table_76ef9277dc8142f1a86962bfc430f2b2 [id=f70558594b8348579a0ad5d4c8fd8885]) successfully deleted
I20250115 03:06:17.275107 14819 ts_tablet_manager.cc:1905] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.276413 14823 ts_tablet_manager.cc:1918] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:17.276490 14823 log.cc:1198] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/a4c3ff892f9b4dc484ffc80d7c5934c4
I20250115 03:06:17.276724 14823 ts_tablet_manager.cc:1939] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:17.276947 14819 ts_tablet_manager.cc:1918] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.277012 14819 log.cc:1198] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/90277f7d279e4c7a83c43f40766cd11b
I20250115 03:06:17.277210 14819 ts_tablet_manager.cc:1939] T 90277f7d279e4c7a83c43f40766cd11b P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.276386 15153 consensus_queue.cc:1035] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.278793 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 90277f7d279e4c7a83c43f40766cd11b (table default.Table_7a132f92dbad4a868f47b81f0107d42e [id=a888fcc40089456fba95b6e634ae85f7]) successfully deleted
I20250115 03:06:17.278978 14819 tablet_replica.cc:331] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.279088 14819 raft_consensus.cc:2238] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.279166 14819 raft_consensus.cc:2267] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.279543 14168 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 745344db22da4102bad39c2f121548a1 (table default.Table_365313892043408c92be8460c31d958e [id=f22f08e9af4a4496b100e7d4b5982121]). Expected version 1 got 0
I20250115 03:06:17.279561 15153 consensus_queue.cc:1035] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.279659 14168 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 6864f88bf6454ee980f624ec2f2d9fb6 (table default.Table_2e68c93eaa1647df9b269879bd4520ab [id=18e0f06feeda4bca860e6f1bc6eab15b]). Expected version 1 got 0
I20250115 03:06:17.279769 14168 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet df44c5f5a9ae4d1aa7647dcbe2303a39 (table default.Table_098c9884c0494d4592f142b4a197f38c [id=8bf1da8d633940258affd0e11c334786]). Expected version 1 got 0
I20250115 03:06:17.280483 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_aea3e4039c704eaba082b3b109362b93" } new_table_name: "default.Table_f802f82d280b4f29a5fd691e8a6f73f6" modify_external_catalogs: true
I20250115 03:06:17.280579 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet a4c3ff892f9b4dc484ffc80d7c5934c4 (table default.Table_76ef9277dc8142f1a86962bfc430f2b2 [id=f70558594b8348579a0ad5d4c8fd8885]) successfully deleted
I20250115 03:06:17.281519 14809 tablet_bootstrap.cc:492] T e63e854c49b34c1ab286f1f97fe443ba P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.281546 14861 raft_consensus.cc:491] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.281599 14809 ts_tablet_manager.cc:1397] T e63e854c49b34c1ab286f1f97fe443ba P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20250115 03:06:17.281985 14809 raft_consensus.cc:357] T e63e854c49b34c1ab286f1f97fe443ba P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.282074 14809 raft_consensus.cc:383] T e63e854c49b34c1ab286f1f97fe443ba P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.282105 14809 raft_consensus.cc:738] T e63e854c49b34c1ab286f1f97fe443ba P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.282209 14809 consensus_queue.cc:260] T e63e854c49b34c1ab286f1f97fe443ba P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.281603 14861 raft_consensus.cc:513] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.283555 14861 leader_election.cc:290] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.283805 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5986558350244e7aa4ddcd5cccb7d676" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:17.284085 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5986558350244e7aa4ddcd5cccb7d676" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:17.284123 14356 leader_election.cc:343] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.284202 14689 raft_consensus.cc:2463] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:17.284484 14170 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet df44c5f5a9ae4d1aa7647dcbe2303a39 (table default.Table_098c9884c0494d4592f142b4a197f38c [id=8bf1da8d633940258affd0e11c334786]). Expected version 1 got 0
I20250115 03:06:17.284546 14358 leader_election.cc:304] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.284579 14170 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 6864f88bf6454ee980f624ec2f2d9fb6 (table default.Table_2e68c93eaa1647df9b269879bd4520ab [id=18e0f06feeda4bca860e6f1bc6eab15b]). Expected version 1 got 0
I20250115 03:06:17.284832 14861 raft_consensus.cc:2798] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:17.284889 14861 raft_consensus.cc:491] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:17.284950 14861 raft_consensus.cc:3054] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.285180 14811 tablet_bootstrap.cc:492] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:17.285252 14811 ts_tablet_manager.cc:1397] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.101s user 0.002s sys 0.000s
I20250115 03:06:17.285622 14811 raft_consensus.cc:357] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.285728 14811 raft_consensus.cc:383] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.285760 14811 raft_consensus.cc:738] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.285773 14861 raft_consensus.cc:513] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.285868 14811 consensus_queue.cc:260] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.286098 14861 leader_election.cc:290] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.286125 14811 ts_tablet_manager.cc:1428] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.286187 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b2f2461378b548f785fdc3595569c0db" } new_table_name: "default.Table_7c8f638a9380432c844dc355d43df492" modify_external_catalogs: true
I20250115 03:06:17.286304 14811 tablet_bootstrap.cc:492] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:17.286366 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5986558350244e7aa4ddcd5cccb7d676" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:17.286763 14356 leader_election.cc:343] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.287082 14811 tablet_bootstrap.cc:654] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.287082 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5986558350244e7aa4ddcd5cccb7d676" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:17.287178 14689 raft_consensus.cc:3054] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.287818 14885 consensus_queue.cc:1035] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.288012 14689 raft_consensus.cc:2463] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:17.288353 14358 leader_election.cc:304] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.289390 15390 raft_consensus.cc:2798] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:17.289489 15390 raft_consensus.cc:695] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:17.289598 15390 consensus_queue.cc:237] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.290354 14820 ts_tablet_manager.cc:1918] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.290423 14820 log.cc:1198] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/506198162e0b4e24924c408542707c4c
I20250115 03:06:17.290441 14809 ts_tablet_manager.cc:1428] T e63e854c49b34c1ab286f1f97fe443ba P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:17.290597 14820 ts_tablet_manager.cc:1939] T 506198162e0b4e24924c408542707c4c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:17.290599 14809 tablet_bootstrap.cc:492] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.291570 14809 tablet_bootstrap.cc:654] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.294672 14811 tablet_bootstrap.cc:492] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:17.294751 14811 ts_tablet_manager.cc:1397] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20250115 03:06:17.295149 14811 raft_consensus.cc:357] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.295239 14811 raft_consensus.cc:383] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.295270 14811 raft_consensus.cc:738] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.295364 14811 consensus_queue.cc:260] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.295521 14819 ts_tablet_manager.cc:1905] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.295620 14811 ts_tablet_manager.cc:1428] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.295765 14811 tablet_bootstrap.cc:492] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:17.297031 14820 tablet_replica.cc:331] T a4484651d964411e90ed93c6caab0a13 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:17.297190 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 506198162e0b4e24924c408542707c4c (table default.Table_75c071e03ba44140bce71e920f31715d [id=13ea98de708e409ba7b66e29bb49416d]) successfully deleted
I20250115 03:06:17.297386 14820 raft_consensus.cc:2238] T a4484651d964411e90ed93c6caab0a13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.298141 14820 raft_consensus.cc:2267] T a4484651d964411e90ed93c6caab0a13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.298564 14812 tablet_bootstrap.cc:492] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.298640 14812 ts_tablet_manager.cc:1397] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.075s user 0.002s sys 0.000s
I20250115 03:06:17.299060 14812 raft_consensus.cc:357] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.299151 14812 raft_consensus.cc:383] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.299181 14812 raft_consensus.cc:738] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.299278 14812 consensus_queue.cc:260] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.299530 14812 ts_tablet_manager.cc:1428] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.299753 14812 tablet_bootstrap.cc:492] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:17.300191 14809 tablet_bootstrap.cc:492] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.300259 14809 ts_tablet_manager.cc:1397] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20250115 03:06:17.300650 14812 tablet_bootstrap.cc:654] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.300655 14809 raft_consensus.cc:357] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.300746 14809 raft_consensus.cc:383] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.300776 14809 raft_consensus.cc:738] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.300871 14809 consensus_queue.cc:260] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.301148 14809 ts_tablet_manager.cc:1428] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.301307 14809 tablet_bootstrap.cc:492] T edf17b41198b4f8fbe5db22c58aa6bb2 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.302143 14809 tablet_bootstrap.cc:654] T edf17b41198b4f8fbe5db22c58aa6bb2 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.302466 14820 ts_tablet_manager.cc:1905] T a4484651d964411e90ed93c6caab0a13 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.303826 14861 raft_consensus.cc:491] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.303901 14861 raft_consensus.cc:513] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.304196 14861 leader_election.cc:290] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.304514 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "341180618b4748eabc568888e2e6a7df" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:17.305130 14358 leader_election.cc:343] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 341180618b4748eabc568888e2e6a7df
W20250115 03:06:17.305410 14356 leader_election.cc:343] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.305485 14356 leader_election.cc:304] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.305716 14861 raft_consensus.cc:2743] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.305968 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 7d894471f7b94aec8a19df043906d7e7
I20250115 03:06:17.306210 14169 catalog_manager.cc:6596] ReplaceTablet: tablet 7d894471f7b94aec8a19df043906d7e7 deleted and replaced by tablet 194d943d173340e7a8fba69967d9b090
I20250115 03:06:17.307250 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d894471f7b94aec8a19df043906d7e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 194d943d173340e7a8fba69967d9b090) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.307438 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d894471f7b94aec8a19df043906d7e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 194d943d173340e7a8fba69967d9b090) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.307849 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet 33380937a3b7428995b06e477a896c02 with delete_type TABLET_DATA_DELETED (replaced by tablet c251960c7c9a44429102a294f35f925c) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.308794 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_cb51431eaddb48858a61d248b688bbe7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:17.309373 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d894471f7b94aec8a19df043906d7e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 194d943d173340e7a8fba69967d9b090) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.309288 14819 ts_tablet_manager.cc:1918] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.309517 14819 log.cc:1198] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/6d10731df02f4e9682b8de6962256635
I20250115 03:06:17.309922 14809 tablet_bootstrap.cc:492] T edf17b41198b4f8fbe5db22c58aa6bb2 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.309988 14809 ts_tablet_manager.cc:1397] T edf17b41198b4f8fbe5db22c58aa6bb2 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
W20250115 03:06:17.310011 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 7d894471f7b94aec8a19df043906d7e7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d894471f7b94aec8a19df043906d7e7 already in progress: creating tablet
I20250115 03:06:17.310403 14809 raft_consensus.cc:357] T edf17b41198b4f8fbe5db22c58aa6bb2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.310504 14809 raft_consensus.cc:383] T edf17b41198b4f8fbe5db22c58aa6bb2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.310536 14809 raft_consensus.cc:738] T edf17b41198b4f8fbe5db22c58aa6bb2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.310643 14809 consensus_queue.cc:260] T edf17b41198b4f8fbe5db22c58aa6bb2 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.310895 14809 ts_tablet_manager.cc:1428] T edf17b41198b4f8fbe5db22c58aa6bb2 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.311038 14809 tablet_bootstrap.cc:492] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.311021 14165 master_service.cc:946] ReplaceTablet: received request to replace tablet 6ed47b26cdf744acb28fb657a1b41087 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:17.311385 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b922b874a412477f8cf1a410e26beb1d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.311698 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b922b874a412477f8cf1a410e26beb1d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.311892 14809 tablet_bootstrap.cc:654] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.312361 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet f8b90f854a5e4221a24986f8c71ca300
W20250115 03:06:17.312698 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d2564a20857f45d9a0c7e3b74eb58a91 consensus state has no leader
I20250115 03:06:17.313561 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet f8b90f854a5e4221a24986f8c71ca300 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.313663 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet a1c93bfd3da44f9f8a8d585f2d21f1be (table default.Table_40b1953a4a364eb1ae58e3387fc65a91 [id=64d20b1792044c5e8add735aaf75a6d9]). Expected version 1 got 0
I20250115 03:06:17.314023 14819 ts_tablet_manager.cc:1939] T 6d10731df02f4e9682b8de6962256635 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.314592 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet c7e9a1d1ada941b68688e0ed6742aa99
I20250115 03:06:17.314595 14528 tablet_service.cc:1514] Processing DeleteTablet for tablet f8b90f854a5e4221a24986f8c71ca300 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.314730 14531 tablet_service.cc:1514] Processing DeleteTablet for tablet 054251151c294e06905092f2a9306849 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.314854 14162 catalog_manager.cc:6596] ReplaceTablet: tablet c7e9a1d1ada941b68688e0ed6742aa99 deleted and replaced by tablet 238e68300f304266aaed94bc3b76a01d
I20250115 03:06:17.315158 14669 tablet_service.cc:1514] Processing DeleteTablet for tablet c7e9a1d1ada941b68688e0ed6742aa99 with delete_type TABLET_DATA_DELETED (replaced by tablet 238e68300f304266aaed94bc3b76a01d) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.315871 14531 tablet_service.cc:1514] Processing DeleteTablet for tablet c7e9a1d1ada941b68688e0ed6742aa99 with delete_type TABLET_DATA_DELETED (replaced by tablet 238e68300f304266aaed94bc3b76a01d) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:17.316095 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet c7e9a1d1ada941b68688e0ed6742aa99 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c7e9a1d1ada941b68688e0ed6742aa99 already in progress: creating tablet
I20250115 03:06:17.316324 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet f8b90f854a5e4221a24986f8c71ca300 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.316490 14823 tablet_replica.cc:331] T f8b90f854a5e4221a24986f8c71ca300 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:17.316440 14819 tablet_replica.cc:331] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.316601 14823 raft_consensus.cc:2238] T f8b90f854a5e4221a24986f8c71ca300 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.316677 14823 raft_consensus.cc:2267] T f8b90f854a5e4221a24986f8c71ca300 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.316825 14819 raft_consensus.cc:2238] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.317014 14823 ts_tablet_manager.cc:1905] T f8b90f854a5e4221a24986f8c71ca300 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.317101 14819 raft_consensus.cc:2267] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.318249 14819 ts_tablet_manager.cc:1905] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.318804 14885 consensus_queue.cc:1035] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.319237 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet c7e9a1d1ada941b68688e0ed6742aa99 with delete_type TABLET_DATA_DELETED (replaced by tablet 238e68300f304266aaed94bc3b76a01d) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.320458 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 6d10731df02f4e9682b8de6962256635 (table default.Table_551ca9fefcb74ed5aa29803f19de6ab6 [id=79eb8c9694b74a31a4253b33d064e230]) successfully deleted
I20250115 03:06:17.321149 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a59a9be8fa94ec4bdb42c723ea0ca17 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.322244 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_db820168492749099144481f4895d059"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:17.322441 14811 tablet_bootstrap.cc:654] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.322623 15617 tablet.cc:1721] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:17.323133 15274 tablet.cc:1721] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20250115 03:06:17.325026 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db820168492749099144481f4895d059 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.326646 15426 tablet.cc:1721] T e4f688e4c40d4653929432b13508348a P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:17.326903 14819 ts_tablet_manager.cc:1918] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.328317 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_c7c26755389347ff821c3e4cd1448247"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.328604 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c7c26755389347ff821c3e4cd1448247 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.330713 14811 tablet_bootstrap.cc:492] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:17.330787 14811 ts_tablet_manager.cc:1397] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.035s user 0.002s sys 0.000s
I20250115 03:06:17.331231 14811 raft_consensus.cc:357] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.331357 14811 raft_consensus.cc:383] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.331390 14811 raft_consensus.cc:738] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.331506 14811 consensus_queue.cc:260] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.331786 14811 ts_tablet_manager.cc:1428] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.331949 14811 tablet_bootstrap.cc:492] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:17.331981 14823 ts_tablet_manager.cc:1918] T f8b90f854a5e4221a24986f8c71ca300 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.332046 14823 log.cc:1198] T f8b90f854a5e4221a24986f8c71ca300 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/f8b90f854a5e4221a24986f8c71ca300
I20250115 03:06:17.332235 14823 ts_tablet_manager.cc:1939] T f8b90f854a5e4221a24986f8c71ca300 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
W20250115 03:06:17.332341 14358 consensus_peers.cc:487] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 -> Peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Couldn't send request to peer 5c0dae7aff4b44c699275b95ba639770. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20250115 03:06:17.325672 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cb51431eaddb48858a61d248b688bbe7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.332818 14811 tablet_bootstrap.cc:654] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.333168 15617 tablet.cc:1721] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.333417 15274 tablet.cc:1721] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.335654 15426 tablet.cc:1721] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.335500 14819 log.cc:1198] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/6110b6c4911d4932b4cac4d86598a5c6
I20250115 03:06:17.336128 14811 tablet_bootstrap.cc:492] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:17.336205 14811 ts_tablet_manager.cc:1397] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250115 03:06:17.336612 14811 raft_consensus.cc:357] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.336699 14811 raft_consensus.cc:383] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.336730 14811 raft_consensus.cc:738] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.336834 14811 consensus_queue.cc:260] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.336982 14819 ts_tablet_manager.cc:1939] T 6110b6c4911d4932b4cac4d86598a5c6 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.337255 14811 ts_tablet_manager.cc:1428] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:17.337549 14811 tablet_bootstrap.cc:492] T 341b24d9b63f4761b2013035023fcd98 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:17.338393 14811 tablet_bootstrap.cc:654] T 341b24d9b63f4761b2013035023fcd98 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.341118 14811 tablet_bootstrap.cc:492] T 341b24d9b63f4761b2013035023fcd98 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:17.341197 14811 ts_tablet_manager.cc:1397] T 341b24d9b63f4761b2013035023fcd98 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20250115 03:06:17.341596 14811 raft_consensus.cc:357] T 341b24d9b63f4761b2013035023fcd98 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.341682 14811 raft_consensus.cc:383] T 341b24d9b63f4761b2013035023fcd98 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.341710 14811 raft_consensus.cc:738] T 341b24d9b63f4761b2013035023fcd98 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.341809 14811 consensus_queue.cc:260] T 341b24d9b63f4761b2013035023fcd98 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.342044 14811 ts_tablet_manager.cc:1428] T 341b24d9b63f4761b2013035023fcd98 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.342175 14811 tablet_bootstrap.cc:492] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:17.342182 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet f8b90f854a5e4221a24986f8c71ca300 (table default.Table_4e74d34a621f4b1898241f62a20ffc10 [id=d8084de64a264d28a3370971617a694c]) successfully deleted
I20250115 03:06:17.342370 14823 tablet_replica.cc:331] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:17.342494 14823 raft_consensus.cc:2238] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.343042 15426 tablet.cc:1721] T 6864f88bf6454ee980f624ec2f2d9fb6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.343143 14811 tablet_bootstrap.cc:654] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.343818 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3fab7f59cc324dcf811b852f62ba1f9e" } modify_external_catalogs: true
I20250115 03:06:17.343926 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3fab7f59cc324dcf811b852f62ba1f9e" } modify_external_catalogs: true
I20250115 03:06:17.345381 14823 raft_consensus.cc:2267] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.345504 14738 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:17.346578 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 6110b6c4911d4932b4cac4d86598a5c6 (table default.Table_3e911fd952934f79bee759f184cbcce0 [id=0cbd9b74bdc84d9291bd5abee773d650]) successfully deleted
I20250115 03:06:17.342738 15274 tablet.cc:1721] T 6864f88bf6454ee980f624ec2f2d9fb6 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.349242 14823 ts_tablet_manager.cc:1905] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.349639 14820 ts_tablet_manager.cc:1918] T a4484651d964411e90ed93c6caab0a13 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:17.349789 14820 log.cc:1198] T a4484651d964411e90ed93c6caab0a13 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/a4484651d964411e90ed93c6caab0a13
I20250115 03:06:17.350627 14820 ts_tablet_manager.cc:1939] T a4484651d964411e90ed93c6caab0a13 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:17.351441 15617 tablet.cc:1721] T 6864f88bf6454ee980f624ec2f2d9fb6 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.352663 14823 ts_tablet_manager.cc:1918] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.352759 14823 log.cc:1198] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/6a59a9be8fa94ec4bdb42c723ea0ca17
I20250115 03:06:17.352994 14823 ts_tablet_manager.cc:1939] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:17.354146 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 6a59a9be8fa94ec4bdb42c723ea0ca17 (table default.Table_68e894a7b2d7416a90c519daa7bf4fb9 [id=5b343e28259444528cbad21c3b048db5]) successfully deleted
I20250115 03:06:17.355262 14166 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet a1c93bfd3da44f9f8a8d585f2d21f1be (table default.Table_40b1953a4a364eb1ae58e3387fc65a91 [id=64d20b1792044c5e8add735aaf75a6d9]). Expected version 1 got 0
I20250115 03:06:17.356315 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet a1c93bfd3da44f9f8a8d585f2d21f1be (table default.Table_40b1953a4a364eb1ae58e3387fc65a91 [id=64d20b1792044c5e8add735aaf75a6d9]). Expected version 1 got 0
I20250115 03:06:17.359290 14812 tablet_bootstrap.cc:492] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.359378 14812 ts_tablet_manager.cc:1397] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.060s user 0.001s sys 0.000s
W20250115 03:06:17.359767 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 7d894471f7b94aec8a19df043906d7e7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d894471f7b94aec8a19df043906d7e7 already in progress: creating tablet
I20250115 03:06:17.359858 14812 raft_consensus.cc:357] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.359985 14812 raft_consensus.cc:383] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.360018 14812 raft_consensus.cc:738] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.360118 14812 consensus_queue.cc:260] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.360154 14819 tablet_replica.cc:331] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.360273 14819 raft_consensus.cc:2238] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:17.360375 14812 ts_tablet_manager.cc:1428] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.360424 14819 pending_rounds.cc:62] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75: Trying to abort 1 pending ops.
I20250115 03:06:17.360451 14819 pending_rounds.cc:69] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384903151779840 op_type: NO_OP noop_request { }
I20250115 03:06:17.360502 14819 raft_consensus.cc:2883] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:17.360548 14819 raft_consensus.cc:2267] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.360558 14812 tablet_bootstrap.cc:492] T 56c600fe18504e05bab2bcde3c89861d P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:17.360651 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 745344db22da4102bad39c2f121548a1 (table default.Table_365313892043408c92be8460c31d958e [id=f22f08e9af4a4496b100e7d4b5982121]). Expected version 1 got 0
I20250115 03:06:17.360769 14163 catalog_manager.cc:5526] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:17.361003 14811 tablet_bootstrap.cc:492] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:17.361073 14811 ts_tablet_manager.cc:1397] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20250115 03:06:17.361024 14163 catalog_manager.cc:5526] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:17.361251 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 6864f88bf6454ee980f624ec2f2d9fb6 (table default.Table_2e68c93eaa1647df9b269879bd4520ab [id=18e0f06feeda4bca860e6f1bc6eab15b]). Expected version 1 got 0
I20250115 03:06:17.361325 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet df44c5f5a9ae4d1aa7647dcbe2303a39 (table default.Table_098c9884c0494d4592f142b4a197f38c [id=8bf1da8d633940258affd0e11c334786]). Expected version 1 got 0
I20250115 03:06:17.361477 14811 raft_consensus.cc:357] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.361578 14811 raft_consensus.cc:383] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.361610 14811 raft_consensus.cc:738] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.361709 14811 consensus_queue.cc:260] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:17.361799 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 38c9e64e1f5e4f2394afa876553357bf consensus state has no leader
I20250115 03:06:17.361986 14811 ts_tablet_manager.cc:1428] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:17.362166 14811 tablet_bootstrap.cc:492] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:17.362185 14812 tablet_bootstrap.cc:654] T 56c600fe18504e05bab2bcde3c89861d P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.362655 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 398b93f1db0741a9b54837bca43b1bad (DEFAULT_TABLE table=default.Table_62e6b3005d334927ae377d56649bda6d [id=9f9e695a3cee41ef872d4b6f3566b6c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.362836 15857 raft_consensus.cc:491] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.362885 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 398b93f1db0741a9b54837bca43b1bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.362910 15857 raft_consensus.cc:513] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.363035 14811 tablet_bootstrap.cc:654] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.363222 15857 leader_election.cc:290] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:17.363826 14493 leader_election.cc:343] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 48aa039bf0364c02abb7f9677eae49e5
I20250115 03:06:17.364444 14819 ts_tablet_manager.cc:1905] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:17.364516 14493 leader_election.cc:343] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 48aa039bf0364c02abb7f9677eae49e5
I20250115 03:06:17.364568 14493 leader_election.cc:304] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.364750 15857 raft_consensus.cc:2743] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.365603 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 398b93f1db0741a9b54837bca43b1bad (DEFAULT_TABLE table=default.Table_62e6b3005d334927ae377d56649bda6d [id=9f9e695a3cee41ef872d4b6f3566b6c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.365904 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 68a04d57feea40eeb8e06cb6a2c61119 (DEFAULT_TABLE table=default.Table_727f8ca159af4f75b43618f937add0b1 [id=1fbd27b641a94b828169753907f952e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.365957 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 68a04d57feea40eeb8e06cb6a2c61119 (DEFAULT_TABLE table=default.Table_727f8ca159af4f75b43618f937add0b1 [id=1fbd27b641a94b828169753907f952e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.366003 14669 tablet_service.cc:1467] Processing CreateTablet for tablet f47da0e523d14388898a4be9c37bb809 (DEFAULT_TABLE table=default.Table_59afdaba8b114e13b37772ac27949580 [id=ca82b44382294c8da6a2ac0f0e8ccf81]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.365904 14398 tablet_service.cc:1467] Processing CreateTablet for tablet f47da0e523d14388898a4be9c37bb809 (DEFAULT_TABLE table=default.Table_59afdaba8b114e13b37772ac27949580 [id=ca82b44382294c8da6a2ac0f0e8ccf81]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.366180 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68a04d57feea40eeb8e06cb6a2c61119. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.366317 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f47da0e523d14388898a4be9c37bb809. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.366348 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 398b93f1db0741a9b54837bca43b1bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.366609 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f47da0e523d14388898a4be9c37bb809. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.366940 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 103f78d0d9fc42308e1e23316895be33 (DEFAULT_TABLE table=default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.367149 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68a04d57feea40eeb8e06cb6a2c61119. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.367709 15325 raft_consensus.cc:491] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.367776 15325 raft_consensus.cc:513] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.368050 15325 leader_election.cc:290] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:17.369172 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 103f78d0d9fc42308e1e23316895be33 (DEFAULT_TABLE table=default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.369376 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 103f78d0d9fc42308e1e23316895be33. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.369879 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 103f78d0d9fc42308e1e23316895be33 (DEFAULT_TABLE table=default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.370087 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 103f78d0d9fc42308e1e23316895be33. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.370513 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e6dfaa9c84d144e68a2d047436d0ecd2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.370846 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e6dfaa9c84d144e68a2d047436d0ecd2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.371192 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edf17b41198b4f8fbe5db22c58aa6bb2" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:17.371318 14420 raft_consensus.cc:2463] T edf17b41198b4f8fbe5db22c58aa6bb2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:17.371632 14531 tablet_service.cc:1467] Processing CreateTablet for tablet 398b93f1db0741a9b54837bca43b1bad (DEFAULT_TABLE table=default.Table_62e6b3005d334927ae377d56649bda6d [id=9f9e695a3cee41ef872d4b6f3566b6c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.371852 14531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 398b93f1db0741a9b54837bca43b1bad. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.372159 14628 leader_election.cc:304] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:17.372458 15325 raft_consensus.cc:2798] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:17.372512 15325 raft_consensus.cc:491] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:17.372546 15325 raft_consensus.cc:3054] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.372611 14820 tablet_replica.cc:331] T 304fab45b38e4426ae6961ae786e5659 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:17.372728 14820 raft_consensus.cc:2238] T 304fab45b38e4426ae6961ae786e5659 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.372813 14820 raft_consensus.cc:2267] T 304fab45b38e4426ae6961ae786e5659 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.373162 14820 ts_tablet_manager.cc:1905] T 304fab45b38e4426ae6961ae786e5659 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.373414 15325 raft_consensus.cc:513] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.373711 15325 leader_election.cc:290] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:17.373761 14528 tablet_service.cc:1467] Processing CreateTablet for tablet 68a04d57feea40eeb8e06cb6a2c61119 (DEFAULT_TABLE table=default.Table_727f8ca159af4f75b43618f937add0b1 [id=1fbd27b641a94b828169753907f952e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.373977 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68a04d57feea40eeb8e06cb6a2c61119. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.374051 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edf17b41198b4f8fbe5db22c58aa6bb2" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:17.374626 14626 leader_election.cc:343] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.375099 14820 ts_tablet_manager.cc:1918] T 304fab45b38e4426ae6961ae786e5659 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.375154 14820 log.cc:1198] T 304fab45b38e4426ae6961ae786e5659 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/304fab45b38e4426ae6961ae786e5659
I20250115 03:06:17.375325 14820 ts_tablet_manager.cc:1939] T 304fab45b38e4426ae6961ae786e5659 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:17.375530 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet a4484651d964411e90ed93c6caab0a13 (table default.Table_2db28e1f2ea5450092ed00f593c4939f [id=6fe98239ee3346df8b25712aaec67b82]) successfully deleted
W20250115 03:06:17.375648 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 80ede4e9696d491aa385b375fcb5cf3b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 80ede4e9696d491aa385b375fcb5cf3b already in progress: creating tablet
I20250115 03:06:17.375883 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edf17b41198b4f8fbe5db22c58aa6bb2" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:17.375895 14529 tablet_service.cc:1467] Processing CreateTablet for tablet f47da0e523d14388898a4be9c37bb809 (DEFAULT_TABLE table=default.Table_59afdaba8b114e13b37772ac27949580 [id=ca82b44382294c8da6a2ac0f0e8ccf81]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.376811 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f47da0e523d14388898a4be9c37bb809. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:17.377292 14626 leader_election.cc:343] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.377612 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 304fab45b38e4426ae6961ae786e5659 (table default.Table_d407111658854f02aa8387a067f33f37 [id=526370a40a0d4f7aa96c8348412d8b26]) successfully deleted
W20250115 03:06:17.377969 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 4122c97f5ba445ee8c79b7d056438229 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4122c97f5ba445ee8c79b7d056438229 already in progress: creating tablet
I20250115 03:06:17.378134 14820 tablet_replica.cc:331] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:17.378237 14820 raft_consensus.cc:2238] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.378314 14820 raft_consensus.cc:2267] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.378670 15401 raft_consensus.cc:491] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.378746 15401 raft_consensus.cc:513] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.378939 14820 ts_tablet_manager.cc:1905] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.379055 15401 leader_election.cc:290] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.379594 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bffef32339a4aa6b2ef7fb5661d035c" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:17.379695 14689 raft_consensus.cc:2463] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:17.379954 14358 leader_election.cc:304] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:17.380442 15401 raft_consensus.cc:2798] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:17.380492 15401 raft_consensus.cc:491] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:17.380529 15401 raft_consensus.cc:3054] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.380714 14820 ts_tablet_manager.cc:1918] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.380766 14820 log.cc:1198] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/acb43d614111477abb10def48a1dfb3b
I20250115 03:06:17.381400 15401 raft_consensus.cc:513] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.381510 14812 tablet_bootstrap.cc:492] T 56c600fe18504e05bab2bcde3c89861d P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.381579 14812 ts_tablet_manager.cc:1397] T 56c600fe18504e05bab2bcde3c89861d P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.002s
I20250115 03:06:17.381664 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edf17b41198b4f8fbe5db22c58aa6bb2" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:17.381713 15401 leader_election.cc:290] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.381769 14420 raft_consensus.cc:3054] T edf17b41198b4f8fbe5db22c58aa6bb2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.382031 14812 raft_consensus.cc:357] T 56c600fe18504e05bab2bcde3c89861d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.382120 14812 raft_consensus.cc:383] T 56c600fe18504e05bab2bcde3c89861d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.382149 14812 raft_consensus.cc:738] T 56c600fe18504e05bab2bcde3c89861d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.382135 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bffef32339a4aa6b2ef7fb5661d035c" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:17.382251 14812 consensus_queue.cc:260] T 56c600fe18504e05bab2bcde3c89861d P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.382501 14812 ts_tablet_manager.cc:1428] T 56c600fe18504e05bab2bcde3c89861d P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:17.382581 14420 raft_consensus.cc:2463] T edf17b41198b4f8fbe5db22c58aa6bb2 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:17.382658 14812 tablet_bootstrap.cc:492] T 93fa227c0de74a3fa86778641fdcda3d P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:17.382913 14356 leader_election.cc:343] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.383524 14628 leader_election.cc:304] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.383741 15453 raft_consensus.cc:2798] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:17.383823 15453 raft_consensus.cc:695] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:17.383929 15453 consensus_queue.cc:237] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.384016 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bffef32339a4aa6b2ef7fb5661d035c" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:17.384109 14689 raft_consensus.cc:3054] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.384943 14689 raft_consensus.cc:2463] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:17.385051 14820 ts_tablet_manager.cc:1939] T acb43d614111477abb10def48a1dfb3b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:17.385222 14358 leader_election.cc:304] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.385358 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet acb43d614111477abb10def48a1dfb3b (table default.Table_85ea53c1d8b245cc94e6e81e157242aa [id=4d8e724e03dc4e2ab9a6e0281159aca8]) successfully deleted
I20250115 03:06:17.385432 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 103f78d0d9fc42308e1e23316895be33. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.388017 14820 tablet_replica.cc:331] T e594ea5cff81475886cdceb8400645a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:17.388204 14820 raft_consensus.cc:2238] T e594ea5cff81475886cdceb8400645a6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.388332 14820 raft_consensus.cc:2267] T e594ea5cff81475886cdceb8400645a6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.389998 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bffef32339a4aa6b2ef7fb5661d035c" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:17.390506 14356 leader_election.cc:343] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.390576 14885 raft_consensus.cc:2798] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:17.390662 14885 raft_consensus.cc:695] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:17.390791 14885 consensus_queue.cc:237] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.390869 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_890029e0a8f049bcaa97eb1e2b702133" } new_table_name: "default.Table_0a27afad2a4244be9772202baff78483" modify_external_catalogs: true
I20250115 03:06:17.391116 14861 raft_consensus.cc:491] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.391181 14861 raft_consensus.cc:513] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.391500 14861 leader_election.cc:290] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.391620 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 6ed47b26cdf744acb28fb657a1b41087
I20250115 03:06:17.391881 14165 catalog_manager.cc:6596] ReplaceTablet: tablet 6ed47b26cdf744acb28fb657a1b41087 deleted and replaced by tablet fe8d94c8fecc4077ba052390810c7430
I20250115 03:06:17.392513 14820 ts_tablet_manager.cc:1905] T e594ea5cff81475886cdceb8400645a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.393565 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcdb5282e41c43dbb235ae78465d19f9" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:17.393620 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ed47b26cdf744acb28fb657a1b41087 with delete_type TABLET_DATA_DELETED (replaced by tablet fe8d94c8fecc4077ba052390810c7430) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.393782 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ed47b26cdf744acb28fb657a1b41087 with delete_type TABLET_DATA_DELETED (replaced by tablet fe8d94c8fecc4077ba052390810c7430) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.393940 14529 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ed47b26cdf744acb28fb657a1b41087 with delete_type TABLET_DATA_DELETED (replaced by tablet fe8d94c8fecc4077ba052390810c7430) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:17.393937 14356 leader_election.cc:343] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:17.394313 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 6ed47b26cdf744acb28fb657a1b41087 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6ed47b26cdf744acb28fb657a1b41087 already in progress: creating tablet
I20250115 03:06:17.394330 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcdb5282e41c43dbb235ae78465d19f9" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:17.394421 14689 raft_consensus.cc:2463] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:17.394673 14358 leader_election.cc:304] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.394963 15390 raft_consensus.cc:2798] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:17.395009 15390 raft_consensus.cc:491] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:17.395044 15390 raft_consensus.cc:3054] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.395867 15390 raft_consensus.cc:513] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.396183 15390 leader_election.cc:290] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.396275 14819 ts_tablet_manager.cc:1918] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:17.396389 14819 log.cc:1198] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/f8b90f854a5e4221a24986f8c71ca300
I20250115 03:06:17.396494 15857 raft_consensus.cc:491] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.396561 15857 raft_consensus.cc:513] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.396785 14819 ts_tablet_manager.cc:1939] T f8b90f854a5e4221a24986f8c71ca300 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.396849 15857 leader_election.cc:290] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.397692 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet f8b90f854a5e4221a24986f8c71ca300 (table default.Table_4e74d34a621f4b1898241f62a20ffc10 [id=d8084de64a264d28a3370971617a694c]) successfully deleted
W20250115 03:06:17.397840 14493 leader_election.cc:343] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 3a0bb9b5093b454d921bf1150d631565
I20250115 03:06:17.397872 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcdb5282e41c43dbb235ae78465d19f9" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:17.398098 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcdb5282e41c43dbb235ae78465d19f9" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:17.398198 14687 raft_consensus.cc:3054] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.399025 14687 raft_consensus.cc:2463] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
W20250115 03:06:17.399416 14356 leader_election.cc:343] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.399469 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet cc67fb818b8146629d92f9ac1944dba6
I20250115 03:06:17.400029 14358 leader_election.cc:304] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.400302 15401 raft_consensus.cc:2798] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:17.400378 15401 raft_consensus.cc:695] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:17.400502 15401 consensus_queue.cc:237] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:17.401198 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet c7e9a1d1ada941b68688e0ed6742aa99 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c7e9a1d1ada941b68688e0ed6742aa99 already in progress: creating tablet
I20250115 03:06:17.403743 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet cc67fb818b8146629d92f9ac1944dba6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.404194 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet cc67fb818b8146629d92f9ac1944dba6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.404320 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet d0c3536533e94174bb1e3888a3458395 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.404423 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet a4c3ff892f9b4dc484ffc80d7c5934c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:17.404680 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet cc67fb818b8146629d92f9ac1944dba6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cc67fb818b8146629d92f9ac1944dba6 already in progress: creating tablet
I20250115 03:06:17.406688 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 341180618b4748eabc568888e2e6a7df with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:17.407066 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 6ed47b26cdf744acb28fb657a1b41087 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6ed47b26cdf744acb28fb657a1b41087 already in progress: creating tablet
W20250115 03:06:17.407406 14493 leader_election.cc:343] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 3a0bb9b5093b454d921bf1150d631565
W20250115 03:06:17.407465 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet cc67fb818b8146629d92f9ac1944dba6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cc67fb818b8146629d92f9ac1944dba6 already in progress: creating tablet
I20250115 03:06:17.407488 14493 leader_election.cc:304] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.407570 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ab72071c412848939b54fcc72d5974fa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:17.407671 15857 raft_consensus.cc:2743] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.407706 14819 tablet_replica.cc:331] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.407819 15390 raft_consensus.cc:491] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.407828 14819 raft_consensus.cc:2238] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
W20250115 03:06:17.407899 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab72071c412848939b54fcc72d5974fa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.407893 15390 raft_consensus.cc:513] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.407934 14819 raft_consensus.cc:2267] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.408200 15390 leader_election.cc:290] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.408452 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c9e64e1f5e4f2394afa876553357bf" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:17.409106 14170 master_service.cc:946] ReplaceTablet: received request to replace tablet 5986558350244e7aa4ddcd5cccb7d676 from {username='slave'} at 127.0.0.1:60432
W20250115 03:06:17.409126 14356 leader_election.cc:343] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.409202 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c9e64e1f5e4f2394afa876553357bf" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:17.409122 14819 ts_tablet_manager.cc:1905] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.409307 14689 raft_consensus.cc:2463] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:17.409529 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet 56c600fe18504e05bab2bcde3c89861d from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:17.409586 14358 leader_election.cc:304] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.409859 15390 raft_consensus.cc:2798] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:17.409911 15390 raft_consensus.cc:491] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:17.409946 15390 raft_consensus.cc:3054] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.410132 14529 tablet_service.cc:1514] Processing DeleteTablet for tablet cc67fb818b8146629d92f9ac1944dba6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.410820 15390 raft_consensus.cc:513] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.411127 15390 leader_election.cc:290] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.411360 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c9e64e1f5e4f2394afa876553357bf" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:17.411661 14356 leader_election.cc:343] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.411756 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_62e6b3005d334927ae377d56649bda6d" } new_table_name: "default.Table_84b19a144ef44b0380ee4e6c66a26370" modify_external_catalogs: true
I20250115 03:06:17.412420 14819 ts_tablet_manager.cc:1918] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.412497 14819 log.cc:1198] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/d0c3536533e94174bb1e3888a3458395
I20250115 03:06:17.412688 14819 ts_tablet_manager.cc:1939] T d0c3536533e94174bb1e3888a3458395 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.413071 14819 tablet_replica.cc:331] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.413177 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet d0c3536533e94174bb1e3888a3458395 (table default.Table_1c52ca26880a48289f5916fabdb05ac4 [id=44c03f21942545319762494029cebc02]) successfully deleted
I20250115 03:06:17.413197 14819 raft_consensus.cc:2238] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.413281 14819 raft_consensus.cc:2267] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:17.413319 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet a4c3ff892f9b4dc484ffc80d7c5934c4 because the tablet was not found. No further retry: Not found: Tablet not found: a4c3ff892f9b4dc484ffc80d7c5934c4
I20250115 03:06:17.413712 14819 ts_tablet_manager.cc:1905] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.414139 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38c9e64e1f5e4f2394afa876553357bf" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:17.414255 14689 raft_consensus.cc:3054] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.413555 14167 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_f802f82d280b4f29a5fd691e8a6f73f6 [id=5beb8c6fadc742c4bd0b5171ed51d0e2] alter complete (version 2)
I20250115 03:06:17.415287 14689 raft_consensus.cc:2463] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:17.415653 14358 leader_election.cc:304] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.415901 15401 raft_consensus.cc:2798] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:17.415977 15401 raft_consensus.cc:695] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:17.416080 15401 consensus_queue.cc:237] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.416893 14819 ts_tablet_manager.cc:1918] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.416981 14819 log.cc:1198] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/341180618b4748eabc568888e2e6a7df
I20250115 03:06:17.417162 14819 ts_tablet_manager.cc:1939] T 341180618b4748eabc568888e2e6a7df P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.417347 15274 tablet.cc:1721] T b1c0f6a5611647eb884c77f7a6d1cd4e P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.417541 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 341180618b4748eabc568888e2e6a7df (table default.Table_65f0fd5511f64bfeb6539f9019a4873c [id=7191c3890ae642558fe99561455d01e7]) successfully deleted
I20250115 03:06:17.417687 14820 ts_tablet_manager.cc:1918] T e594ea5cff81475886cdceb8400645a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:17.417753 14820 log.cc:1198] T e594ea5cff81475886cdceb8400645a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/e594ea5cff81475886cdceb8400645a6
W20250115 03:06:17.418653 14157 catalog_manager.cc:4670] Async tablet task AlterTable RPC for tablet 312dcdda8cf042648db9fdf6228062d9 (table default.Table_341a3cd1c8c94ff4b05fc21960a4e9b4 [id=c096ce170874405ea740b9ea829f886e], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 312dcdda8cf042648db9fdf6228062d9 consensus state has no leader
I20250115 03:06:17.420063 15426 tablet.cc:1721] T b1c0f6a5611647eb884c77f7a6d1cd4e P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.420171 14820 ts_tablet_manager.cc:1939] T e594ea5cff81475886cdceb8400645a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
W20250115 03:06:17.420333 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 398b93f1db0741a9b54837bca43b1bad consensus state has no leader
I20250115 03:06:17.420581 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet e594ea5cff81475886cdceb8400645a6 (table default.Table_bd6a79400bc244d69676d8df1069cbb2 [id=e8ef3a6c99c14ba585efea22568617a9]) successfully deleted
I20250115 03:06:17.420917 14820 tablet_replica.cc:331] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:17.421063 14820 raft_consensus.cc:2238] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.422703 14820 raft_consensus.cc:2267] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.424237 15617 tablet.cc:1721] T b1c0f6a5611647eb884c77f7a6d1cd4e P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.424341 14820 ts_tablet_manager.cc:1905] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.426250 14820 ts_tablet_manager.cc:1918] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.428619 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 75c581f11be84e6293a3d14966fd584f (DEFAULT_TABLE table=default.Table_cb51431eaddb48858a61d248b688bbe7 [id=5624f7f1c61140a798718c8c85da14ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.428619 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 75c581f11be84e6293a3d14966fd584f (DEFAULT_TABLE table=default.Table_cb51431eaddb48858a61d248b688bbe7 [id=5624f7f1c61140a798718c8c85da14ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.428840 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75c581f11be84e6293a3d14966fd584f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.428840 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75c581f11be84e6293a3d14966fd584f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.430488 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 0c580cdf3a854df988df32a9294c83bc (DEFAULT_TABLE table=default.Table_db820168492749099144481f4895d059 [id=e0b2dd40d51e490c8fa2a5ca2c166f91]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.430774 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c580cdf3a854df988df32a9294c83bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.430917 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 0c580cdf3a854df988df32a9294c83bc (DEFAULT_TABLE table=default.Table_db820168492749099144481f4895d059 [id=e0b2dd40d51e490c8fa2a5ca2c166f91]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.431123 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c580cdf3a854df988df32a9294c83bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.432783 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 781f6a9148cd4bd684f24c294c0b019b (DEFAULT_TABLE table=default.Table_c7c26755389347ff821c3e4cd1448247 [id=b9f32421ef8a4c549dab58ffb0f71fa7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.432799 14166 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_59afdaba8b114e13b37772ac27949580" } modify_external_catalogs: true
I20250115 03:06:17.432893 14166 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_59afdaba8b114e13b37772ac27949580" } modify_external_catalogs: true
I20250115 03:06:17.433015 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 781f6a9148cd4bd684f24c294c0b019b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.433615 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 781f6a9148cd4bd684f24c294c0b019b (DEFAULT_TABLE table=default.Table_c7c26755389347ff821c3e4cd1448247 [id=b9f32421ef8a4c549dab58ffb0f71fa7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.433825 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 781f6a9148cd4bd684f24c294c0b019b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.434216 15401 raft_consensus.cc:491] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.434284 15401 raft_consensus.cc:513] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.434835 15401 leader_election.cc:290] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.435246 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d895f75911b7456a88e7a5159db7d60b" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:17.435374 14687 raft_consensus.cc:2463] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:17.435672 14358 leader_election.cc:304] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:17.435843 15401 raft_consensus.cc:2798] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:17.435946 15401 raft_consensus.cc:491] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:17.436043 15401 raft_consensus.cc:3054] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.436803 14669 tablet_service.cc:1467] Processing CreateTablet for tablet f72c9bf04a3942ceacc37fd407bff033 (DEFAULT_TABLE table=default.Table_5ddb4f54525d4439b54c2c898471fa7e [id=85a660e149c4457fa9bc97df0f6f06e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.436902 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 5986558350244e7aa4ddcd5cccb7d676
I20250115 03:06:17.436901 15401 raft_consensus.cc:513] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.437078 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f72c9bf04a3942ceacc37fd407bff033. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.437191 14170 catalog_manager.cc:6596] ReplaceTablet: tablet 5986558350244e7aa4ddcd5cccb7d676 deleted and replaced by tablet d5e4743e016e49ed8ba26b1ac96b1470
I20250115 03:06:17.437374 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 56c600fe18504e05bab2bcde3c89861d
I20250115 03:06:17.437481 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 5986558350244e7aa4ddcd5cccb7d676 with delete_type TABLET_DATA_DELETED (replaced by tablet d5e4743e016e49ed8ba26b1ac96b1470) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.437635 14163 catalog_manager.cc:6596] ReplaceTablet: tablet 56c600fe18504e05bab2bcde3c89861d deleted and replaced by tablet cef4442bfcfd4ad4b343a2785c3a9a74
I20250115 03:06:17.438051 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d895f75911b7456a88e7a5159db7d60b" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:17.438230 14689 raft_consensus.cc:3054] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.438598 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 5986558350244e7aa4ddcd5cccb7d676 with delete_type TABLET_DATA_DELETED (replaced by tablet d5e4743e016e49ed8ba26b1ac96b1470) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.438776 14823 tablet_replica.cc:331] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:17.439007 14165 catalog_manager.cc:5526] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:17.439179 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet dcdb5282e41c43dbb235ae78465d19f9 (table default.Table_375bb4e2095c4675b734247fabacbec7 [id=b298dce7ab664327893633f821ed4d89]). Expected version 1 got 0
I20250115 03:06:17.439250 14689 raft_consensus.cc:2463] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:17.439301 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet b1c0f6a5611647eb884c77f7a6d1cd4e (table default.Table_0a27afad2a4244be9772202baff78483 [id=384b6711c99840f094550519558797fc]). Expected version 1 got 0
I20250115 03:06:17.439371 14165 catalog_manager.cc:5526] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:17.439563 14823 raft_consensus.cc:2238] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.439575 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet a1c93bfd3da44f9f8a8d585f2d21f1be (table default.Table_40b1953a4a364eb1ae58e3387fc65a91 [id=64d20b1792044c5e8add735aaf75a6d9]). Expected version 1 got 0
I20250115 03:06:17.439627 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 56c600fe18504e05bab2bcde3c89861d with delete_type TABLET_DATA_DELETED (replaced by tablet cef4442bfcfd4ad4b343a2785c3a9a74) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.439664 14823 raft_consensus.cc:2267] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.439893 14169 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_098c9884c0494d4592f142b4a197f38c [id=8bf1da8d633940258affd0e11c334786] alter complete (version 1)
I20250115 03:06:17.439970 14823 ts_tablet_manager.cc:1905] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.440311 14167 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_2e68c93eaa1647df9b269879bd4520ab [id=18e0f06feeda4bca860e6f1bc6eab15b] alter complete (version 1)
I20250115 03:06:17.442020 14823 ts_tablet_manager.cc:1918] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.442090 14823 log.cc:1198] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/5986558350244e7aa4ddcd5cccb7d676
I20250115 03:06:17.442276 14823 ts_tablet_manager.cc:1939] T 5986558350244e7aa4ddcd5cccb7d676 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:17.442252 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_eec974cf89de4f45930dbf47fa754f2a" } modify_external_catalogs: true
I20250115 03:06:17.442339 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_eec974cf89de4f45930dbf47fa754f2a" } modify_external_catalogs: true
I20250115 03:06:17.442560 14399 tablet_service.cc:1467] Processing CreateTablet for tablet f72c9bf04a3942ceacc37fd407bff033 (DEFAULT_TABLE table=default.Table_5ddb4f54525d4439b54c2c898471fa7e [id=85a660e149c4457fa9bc97df0f6f06e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.442765 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f72c9bf04a3942ceacc37fd407bff033. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.445281 14823 tablet_replica.cc:331] T 56c600fe18504e05bab2bcde3c89861d P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:17.445391 14823 raft_consensus.cc:2238] T 56c600fe18504e05bab2bcde3c89861d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.445488 14823 raft_consensus.cc:2267] T 56c600fe18504e05bab2bcde3c89861d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.445755 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 56c600fe18504e05bab2bcde3c89861d with delete_type TABLET_DATA_DELETED (replaced by tablet cef4442bfcfd4ad4b343a2785c3a9a74) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.383529 14812 tablet_bootstrap.cc:654] T 93fa227c0de74a3fa86778641fdcda3d P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:17.446293 14356 consensus_peers.cc:487] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:17.446338 15325 raft_consensus.cc:491] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.446372 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 5986558350244e7aa4ddcd5cccb7d676 (table default.Table_e72d1ee69400410c9da9082d4bd8bc9a [id=6ab343bee1254fb19a3024ebafa44464]) successfully deleted
I20250115 03:06:17.446405 15325 raft_consensus.cc:513] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.446714 15325 leader_election.cc:290] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:17.447154 14358 leader_election.cc:304] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:17.447356 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 83a4f95eb9494a5295a81a8a51f58bd3 (DEFAULT_TABLE table=default.Table_1fd65820535a4dc7a4fc884c4184bba4 [id=c8ca3cd85578452bb7b8659c9e1a4e2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.447597 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a4f95eb9494a5295a81a8a51f58bd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.447727 15390 raft_consensus.cc:2798] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:17.447751 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e63e854c49b34c1ab286f1f97fe443ba" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:17.447765 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 83a4f95eb9494a5295a81a8a51f58bd3 (DEFAULT_TABLE table=default.Table_1fd65820535a4dc7a4fc884c4184bba4 [id=c8ca3cd85578452bb7b8659c9e1a4e2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.447855 14420 raft_consensus.cc:2463] T e63e854c49b34c1ab286f1f97fe443ba P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:17.447970 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a4f95eb9494a5295a81a8a51f58bd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.448213 14823 ts_tablet_manager.cc:1905] T 56c600fe18504e05bab2bcde3c89861d P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.448215 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet f47da0e523d14388898a4be9c37bb809
I20250115 03:06:17.449110 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet f47da0e523d14388898a4be9c37bb809 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.450992 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet f47da0e523d14388898a4be9c37bb809 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.452293 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f1bb182509e64b589f4cb90482cec747"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:17.452330 14823 ts_tablet_manager.cc:1918] T 56c600fe18504e05bab2bcde3c89861d P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.452415 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 85cceb5a3f234d85937ad0bb75348991
I20250115 03:06:17.452425 14823 log.cc:1198] T 56c600fe18504e05bab2bcde3c89861d P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/56c600fe18504e05bab2bcde3c89861d
W20250115 03:06:17.452629 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f1bb182509e64b589f4cb90482cec747 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.452760 14823 ts_tablet_manager.cc:1939] T 56c600fe18504e05bab2bcde3c89861d P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:17.453159 14167 catalog_manager.cc:5526] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:17.453506 14628 leader_election.cc:304] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:17.453799 15325 raft_consensus.cc:2798] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:17.453850 15325 raft_consensus.cc:491] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:17.453882 15325 raft_consensus.cc:3054] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.454214 15390 raft_consensus.cc:695] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:17.454341 15390 consensus_queue.cc:237] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.454466 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a59a9be8fa94ec4bdb42c723ea0ca17 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.454572 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 858bbbb62fc249d5a915697c12362d0f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.454687 15325 raft_consensus.cc:513] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.455085 14396 tablet_service.cc:1467] Processing CreateTablet for tablet a8a5743475814401ac2c0a51cfd70207 (DEFAULT_TABLE table=default.Table_39b71235b2df40ec88411bb02d370d7b [id=3c0f75dc669f4fb181b0d9e32f38f761]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.455322 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8a5743475814401ac2c0a51cfd70207. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.455164 15325 leader_election.cc:290] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
W20250115 03:06:17.455677 14356 consensus_peers.cc:487] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:17.456493 14689 raft_consensus.cc:1270] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:17.456684 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 85cceb5a3f234d85937ad0bb75348991 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.457008 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e63e854c49b34c1ab286f1f97fe443ba" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:17.457059 15390 consensus_queue.cc:1035] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.457191 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 3e170ced1612442ebcfdaff6d9826ea3 (DEFAULT_TABLE table=default.Table_77d4ac27a2c44250a2dcc62121f202c7 [id=6af8f6f562ac47b484eeb4d67ac6c7df]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.457278 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 238e68300f304266aaed94bc3b76a01d (DEFAULT_TABLE table=default.Table_169e728924b749e4a9118c12d9b56eab [id=a2689bcea23843c9bb9e1f08e335e58f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.457372 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e170ced1612442ebcfdaff6d9826ea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.457471 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 238e68300f304266aaed94bc3b76a01d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.457577 15401 leader_election.cc:290] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.459344 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 3e170ced1612442ebcfdaff6d9826ea3 (DEFAULT_TABLE table=default.Table_77d4ac27a2c44250a2dcc62121f202c7 [id=6af8f6f562ac47b484eeb4d67ac6c7df]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.459537 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e170ced1612442ebcfdaff6d9826ea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.460991 14662 tablet_service.cc:1467] Processing CreateTablet for tablet a8a5743475814401ac2c0a51cfd70207 (DEFAULT_TABLE table=default.Table_39b71235b2df40ec88411bb02d370d7b [id=3c0f75dc669f4fb181b0d9e32f38f761]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.461189 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8a5743475814401ac2c0a51cfd70207. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.457108 14420 raft_consensus.cc:3054] T e63e854c49b34c1ab286f1f97fe443ba P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.461602 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_db8a2673977247e0b14b7995f7e09661" } modify_external_catalogs: true
I20250115 03:06:17.461705 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_db8a2673977247e0b14b7995f7e09661" } modify_external_catalogs: true
I20250115 03:06:17.462250 14420 raft_consensus.cc:2463] T e63e854c49b34c1ab286f1f97fe443ba P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:17.462455 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 238e68300f304266aaed94bc3b76a01d (DEFAULT_TABLE table=default.Table_169e728924b749e4a9118c12d9b56eab [id=a2689bcea23843c9bb9e1f08e335e58f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.462666 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 238e68300f304266aaed94bc3b76a01d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.462729 14628 leader_election.cc:304] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:17.462875 15325 raft_consensus.cc:2798] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:17.462955 15325 raft_consensus.cc:695] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:17.463053 15325 consensus_queue.cc:237] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.464864 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 85cceb5a3f234d85937ad0bb75348991 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.464937 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 56c600fe18504e05bab2bcde3c89861d (table default.Table_b94f9ef5ede64627b994e94e674b1cfa [id=3f4310bb3227447ba6fcd011f05d6924]) successfully deleted
I20250115 03:06:17.465587 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 194d943d173340e7a8fba69967d9b090 (DEFAULT_TABLE table=default.Table_1a2ec424c5cb40fbb3c93f327531e36f [id=50a51914c37b462aa45a71fb260c809b]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:17.465776 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet f47da0e523d14388898a4be9c37bb809 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f47da0e523d14388898a4be9c37bb809 already in progress: creating tablet
I20250115 03:06:17.465924 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 194d943d173340e7a8fba69967d9b090. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.466136 14819 tablet_replica.cc:331] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.466382 14819 raft_consensus.cc:2238] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
W20250115 03:06:17.467147 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 85cceb5a3f234d85937ad0bb75348991 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 85cceb5a3f234d85937ad0bb75348991 already in progress: creating tablet
I20250115 03:06:17.467437 14819 pending_rounds.cc:62] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75: Trying to abort 1 pending ops.
I20250115 03:06:17.467528 14819 pending_rounds.cc:69] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384905379442688 op_type: NO_OP noop_request { }
I20250115 03:06:17.467706 14819 raft_consensus.cc:2883] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:17.467865 14819 raft_consensus.cc:2267] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.468381 15401 raft_consensus.cc:491] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.468801 15401 raft_consensus.cc:513] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.469034 14664 tablet_service.cc:1467] Processing CreateTablet for tablet a3eaec2fff184aad816349fc36fa3db1 (DEFAULT_TABLE table=default.Table_f3b64ddeafb3400583229c6698c2a5e5 [id=538723971ef44b719ff8bf684fdc5676]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.469918 14819 ts_tablet_manager.cc:1905] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.470003 14169 catalog_manager.cc:5526] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:17.470180 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 38c9e64e1f5e4f2394afa876553357bf (table default.Table_d94cabfa959d4cbbbd0d5552a6b1829b [id=91731a4895d4478fad68b1bcbdc45d5d]). Expected version 1 got 0
I20250115 03:06:17.470623 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3eaec2fff184aad816349fc36fa3db1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.470652 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet f70a69acbdc049beb9d98eada09d9eb8
I20250115 03:06:17.473073 15401 leader_election.cc:290] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.473320 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet f70a69acbdc049beb9d98eada09d9eb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.473906 14167 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_0a27afad2a4244be9772202baff78483 [id=384b6711c99840f094550519558797fc] alter complete (version 1)
I20250115 03:06:17.473348 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet f70a69acbdc049beb9d98eada09d9eb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:17.473601 14358 leader_election.cc:343] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 858bbbb62fc249d5a915697c12362d0f
I20250115 03:06:17.473618 14399 tablet_service.cc:1467] Processing CreateTablet for tablet a3eaec2fff184aad816349fc36fa3db1 (DEFAULT_TABLE table=default.Table_f3b64ddeafb3400583229c6698c2a5e5 [id=538723971ef44b719ff8bf684fdc5676]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.475317 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 56c600fe18504e05bab2bcde3c89861d with delete_type TABLET_DATA_DELETED (replaced by tablet cef4442bfcfd4ad4b343a2785c3a9a74) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.475304 14669 tablet_service.cc:1467] Processing CreateTablet for tablet de14b1d525cc42808500279ae6b60e80 (DEFAULT_TABLE table=default.Table_b922b874a412477f8cf1a410e26beb1d [id=76a2384a834c40a2a8ec940808a94290]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.475199 14396 tablet_service.cc:1467] Processing CreateTablet for tablet de14b1d525cc42808500279ae6b60e80 (DEFAULT_TABLE table=default.Table_b922b874a412477f8cf1a410e26beb1d [id=76a2384a834c40a2a8ec940808a94290]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.475598 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de14b1d525cc42808500279ae6b60e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.476210 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 6132d15b17ab4d0aaa10a7ea04cda7ac (DEFAULT_TABLE table=default.Table_35de20a1cb6a488bbc86997188d7ce82 [id=01aa6b0cc55b46d0b4216c601c16b065]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.476485 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6132d15b17ab4d0aaa10a7ea04cda7ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.476851 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 13f36a563f13403eab372deb10b67451 (table default.Table_70d0b700eac34892b4ef769d739481fd [id=61f3625fed2b4a70b5331d6c6ac73f1c]). Expected version 1 got 0
I20250115 03:06:17.476992 14163 catalog_manager.cc:5526] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:17.477368 14819 ts_tablet_manager.cc:1918] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:17.477458 14819 log.cc:1198] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/5986558350244e7aa4ddcd5cccb7d676
I20250115 03:06:17.477551 14823 tablet_replica.cc:331] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:17.477666 14823 raft_consensus.cc:2238] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.477773 14823 raft_consensus.cc:2267] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.478153 14819 ts_tablet_manager.cc:1939] T 5986558350244e7aa4ddcd5cccb7d676 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.478818 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 6132d15b17ab4d0aaa10a7ea04cda7ac (DEFAULT_TABLE table=default.Table_35de20a1cb6a488bbc86997188d7ce82 [id=01aa6b0cc55b46d0b4216c601c16b065]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.479071 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 194d943d173340e7a8fba69967d9b090 (DEFAULT_TABLE table=default.Table_1a2ec424c5cb40fbb3c93f327531e36f [id=50a51914c37b462aa45a71fb260c809b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.479270 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 194d943d173340e7a8fba69967d9b090. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.479408 14823 ts_tablet_manager.cc:1905] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.479859 14809 tablet_bootstrap.cc:492] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.479928 14809 ts_tablet_manager.cc:1397] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.169s user 0.001s sys 0.000s
I20250115 03:06:17.480707 14809 raft_consensus.cc:357] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.480798 14809 raft_consensus.cc:383] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.480826 14809 raft_consensus.cc:738] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.480952 14809 consensus_queue.cc:260] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.481225 14809 ts_tablet_manager.cc:1428] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20250115 03:06:17.481173 14169 catalog_manager.cc:5526] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:17.482533 14823 ts_tablet_manager.cc:1918] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:17.482601 14823 log.cc:1198] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/f70a69acbdc049beb9d98eada09d9eb8
I20250115 03:06:17.482900 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de14b1d525cc42808500279ae6b60e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.483244 14398 tablet_service.cc:1467] Processing CreateTablet for tablet e47012a789bd469f9691e5a4162ff512 (DEFAULT_TABLE table=default.Table_99e4573f330b4bf5b33dac953542aca5 [id=31ee40f75b634219ba0ae4b38e2c757e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.483512 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e47012a789bd469f9691e5a4162ff512. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.428628 14529 tablet_service.cc:1467] Processing CreateTablet for tablet 75c581f11be84e6293a3d14966fd584f (DEFAULT_TABLE table=default.Table_cb51431eaddb48858a61d248b688bbe7 [id=5624f7f1c61140a798718c8c85da14ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.484048 14820 log.cc:1198] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/48aa039bf0364c02abb7f9677eae49e5
I20250115 03:06:17.484203 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75c581f11be84e6293a3d14966fd584f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.484256 14820 ts_tablet_manager.cc:1939] T 48aa039bf0364c02abb7f9677eae49e5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:17.484831 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 48aa039bf0364c02abb7f9677eae49e5 (table default.Table_35005559d2f34672bc3e70f6bbfb31d6 [id=d73a4eb552114e20887f9599f244e6ed]) successfully deleted
I20250115 03:06:17.485102 14662 tablet_service.cc:1467] Processing CreateTablet for tablet e47012a789bd469f9691e5a4162ff512 (DEFAULT_TABLE table=default.Table_99e4573f330b4bf5b33dac953542aca5 [id=31ee40f75b634219ba0ae4b38e2c757e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.485293 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e47012a789bd469f9691e5a4162ff512. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.486682 14529 tablet_service.cc:1467] Processing CreateTablet for tablet aadcec8aca2b4ece915707c17f60b8b5 (DEFAULT_TABLE table=default.Table_77127dee522147cf8e0f3ec2f5539301 [id=9643860dccb54c598714ceccd0f0e3df]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.486905 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aadcec8aca2b4ece915707c17f60b8b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.486469 15401 raft_consensus.cc:491] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.487015 15401 raft_consensus.cc:513] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:17.487294 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet b2629ae9d9854ae4b6e7ddddbb6f341b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b2629ae9d9854ae4b6e7ddddbb6f341b already in progress: creating tablet
I20250115 03:06:17.487475 14820 tablet_replica.cc:331] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:17.487598 14820 raft_consensus.cc:2238] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.487696 14820 raft_consensus.cc:2267] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.487807 15401 leader_election.cc:290] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:17.488062 14358 leader_election.cc:343] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 6a59a9be8fa94ec4bdb42c723ea0ca17
I20250115 03:06:17.431105 14528 tablet_service.cc:1467] Processing CreateTablet for tablet 0c580cdf3a854df988df32a9294c83bc (DEFAULT_TABLE table=default.Table_db820168492749099144481f4895d059 [id=e0b2dd40d51e490c8fa2a5ca2c166f91]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.488734 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c580cdf3a854df988df32a9294c83bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.489130 14820 ts_tablet_manager.cc:1905] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.432245 14531 tablet_service.cc:1467] Processing CreateTablet for tablet 781f6a9148cd4bd684f24c294c0b019b (DEFAULT_TABLE table=default.Table_c7c26755389347ff821c3e4cd1448247 [id=b9f32421ef8a4c549dab58ffb0f71fa7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.434811 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d895f75911b7456a88e7a5159db7d60b" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:17.490859 14531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 781f6a9148cd4bd684f24c294c0b019b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.437093 14537 tablet_service.cc:1467] Processing CreateTablet for tablet f72c9bf04a3942ceacc37fd407bff033 (DEFAULT_TABLE table=default.Table_5ddb4f54525d4439b54c2c898471fa7e [id=85a660e149c4457fa9bc97df0f6f06e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.491156 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f72c9bf04a3942ceacc37fd407bff033. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.492478 14820 ts_tablet_manager.cc:1918] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.492532 14820 log.cc:1198] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/3a0bb9b5093b454d921bf1150d631565
W20250115 03:06:17.492780 14356 leader_election.cc:343] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.437512 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d895f75911b7456a88e7a5159db7d60b" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:17.438230 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 5986558350244e7aa4ddcd5cccb7d676 with delete_type TABLET_DATA_DELETED (replaced by tablet d5e4743e016e49ed8ba26b1ac96b1470) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.439527 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet 56c600fe18504e05bab2bcde3c89861d with delete_type TABLET_DATA_DELETED (replaced by tablet cef4442bfcfd4ad4b343a2785c3a9a74) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.442978 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet 420b7c8d43a04bae972eed2014d5634c with delete_type TABLET_DATA_DELETED (replaced by tablet b2629ae9d9854ae4b6e7ddddbb6f341b) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.446916 14534 tablet_service.cc:1467] Processing CreateTablet for tablet 83a4f95eb9494a5295a81a8a51f58bd3 (DEFAULT_TABLE table=default.Table_1fd65820535a4dc7a4fc884c4184bba4 [id=c8ca3cd85578452bb7b8659c9e1a4e2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.447417 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e63e854c49b34c1ab286f1f97fe443ba" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:17.448621 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet f47da0e523d14388898a4be9c37bb809 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.451406 14530 tablet_service.cc:1467] Processing CreateTablet for tablet a8a5743475814401ac2c0a51cfd70207 (DEFAULT_TABLE table=default.Table_39b71235b2df40ec88411bb02d370d7b [id=3c0f75dc669f4fb181b0d9e32f38f761]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.493160 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83a4f95eb9494a5295a81a8a51f58bd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.493258 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8a5743475814401ac2c0a51cfd70207. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:17.493292 14626 leader_election.cc:343] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.493510 14820 ts_tablet_manager.cc:1939] T 3a0bb9b5093b454d921bf1150d631565 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:17.493582 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a59a9be8fa94ec4bdb42c723ea0ca17" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:17.493856 14356 leader_election.cc:343] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:17.494022 14356 leader_election.cc:343] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.494048 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 3a0bb9b5093b454d921bf1150d631565 (table default.Table_8daa0abab4d1464488bec21bb7ad375b [id=37d965d8a5eb408ea3f33313bd22c152]) successfully deleted
I20250115 03:06:17.494076 14356 leader_election.cc:304] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.494228 14820 tablet_replica.cc:331] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:17.494333 14820 raft_consensus.cc:2238] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.494446 14820 raft_consensus.cc:2267] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.453104 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 85cceb5a3f234d85937ad0bb75348991 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.455379 14526 tablet_service.cc:1467] Processing CreateTablet for tablet 3e170ced1612442ebcfdaff6d9826ea3 (DEFAULT_TABLE table=default.Table_77d4ac27a2c44250a2dcc62121f202c7 [id=6af8f6f562ac47b484eeb4d67ac6c7df]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.495342 14526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e170ced1612442ebcfdaff6d9826ea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.496656 15401 raft_consensus.cc:2743] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.455844 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e63e854c49b34c1ab286f1f97fe443ba" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:17.456177 14525 tablet_service.cc:1467] Processing CreateTablet for tablet 238e68300f304266aaed94bc3b76a01d (DEFAULT_TABLE table=default.Table_169e728924b749e4a9118c12d9b56eab [id=a2689bcea23843c9bb9e1f08e335e58f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.496987 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3eaec2fff184aad816349fc36fa3db1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.497148 14525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 238e68300f304266aaed94bc3b76a01d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.497192 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6132d15b17ab4d0aaa10a7ea04cda7ac. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:17.497225 14626 leader_election.cc:343] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.464516 14553 raft_consensus.cc:3054] T 341b24d9b63f4761b2013035023fcd98 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.465629 14524 tablet_service.cc:1467] Processing CreateTablet for tablet 194d943d173340e7a8fba69967d9b090 (DEFAULT_TABLE table=default.Table_1a2ec424c5cb40fbb3c93f327531e36f [id=50a51914c37b462aa45a71fb260c809b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.499881 14524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 194d943d173340e7a8fba69967d9b090. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.500054 14861 consensus_queue.cc:1035] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.468695 14522 tablet_service.cc:1467] Processing CreateTablet for tablet a3eaec2fff184aad816349fc36fa3db1 (DEFAULT_TABLE table=default.Table_f3b64ddeafb3400583229c6698c2a5e5 [id=538723971ef44b719ff8bf684fdc5676]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.501703 14522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3eaec2fff184aad816349fc36fa3db1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.502365 15617 tablet.cc:1721] T 745344db22da4102bad39c2f121548a1 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.471206 14523 tablet_service.cc:1514] Processing DeleteTablet for tablet f70a69acbdc049beb9d98eada09d9eb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.503523 15426 tablet.cc:1721] T 745344db22da4102bad39c2f121548a1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.473048 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "858bbbb62fc249d5a915697c12362d0f" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:17.474277 14521 tablet_service.cc:1467] Processing CreateTablet for tablet de14b1d525cc42808500279ae6b60e80 (DEFAULT_TABLE table=default.Table_b922b874a412477f8cf1a410e26beb1d [id=76a2384a834c40a2a8ec940808a94290]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:17.503958 14356 leader_election.cc:343] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.503983 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de14b1d525cc42808500279ae6b60e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.504015 14356 leader_election.cc:304] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.504381 14861 raft_consensus.cc:2743] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.504838 14662 tablet_service.cc:1467] Processing CreateTablet for tablet aadcec8aca2b4ece915707c17f60b8b5 (DEFAULT_TABLE table=default.Table_77127dee522147cf8e0f3ec2f5539301 [id=9643860dccb54c598714ceccd0f0e3df]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.505118 14823 ts_tablet_manager.cc:1939] T f70a69acbdc049beb9d98eada09d9eb8 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:17.505173 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aadcec8aca2b4ece915707c17f60b8b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.475054 14520 tablet_service.cc:1467] Processing CreateTablet for tablet 6132d15b17ab4d0aaa10a7ea04cda7ac (DEFAULT_TABLE table=default.Table_35de20a1cb6a488bbc86997188d7ce82 [id=01aa6b0cc55b46d0b4216c601c16b065]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.505733 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet f70a69acbdc049beb9d98eada09d9eb8 (table default.Table_db8a2673977247e0b14b7995f7e09661 [id=f7e4534c1f6f48f09e10b84c1d5c458c]) successfully deleted
I20250115 03:06:17.505838 14520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6132d15b17ab4d0aaa10a7ea04cda7ac. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:17.506086 14158 catalog_manager.cc:4839] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 56c600fe18504e05bab2bcde3c89861d because the tablet was not found. No further retry: Not found: Tablet not found: 56c600fe18504e05bab2bcde3c89861d
I20250115 03:06:17.506497 14820 ts_tablet_manager.cc:1905] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.475663 14519 tablet_service.cc:1514] Processing DeleteTablet for tablet a75b9401cb4b4f8b92a4779c3704eb46 with delete_type TABLET_DATA_DELETED (replaced by tablet 21524aef80f74a6c899ea66b9b022389) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.481835 14853 raft_consensus.cc:491] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.507791 14853 raft_consensus.cc:513] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.508093 14853 leader_election.cc:290] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.482157 14518 tablet_service.cc:1467] Processing CreateTablet for tablet e47012a789bd469f9691e5a4162ff512 (DEFAULT_TABLE table=default.Table_99e4573f330b4bf5b33dac953542aca5 [id=31ee40f75b634219ba0ae4b38e2c757e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.508334 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e47012a789bd469f9691e5a4162ff512. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:17.508591 14493 leader_election.cc:343] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 420b7c8d43a04bae972eed2014d5634c
W20250115 03:06:17.508706 14493 leader_election.cc:343] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 420b7c8d43a04bae972eed2014d5634c
I20250115 03:06:17.508759 14493 leader_election.cc:304] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.509212 14397 tablet_service.cc:1467] Processing CreateTablet for tablet aadcec8aca2b4ece915707c17f60b8b5 (DEFAULT_TABLE table=default.Table_77127dee522147cf8e0f3ec2f5539301 [id=9643860dccb54c598714ceccd0f0e3df]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.509397 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aadcec8aca2b4ece915707c17f60b8b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.510715 14812 tablet_bootstrap.cc:492] T 93fa227c0de74a3fa86778641fdcda3d P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.510790 14812 ts_tablet_manager.cc:1397] T 93fa227c0de74a3fa86778641fdcda3d P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.128s user 0.000s sys 0.001s
I20250115 03:06:17.511227 14812 raft_consensus.cc:357] T 93fa227c0de74a3fa86778641fdcda3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.511337 14812 raft_consensus.cc:383] T 93fa227c0de74a3fa86778641fdcda3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.511373 14812 raft_consensus.cc:738] T 93fa227c0de74a3fa86778641fdcda3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.511488 14812 consensus_queue.cc:260] T 93fa227c0de74a3fa86778641fdcda3d P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.511756 14812 ts_tablet_manager.cc:1428] T 93fa227c0de74a3fa86778641fdcda3d P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:17.513290 15857 raft_consensus.cc:2743] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.513710 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_4b4d4f2b2a2d40b5b717b010b99c5c26"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.514029 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4b4d4f2b2a2d40b5b717b010b99c5c26 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.515625 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b30b12a256e14787bb0f14692ea746ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.515926 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b30b12a256e14787bb0f14692ea746ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.517052 14809 tablet_bootstrap.cc:492] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.517953 14809 tablet_bootstrap.cc:654] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.518730 15390 consensus_queue.cc:1035] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.520648 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4a5e8b0dcd8c40de9598eecf00fec54e" } new_table_name: "default.Table_77b3fb817e8b4502af7fae7bcb0124a5" modify_external_catalogs: true
I20250115 03:06:17.520951 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_c28b01bcd9784e5ead9f2735dec17850" } modify_external_catalogs: true
I20250115 03:06:17.521045 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_c28b01bcd9784e5ead9f2735dec17850" } modify_external_catalogs: true
I20250115 03:06:17.521732 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3e704f1e03e34a2ca2377db61214c5a0" } new_table_name: "default.Table_bb5db03314f149ad8d2a9efb8743db6f" modify_external_catalogs: true
I20250115 03:06:17.524475 14811 tablet_bootstrap.cc:492] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:17.524422 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ab60aad92f7f48fb8519f90f108f57ad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:17.524554 14811 ts_tablet_manager.cc:1397] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.162s user 0.002s sys 0.000s
W20250115 03:06:17.524708 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab60aad92f7f48fb8519f90f108f57ad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.524971 14811 raft_consensus.cc:357] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.525072 14811 raft_consensus.cc:383] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.525099 14811 raft_consensus.cc:738] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.525202 14811 consensus_queue.cc:260] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.525463 14811 ts_tablet_manager.cc:1428] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.525622 14811 tablet_bootstrap.cc:492] T d28ed8538c4a44d18279371b826c22a5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:17.526803 15390 raft_consensus.cc:491] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.526880 15390 raft_consensus.cc:513] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.527192 15390 leader_election.cc:290] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.527556 15274 tablet.cc:1721] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.527807 14165 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_365313892043408c92be8460c31d958e [id=f22f08e9af4a4496b100e7d4b5982121] alter complete (version 1)
I20250115 03:06:17.528256 15617 tablet.cc:1721] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.529098 15274 tablet.cc:1721] T 745344db22da4102bad39c2f121548a1 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20250115 03:06:17.530107 14358 consensus_peers.cc:487] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 -> Peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Couldn't send request to peer 5c0dae7aff4b44c699275b95ba639770. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f70a69acbdc049beb9d98eada09d9eb8. This is attempt 1: this message will repeat every 5th retry.
W20250115 03:06:17.531332 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5cb34cb2006f4716b87bef67f3a6dd8e consensus state has no leader
I20250115 03:06:17.531562 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet b5e7672ebc4442369156d12695261269
I20250115 03:06:17.531600 14518 tablet_service.cc:1514] Processing DeleteTablet for tablet 90277f7d279e4c7a83c43f40766cd11b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.532270 14603 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:17.532446 14519 tablet_service.cc:1514] Processing DeleteTablet for tablet b5e7672ebc4442369156d12695261269 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:17.533569 14358 leader_election.cc:343] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 56c600fe18504e05bab2bcde3c89861d
I20250115 03:06:17.536489 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56c600fe18504e05bab2bcde3c89861d" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:17.537447 14356 leader_election.cc:343] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.537509 14356 leader_election.cc:304] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.537751 14885 raft_consensus.cc:2743] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.539621 14811 tablet_bootstrap.cc:654] T d28ed8538c4a44d18279371b826c22a5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.542680 14812 tablet_bootstrap.cc:492] T 5780da67ce984297ba33d3c6901c9e3a P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:17.544049 14812 tablet_bootstrap.cc:654] T 5780da67ce984297ba33d3c6901c9e3a P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.544128 14738 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:17.544698 14820 ts_tablet_manager.cc:1918] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.544826 14820 log.cc:1198] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/b0a5d554764249f8a7eefdc943271090
I20250115 03:06:17.545114 14820 ts_tablet_manager.cc:1939] T b0a5d554764249f8a7eefdc943271090 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:17.547681 14819 tablet_replica.cc:331] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.548064 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet b0a5d554764249f8a7eefdc943271090 (table default.Table_39e0d37b84fc42c0807a4cbffb7bae0a [id=4b6ce1381c024498a78d91e346f488f2]) successfully deleted
W20250115 03:06:17.548238 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet bb68869eaf55477090cf12f40f76a89c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bb68869eaf55477090cf12f40f76a89c already in progress: creating tablet
I20250115 03:06:17.548270 14819 raft_consensus.cc:2238] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
W20250115 03:06:17.548378 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet ca117aca95f640be880b640c85e18cd4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ca117aca95f640be880b640c85e18cd4 already in progress: creating tablet
I20250115 03:06:17.548512 14820 tablet_replica.cc:331] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:17.548509 14819 raft_consensus.cc:2267] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:17.548534 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 2c69daff62144976ae9082c89cc58eca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2c69daff62144976ae9082c89cc58eca already in progress: creating tablet
I20250115 03:06:17.548720 14820 raft_consensus.cc:2238] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:17.548995 14819 ts_tablet_manager.cc:1905] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.549558 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet b5e7672ebc4442369156d12695261269 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.549594 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 5986558350244e7aa4ddcd5cccb7d676 (table default.Table_e72d1ee69400410c9da9082d4bd8bc9a [id=6ab343bee1254fb19a3024ebafa44464]) successfully deleted
I20250115 03:06:17.549847 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet 93fa227c0de74a3fa86778641fdcda3d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.550690 14823 tablet_replica.cc:331] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:17.550823 14823 raft_consensus.cc:2238] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:17.551010 14823 raft_consensus.cc:2267] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.551417 14823 ts_tablet_manager.cc:1905] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.551132 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet b5e7672ebc4442369156d12695261269 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.553655 14819 ts_tablet_manager.cc:1918] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.553774 14819 log.cc:1198] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/56c600fe18504e05bab2bcde3c89861d
I20250115 03:06:17.554142 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_04c92b10d83444518cc1805f51e8babb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.554451 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_04c92b10d83444518cc1805f51e8babb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.555030 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_dca8868e4ea14ebeaefe647e2188876c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.555294 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dca8868e4ea14ebeaefe647e2188876c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.555884 14823 ts_tablet_manager.cc:1918] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:17.555944 14823 log.cc:1198] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/b5e7672ebc4442369156d12695261269
I20250115 03:06:17.555964 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ba92f093e14445bebf601200eebd2b59"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:17.556159 14823 ts_tablet_manager.cc:1939] T b5e7672ebc4442369156d12695261269 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:17.556306 14170 master_service.cc:946] ReplaceTablet: received request to replace tablet 13ce129d8a9245baab7c772f0f5fbaa7 from {username='slave'} at 127.0.0.1:60432
W20250115 03:06:17.556982 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba92f093e14445bebf601200eebd2b59 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.557322 14819 ts_tablet_manager.cc:1939] T 56c600fe18504e05bab2bcde3c89861d P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.558375 14819 tablet_replica.cc:331] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.558499 14819 raft_consensus.cc:2238] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.558059 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 56c600fe18504e05bab2bcde3c89861d (table default.Table_b94f9ef5ede64627b994e94e674b1cfa [id=3f4310bb3227447ba6fcd011f05d6924]) successfully deleted
I20250115 03:06:17.558537 14468 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:17.558665 14819 raft_consensus.cc:2267] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:17.558738 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet f47da0e523d14388898a4be9c37bb809 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f47da0e523d14388898a4be9c37bb809 already in progress: creating tablet
I20250115 03:06:17.559196 14819 ts_tablet_manager.cc:1905] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.561017 14819 ts_tablet_manager.cc:1918] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.561053 14809 tablet_bootstrap.cc:492] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.561561 14809 ts_tablet_manager.cc:1397] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.045s user 0.002s sys 0.000s
I20250115 03:06:17.561538 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_4db09fb126344de28b8a6b6baa54183a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.561800 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4db09fb126344de28b8a6b6baa54183a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.561970 14809 raft_consensus.cc:357] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.562059 14809 raft_consensus.cc:383] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.562088 14809 raft_consensus.cc:738] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.562187 14809 consensus_queue.cc:260] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.562420 14809 ts_tablet_manager.cc:1428] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.562561 14809 tablet_bootstrap.cc:492] T 93fa227c0de74a3fa86778641fdcda3d P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.563351 14809 tablet_bootstrap.cc:654] T 93fa227c0de74a3fa86778641fdcda3d P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.565261 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet 93fa227c0de74a3fa86778641fdcda3d from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:17.565340 15617 tablet.cc:1721] T 0580a8f7b2984162b9275d74334b1905 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:17.566411 14819 log.cc:1198] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/6a59a9be8fa94ec4bdb42c723ea0ca17
I20250115 03:06:17.566857 14819 ts_tablet_manager.cc:1939] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.567368 14819 tablet_replica.cc:331] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.567492 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 6a59a9be8fa94ec4bdb42c723ea0ca17 (table default.Table_68e894a7b2d7416a90c519daa7bf4fb9 [id=5b343e28259444528cbad21c3b048db5]) successfully deleted
I20250115 03:06:17.567684 14819 raft_consensus.cc:2238] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.567828 14819 raft_consensus.cc:2267] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.568259 14819 ts_tablet_manager.cc:1905] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.569278 14812 tablet_bootstrap.cc:492] T 5780da67ce984297ba33d3c6901c9e3a P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.569363 14812 ts_tablet_manager.cc:1397] T 5780da67ce984297ba33d3c6901c9e3a P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.001s
I20250115 03:06:17.569907 14812 raft_consensus.cc:357] T 5780da67ce984297ba33d3c6901c9e3a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.569999 14812 raft_consensus.cc:383] T 5780da67ce984297ba33d3c6901c9e3a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.570029 14812 raft_consensus.cc:738] T 5780da67ce984297ba33d3c6901c9e3a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.570039 14819 ts_tablet_manager.cc:1918] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.570091 14819 log.cc:1198] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/858bbbb62fc249d5a915697c12362d0f
I20250115 03:06:17.570139 14812 consensus_queue.cc:260] T 5780da67ce984297ba33d3c6901c9e3a P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.570211 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 13ce129d8a9245baab7c772f0f5fbaa7
I20250115 03:06:17.570259 14819 ts_tablet_manager.cc:1939] T 858bbbb62fc249d5a915697c12362d0f P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.570394 14812 ts_tablet_manager.cc:1428] T 5780da67ce984297ba33d3c6901c9e3a P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.570457 14170 catalog_manager.cc:6596] ReplaceTablet: tablet 13ce129d8a9245baab7c772f0f5fbaa7 deleted and replaced by tablet b3334f58b90b4947af52150546ccaa28
I20250115 03:06:17.570573 14812 tablet_bootstrap.cc:492] T 8efd181c42eb42628a9f36fad4fa8d64 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:17.571436 14812 tablet_bootstrap.cc:654] T 8efd181c42eb42628a9f36fad4fa8d64 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.572023 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 13ce129d8a9245baab7c772f0f5fbaa7 with delete_type TABLET_DATA_DELETED (replaced by tablet b3334f58b90b4947af52150546ccaa28) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.573024 14163 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_70d0b700eac34892b4ef769d739481fd [id=61f3625fed2b4a70b5331d6c6ac73f1c] alter complete (version 1)
I20250115 03:06:17.573860 14688 raft_consensus.cc:1270] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250115 03:06:17.574173 14356 consensus_peers.cc:487] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:17.574786 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet 13ce129d8a9245baab7c772f0f5fbaa7 with delete_type TABLET_DATA_DELETED (replaced by tablet b3334f58b90b4947af52150546ccaa28) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.575145 14809 tablet_bootstrap.cc:492] T 93fa227c0de74a3fa86778641fdcda3d P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.575219 14809 ts_tablet_manager.cc:1397] T 93fa227c0de74a3fa86778641fdcda3d P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20250115 03:06:17.575604 14809 raft_consensus.cc:357] T 93fa227c0de74a3fa86778641fdcda3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.575700 14809 raft_consensus.cc:383] T 93fa227c0de74a3fa86778641fdcda3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.575738 14809 raft_consensus.cc:738] T 93fa227c0de74a3fa86778641fdcda3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.575848 14809 consensus_queue.cc:260] T 93fa227c0de74a3fa86778641fdcda3d P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.576094 14809 ts_tablet_manager.cc:1428] T 93fa227c0de74a3fa86778641fdcda3d P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.576231 14809 tablet_bootstrap.cc:492] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.576575 14169 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 5780da67ce984297ba33d3c6901c9e3a (table default.Table_551ca9fefcb74ed5aa29803f19de6ab6 [id=79eb8c9694b74a31a4253b33d064e230]). Expected version 1 got 0
I20250115 03:06:17.552999 14853 raft_consensus.cc:491] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.577018 15390 consensus_queue.cc:1035] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.577004 14853 raft_consensus.cc:513] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.577116 14809 tablet_bootstrap.cc:654] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.577353 14853 leader_election.cc:290] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.562004 15426 tablet.cc:1721] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:17.567932 14858 raft_consensus.cc:491] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.574110 14519 tablet_service.cc:1514] Processing DeleteTablet for tablet 13ce129d8a9245baab7c772f0f5fbaa7 with delete_type TABLET_DATA_DELETED (replaced by tablet b3334f58b90b4947af52150546ccaa28) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.574326 14518 tablet_service.cc:1467] Processing CreateTablet for tablet 78cc96de8b414ac19285338ccb1024e0 (DEFAULT_TABLE table=default.Table_e6dfaa9c84d144e68a2d047436d0ecd2 [id=63ea052eebb34ba4a66eded622ccd4a4]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:17.577872 14493 leader_election.cc:343] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: f8b90f854a5e4221a24986f8c71ca300
I20250115 03:06:17.575671 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 78b5e2e16442407880b75d256e6b2614 (DEFAULT_TABLE table=default.Table_ab72071c412848939b54fcc72d5974fa [id=aff8cb5016d84881917b303cb3c2e4d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.574581 14520 tablet_service.cc:1467] Processing CreateTablet for tablet fe8d94c8fecc4077ba052390810c7430 (DEFAULT_TABLE table=default.Table_4f5d0494ec384c479419c10b7ff36dc0 [id=b9026c7e60a14240b2f5c596d31ba517]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.575490 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 78cc96de8b414ac19285338ccb1024e0 (DEFAULT_TABLE table=default.Table_e6dfaa9c84d144e68a2d047436d0ecd2 [id=63ea052eebb34ba4a66eded622ccd4a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.576377 14522 tablet_service.cc:1467] Processing CreateTablet for tablet d5e4743e016e49ed8ba26b1ac96b1470 (DEFAULT_TABLE table=default.Table_e72d1ee69400410c9da9082d4bd8bc9a [id=6ab343bee1254fb19a3024ebafa44464]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.576607 14524 tablet_service.cc:1467] Processing CreateTablet for tablet cef4442bfcfd4ad4b343a2785c3a9a74 (DEFAULT_TABLE table=default.Table_b94f9ef5ede64627b994e94e674b1cfa [id=3f4310bb3227447ba6fcd011f05d6924]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:17.578359 14493 leader_election.cc:343] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: f8b90f854a5e4221a24986f8c71ca300
I20250115 03:06:17.578416 14493 leader_election.cc:304] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.576169 14523 tablet_service.cc:1467] Processing CreateTablet for tablet f2b8bd783b614bf9b5526de197c51df3 (DEFAULT_TABLE table=default.Table_f1bb182509e64b589f4cb90482cec747 [id=5ebda2c8e8c14092a6e894b5502e4763]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.577658 14858 raft_consensus.cc:513] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.577893 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78cc96de8b414ac19285338ccb1024e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.578737 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78cc96de8b414ac19285338ccb1024e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.578755 14858 leader_election.cc:290] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.578825 14858 raft_consensus.cc:2743] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:17.579452 14493 leader_election.cc:343] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 33380937a3b7428995b06e477a896c02
I20250115 03:06:17.579999 14819 tablet_replica.cc:331] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.580111 14819 raft_consensus.cc:2238] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:17.580260 14819 raft_consensus.cc:2267] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:17.580569 14493 leader_election.cc:343] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 33380937a3b7428995b06e477a896c02
I20250115 03:06:17.580646 14493 leader_election.cc:304] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.580757 14853 raft_consensus.cc:2743] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.580955 14820 raft_consensus.cc:2267] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.581569 14819 ts_tablet_manager.cc:1905] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.581583 14823 tablet_replica.cc:331] T 93fa227c0de74a3fa86778641fdcda3d P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:17.581701 14823 raft_consensus.cc:2238] T 93fa227c0de74a3fa86778641fdcda3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.581794 14823 raft_consensus.cc:2267] T 93fa227c0de74a3fa86778641fdcda3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.582193 14823 ts_tablet_manager.cc:1905] T 93fa227c0de74a3fa86778641fdcda3d P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.582580 14522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5e4743e016e49ed8ba26b1ac96b1470. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.582676 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 78cc96de8b414ac19285338ccb1024e0 (DEFAULT_TABLE table=default.Table_e6dfaa9c84d144e68a2d047436d0ecd2 [id=63ea052eebb34ba4a66eded622ccd4a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.582878 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78cc96de8b414ac19285338ccb1024e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.584134 14823 ts_tablet_manager.cc:1918] T 93fa227c0de74a3fa86778641fdcda3d P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.584194 14823 log.cc:1198] T 93fa227c0de74a3fa86778641fdcda3d P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/93fa227c0de74a3fa86778641fdcda3d
I20250115 03:06:17.584506 14524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cef4442bfcfd4ad4b343a2785c3a9a74. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.584594 14823 ts_tablet_manager.cc:1939] T 93fa227c0de74a3fa86778641fdcda3d P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:17.586205 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b5e2e16442407880b75d256e6b2614. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.586612 14169 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_bb5db03314f149ad8d2a9efb8743db6f [id=0790b08615854d3688f940aba20b6520] alter complete (version 2)
I20250115 03:06:17.587914 14520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe8d94c8fecc4077ba052390810c7430. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.588238 14664 tablet_service.cc:1467] Processing CreateTablet for tablet fe8d94c8fecc4077ba052390810c7430 (DEFAULT_TABLE table=default.Table_4f5d0494ec384c479419c10b7ff36dc0 [id=b9026c7e60a14240b2f5c596d31ba517]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.588433 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe8d94c8fecc4077ba052390810c7430. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.588570 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 78b5e2e16442407880b75d256e6b2614 (DEFAULT_TABLE table=default.Table_ab72071c412848939b54fcc72d5974fa [id=aff8cb5016d84881917b303cb3c2e4d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.588773 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b5e2e16442407880b75d256e6b2614. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.590027 14523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2b8bd783b614bf9b5526de197c51df3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.591121 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 78b5e2e16442407880b75d256e6b2614 (DEFAULT_TABLE table=default.Table_ab72071c412848939b54fcc72d5974fa [id=aff8cb5016d84881917b303cb3c2e4d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.591336 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78b5e2e16442407880b75d256e6b2614. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.591624 14669 tablet_service.cc:1467] Processing CreateTablet for tablet f2b8bd783b614bf9b5526de197c51df3 (DEFAULT_TABLE table=default.Table_f1bb182509e64b589f4cb90482cec747 [id=5ebda2c8e8c14092a6e894b5502e4763]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.591809 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2b8bd783b614bf9b5526de197c51df3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.593667 14169 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 38c9e64e1f5e4f2394afa876553357bf (table default.Table_d94cabfa959d4cbbbd0d5552a6b1829b [id=91731a4895d4478fad68b1bcbdc45d5d]). Expected version 1 got 0
I20250115 03:06:17.593966 14399 tablet_service.cc:1467] Processing CreateTablet for tablet fe8d94c8fecc4077ba052390810c7430 (DEFAULT_TABLE table=default.Table_4f5d0494ec384c479419c10b7ff36dc0 [id=b9026c7e60a14240b2f5c596d31ba517]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.594203 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe8d94c8fecc4077ba052390810c7430. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.594998 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d94cabfa959d4cbbbd0d5552a6b1829b" } new_table_name: "default.Table_57e91bade4c54372abe1dbba44434098" modify_external_catalogs: true
I20250115 03:06:17.595424 14396 tablet_service.cc:1467] Processing CreateTablet for tablet d5e4743e016e49ed8ba26b1ac96b1470 (DEFAULT_TABLE table=default.Table_e72d1ee69400410c9da9082d4bd8bc9a [id=6ab343bee1254fb19a3024ebafa44464]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.595630 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5e4743e016e49ed8ba26b1ac96b1470. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.596138 14397 tablet_service.cc:1467] Processing CreateTablet for tablet f2b8bd783b614bf9b5526de197c51df3 (DEFAULT_TABLE table=default.Table_f1bb182509e64b589f4cb90482cec747 [id=5ebda2c8e8c14092a6e894b5502e4763]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.596320 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2b8bd783b614bf9b5526de197c51df3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.597529 14820 ts_tablet_manager.cc:1905] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.600979 14820 ts_tablet_manager.cc:1918] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:17.601191 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 56c600fe18504e05bab2bcde3c89861d with delete_type TABLET_DATA_DELETED (replaced by tablet cef4442bfcfd4ad4b343a2785c3a9a74) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.601449 14820 log.cc:1198] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/a4c3ff892f9b4dc484ffc80d7c5934c4
I20250115 03:06:17.601699 14820 ts_tablet_manager.cc:1939] T a4c3ff892f9b4dc484ffc80d7c5934c4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:17.601920 14664 tablet_service.cc:1467] Processing CreateTablet for tablet cef4442bfcfd4ad4b343a2785c3a9a74 (DEFAULT_TABLE table=default.Table_b94f9ef5ede64627b994e94e674b1cfa [id=3f4310bb3227447ba6fcd011f05d6924]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.601886 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet b5e7672ebc4442369156d12695261269 (table default.Table_c28b01bcd9784e5ead9f2735dec17850 [id=ab8607ff8b974b59b7a168cf5897ed9f]) successfully deleted
I20250115 03:06:17.602110 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cef4442bfcfd4ad4b343a2785c3a9a74. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.602113 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 93fa227c0de74a3fa86778641fdcda3d (table default.Table_4f241c86102a4bbfb509a2d5d76953e9 [id=03b92c08d0b0473d83090c58e450c84b]) successfully deleted
I20250115 03:06:17.602375 14812 tablet_bootstrap.cc:492] T 8efd181c42eb42628a9f36fad4fa8d64 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.602440 14812 ts_tablet_manager.cc:1397] T 8efd181c42eb42628a9f36fad4fa8d64 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.000s
I20250115 03:06:17.602663 14669 tablet_service.cc:1467] Processing CreateTablet for tablet d5e4743e016e49ed8ba26b1ac96b1470 (DEFAULT_TABLE table=default.Table_e72d1ee69400410c9da9082d4bd8bc9a [id=6ab343bee1254fb19a3024ebafa44464]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.602875 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5e4743e016e49ed8ba26b1ac96b1470. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.602855 14812 raft_consensus.cc:357] T 8efd181c42eb42628a9f36fad4fa8d64 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.602941 14812 raft_consensus.cc:383] T 8efd181c42eb42628a9f36fad4fa8d64 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.602970 14812 raft_consensus.cc:738] T 8efd181c42eb42628a9f36fad4fa8d64 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.603080 14812 consensus_queue.cc:260] T 8efd181c42eb42628a9f36fad4fa8d64 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.603339 14812 ts_tablet_manager.cc:1428] T 8efd181c42eb42628a9f36fad4fa8d64 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.603506 14812 tablet_bootstrap.cc:492] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:17.601691 14399 tablet_service.cc:1467] Processing CreateTablet for tablet cef4442bfcfd4ad4b343a2785c3a9a74 (DEFAULT_TABLE table=default.Table_b94f9ef5ede64627b994e94e674b1cfa [id=3f4310bb3227447ba6fcd011f05d6924]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.604326 14812 tablet_bootstrap.cc:654] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.608314 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4f5d0494ec384c479419c10b7ff36dc0" } modify_external_catalogs: true
I20250115 03:06:17.608371 14809 tablet_bootstrap.cc:492] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.608436 14809 ts_tablet_manager.cc:1397] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20250115 03:06:17.608784 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4f5d0494ec384c479419c10b7ff36dc0" } modify_external_catalogs: true
I20250115 03:06:17.608978 14809 raft_consensus.cc:357] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.609086 14809 raft_consensus.cc:383] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.609115 14809 raft_consensus.cc:738] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.609215 14809 consensus_queue.cc:260] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.609419 14809 ts_tablet_manager.cc:1428] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.608553 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_c1f030f5a1ef49a8826714b1b730459a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.609815 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c1f030f5a1ef49a8826714b1b730459a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.611347 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_018ec24d90424975aca7b77736de6136" } new_table_name: "default.Table_b900c4395de448a08b0285a5807f2fdf" modify_external_catalogs: true
I20250115 03:06:17.613041 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cef4442bfcfd4ad4b343a2785c3a9a74. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.613807 14819 ts_tablet_manager.cc:1918] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:17.614944 14809 tablet_bootstrap.cc:492] T 6784e353df554bb2aa10b667c8b488da P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.616328 14819 log.cc:1198] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/f70a69acbdc049beb9d98eada09d9eb8
I20250115 03:06:17.616559 14819 ts_tablet_manager.cc:1939] T f70a69acbdc049beb9d98eada09d9eb8 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.617199 14809 tablet_bootstrap.cc:654] T 6784e353df554bb2aa10b667c8b488da P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.617221 14819 tablet_replica.cc:331] T b5e7672ebc4442369156d12695261269 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.617323 14819 raft_consensus.cc:2238] T b5e7672ebc4442369156d12695261269 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.617404 14819 raft_consensus.cc:2267] T b5e7672ebc4442369156d12695261269 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.617689 14819 ts_tablet_manager.cc:1905] T b5e7672ebc4442369156d12695261269 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.618983 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet fe8d94c8fecc4077ba052390810c7430
I20250115 03:06:17.619408 14819 ts_tablet_manager.cc:1918] T b5e7672ebc4442369156d12695261269 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:17.619515 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_bf4a2c34cc7d42acabda873e7d47d3fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:17.619575 14523 tablet_service.cc:1514] Processing DeleteTablet for tablet fe8d94c8fecc4077ba052390810c7430 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:17.619823 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bf4a2c34cc7d42acabda873e7d47d3fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.622648 14812 tablet_bootstrap.cc:492] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.622740 14812 ts_tablet_manager.cc:1397] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.001s
I20250115 03:06:17.623171 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_1a535f4e7104456598818042ce64476f" } new_table_name: "default.Table_f7799772605a4284a062cd0e4ae15ed3" modify_external_catalogs: true
I20250115 03:06:17.623160 14812 raft_consensus.cc:357] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.623250 14812 raft_consensus.cc:383] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.623279 14812 raft_consensus.cc:738] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.623391 14812 consensus_queue.cc:260] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.624539 14812 ts_tablet_manager.cc:1428] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:17.625650 14811 tablet_bootstrap.cc:492] T d28ed8538c4a44d18279371b826c22a5 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:17.625741 14811 ts_tablet_manager.cc:1397] T d28ed8538c4a44d18279371b826c22a5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.100s user 0.003s sys 0.000s
I20250115 03:06:17.626181 14811 raft_consensus.cc:357] T d28ed8538c4a44d18279371b826c22a5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.626281 14811 raft_consensus.cc:383] T d28ed8538c4a44d18279371b826c22a5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.626312 14811 raft_consensus.cc:738] T d28ed8538c4a44d18279371b826c22a5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.626425 14811 consensus_queue.cc:260] T d28ed8538c4a44d18279371b826c22a5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.626506 14812 tablet_bootstrap.cc:492] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:17.626704 14811 ts_tablet_manager.cc:1428] T d28ed8538c4a44d18279371b826c22a5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.627326 14823 tablet_replica.cc:331] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:17.627352 14812 tablet_bootstrap.cc:654] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.627460 14819 log.cc:1198] T b5e7672ebc4442369156d12695261269 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/b5e7672ebc4442369156d12695261269
I20250115 03:06:17.627779 14819 ts_tablet_manager.cc:1939] T b5e7672ebc4442369156d12695261269 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.627806 15617 tablet.cc:1721] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.628194 14823 raft_consensus.cc:2238] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.628297 14823 raft_consensus.cc:2267] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.629177 15274 tablet.cc:1721] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:17.631239 14819 tablet_replica.cc:331] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.631450 15274 tablet.cc:1721] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.631429 14819 raft_consensus.cc:2238] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:17.631656 14819 raft_consensus.cc:2267] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.633121 14823 ts_tablet_manager.cc:1905] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.635090 14823 ts_tablet_manager.cc:1918] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:17.635205 14823 log.cc:1198] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/13ce129d8a9245baab7c772f0f5fbaa7
I20250115 03:06:17.635499 14823 ts_tablet_manager.cc:1939] T 13ce129d8a9245baab7c772f0f5fbaa7 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:17.635452 14523 tablet_service.cc:1467] Processing CreateTablet for tablet 87b234185fc64505b097f2ae7d52d813 (DEFAULT_TABLE table=default.Table_dca8868e4ea14ebeaefe647e2188876c [id=bd521742ddaa43d18ef24465d5ba313b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.635699 14523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87b234185fc64505b097f2ae7d52d813. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.636057 14812 tablet_bootstrap.cc:492] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.636124 14812 ts_tablet_manager.cc:1397] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20250115 03:06:17.636615 14812 raft_consensus.cc:357] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.636703 14812 raft_consensus.cc:383] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.636734 14812 raft_consensus.cc:738] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.636831 14812 consensus_queue.cc:260] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:17.636982 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8b0f0539530643f78207e528c39530af consensus state has no leader
I20250115 03:06:17.637159 14812 ts_tablet_manager.cc:1428] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:17.637305 14812 tablet_bootstrap.cc:492] T 58130f2b27b745d08c0002ac96af6d72 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:17.637452 14819 ts_tablet_manager.cc:1905] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.637763 14520 tablet_service.cc:1467] Processing CreateTablet for tablet 041a30d6ee3247df883d00b5896b0eb0 (DEFAULT_TABLE table=default.Table_b30b12a256e14787bb0f14692ea746ee [id=17f2b7f18af342278fa6a7a95d8ea85c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.637962 14520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 041a30d6ee3247df883d00b5896b0eb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.638339 14812 tablet_bootstrap.cc:654] T 58130f2b27b745d08c0002ac96af6d72 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.639770 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 2e7cf4fc3be8446292b0c91141eeadcc (DEFAULT_TABLE table=default.Table_4b4d4f2b2a2d40b5b717b010b99c5c26 [id=cab546ca09ce4671a31dd9d260155528]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.639976 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e7cf4fc3be8446292b0c91141eeadcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.641157 14811 tablet_bootstrap.cc:492] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:17.641974 14811 tablet_bootstrap.cc:654] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.642696 14524 tablet_service.cc:1467] Processing CreateTablet for tablet 1202288f2442477badeb35667dd08dab (DEFAULT_TABLE table=default.Table_ba92f093e14445bebf601200eebd2b59 [id=8b4cf3ac45024f328efaa975fd6c6014]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.642925 14524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1202288f2442477badeb35667dd08dab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.644721 14522 tablet_service.cc:1467] Processing CreateTablet for tablet b3334f58b90b4947af52150546ccaa28 (DEFAULT_TABLE table=default.Table_d58c19b62a194b46858ee8955ebf3f82 [id=6cba4fb2f7104c6e953340b04e5dc494]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.645041 14518 tablet_service.cc:1467] Processing CreateTablet for tablet 60997f625d074960ad75aef5ba14b9c2 (DEFAULT_TABLE table=default.Table_4db09fb126344de28b8a6b6baa54183a [id=bace8b6ea7894ccb96e0e31954917604]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.645761 14522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3334f58b90b4947af52150546ccaa28. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.647900 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60997f625d074960ad75aef5ba14b9c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.648962 14853 raft_consensus.cc:491] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.649040 14853 raft_consensus.cc:513] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.649338 14853 leader_election.cc:290] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.651217 14520 tablet_service.cc:1467] Processing CreateTablet for tablet ef0994b1e51b4c688354998730e7b743 (DEFAULT_TABLE table=default.Table_04c92b10d83444518cc1805f51e8babb [id=ecace87761f746999c8cca0525485c6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.651386 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet a4c3ff892f9b4dc484ffc80d7c5934c4 (table default.Table_76ef9277dc8142f1a86962bfc430f2b2 [id=f70558594b8348579a0ad5d4c8fd8885]) successfully deleted
I20250115 03:06:17.651448 14520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef0994b1e51b4c688354998730e7b743. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:17.651232 14493 leader_election.cc:343] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: a75b9401cb4b4f8b92a4779c3704eb46
W20250115 03:06:17.651705 14493 leader_election.cc:343] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: a75b9401cb4b4f8b92a4779c3704eb46
I20250115 03:06:17.651753 14493 leader_election.cc:304] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.651925 14853 raft_consensus.cc:2743] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.652175 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_96c8d5f733034ee3a6dd8a61dabfd288" } modify_external_catalogs: true
I20250115 03:06:17.652266 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_96c8d5f733034ee3a6dd8a61dabfd288" } modify_external_catalogs: true
I20250115 03:06:17.653540 14819 ts_tablet_manager.cc:1918] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:17.653604 14819 log.cc:1198] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/13ce129d8a9245baab7c772f0f5fbaa7
I20250115 03:06:17.654139 14819 ts_tablet_manager.cc:1939] T 13ce129d8a9245baab7c772f0f5fbaa7 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.656570 14885 consensus_queue.cc:1035] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.656690 14523 tablet_service.cc:1467] Processing CreateTablet for tablet 836541bcc82d4644b4c5eb2f3f25c030 (DEFAULT_TABLE table=default.Table_ab60aad92f7f48fb8519f90f108f57ad [id=5159afd6260c4871a21e6b2442795e9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.656905 14523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 836541bcc82d4644b4c5eb2f3f25c030. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:17.658972 14356 consensus_peers.cc:487] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:17.659206 14809 tablet_bootstrap.cc:492] T 6784e353df554bb2aa10b667c8b488da P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.659281 14809 ts_tablet_manager.cc:1397] T 6784e353df554bb2aa10b667c8b488da P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.044s user 0.002s sys 0.000s
I20250115 03:06:17.659693 14809 raft_consensus.cc:357] T 6784e353df554bb2aa10b667c8b488da P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.659786 14809 raft_consensus.cc:383] T 6784e353df554bb2aa10b667c8b488da P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.659813 14809 raft_consensus.cc:738] T 6784e353df554bb2aa10b667c8b488da P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.659907 14809 consensus_queue.cc:260] T 6784e353df554bb2aa10b667c8b488da P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.660149 14809 ts_tablet_manager.cc:1428] T 6784e353df554bb2aa10b667c8b488da P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.660264 14809 tablet_bootstrap.cc:492] T a1b8b5b7fe03496ea2639dbc9c920974 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.660887 15325 raft_consensus.cc:491] T 8efd181c42eb42628a9f36fad4fa8d64 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.660986 15325 raft_consensus.cc:513] T 8efd181c42eb42628a9f36fad4fa8d64 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.661270 15325 leader_election.cc:290] T 8efd181c42eb42628a9f36fad4fa8d64 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:17.661800 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet 6ed47b26cdf744acb28fb657a1b41087 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:17.661881 14809 tablet_bootstrap.cc:654] T a1b8b5b7fe03496ea2639dbc9c920974 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.662556 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8efd181c42eb42628a9f36fad4fa8d64" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:17.662914 14626 leader_election.cc:343] T 8efd181c42eb42628a9f36fad4fa8d64 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.663712 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8efd181c42eb42628a9f36fad4fa8d64" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:17.663897 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet 398b93f1db0741a9b54837bca43b1bad from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:17.663980 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_561cca4d810b4e73934fcee99a8994c7" } modify_external_catalogs: true
W20250115 03:06:17.664050 14628 leader_election.cc:343] T 8efd181c42eb42628a9f36fad4fa8d64 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.664077 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_561cca4d810b4e73934fcee99a8994c7" } modify_external_catalogs: true
I20250115 03:06:17.664108 14628 leader_election.cc:304] T 8efd181c42eb42628a9f36fad4fa8d64 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.664270 15325 raft_consensus.cc:2743] T 8efd181c42eb42628a9f36fad4fa8d64 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.666338 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ea8049593bb4406883df33d6635d42e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.666666 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ea8049593bb4406883df33d6635d42e5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.668339 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 858bbbb62fc249d5a915697c12362d0f (table default.Table_79d7aa1c665d4695a2a17501157b3807 [id=5cd92dd0835f4c9d88d917d8ff277bbb]) successfully deleted
W20250115 03:06:17.668545 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 85cceb5a3f234d85937ad0bb75348991 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 85cceb5a3f234d85937ad0bb75348991 already in progress: creating tablet
I20250115 03:06:17.668992 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 0f3bf849316a4ebd841c3345ca17087e
I20250115 03:06:17.669903 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet f70a69acbdc049beb9d98eada09d9eb8 (table default.Table_db8a2673977247e0b14b7995f7e09661 [id=f7e4534c1f6f48f09e10b84c1d5c458c]) successfully deleted
I20250115 03:06:17.669966 14523 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3bf849316a4ebd841c3345ca17087e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.670060 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet b5e7672ebc4442369156d12695261269 (table default.Table_c28b01bcd9784e5ead9f2735dec17850 [id=ab8607ff8b974b59b7a168cf5897ed9f]) successfully deleted
I20250115 03:06:17.670356 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 13ce129d8a9245baab7c772f0f5fbaa7 (table default.Table_d58c19b62a194b46858ee8955ebf3f82 [id=6cba4fb2f7104c6e953340b04e5dc494]) successfully deleted
W20250115 03:06:17.670542 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 56c600fe18504e05bab2bcde3c89861d because the tablet was not found. No further retry: Not found: Tablet not found: 56c600fe18504e05bab2bcde3c89861d
I20250115 03:06:17.671628 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet fe8d94c8fecc4077ba052390810c7430 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.672761 14520 tablet_service.cc:1467] Processing CreateTablet for tablet bf6199cdd6e940d1970dd43d8090100d (DEFAULT_TABLE table=default.Table_bf4a2c34cc7d42acabda873e7d47d3fb [id=0f03d804c7114afcb8d40e7e00025ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.673045 14520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf6199cdd6e940d1970dd43d8090100d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.673635 15617 tablet.cc:1721] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:17.674706 15274 tablet.cc:1721] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:17.677718 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_84b19a144ef44b0380ee4e6c66a26370" } modify_external_catalogs: true
I20250115 03:06:17.677812 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_84b19a144ef44b0380ee4e6c66a26370" } modify_external_catalogs: true
I20250115 03:06:17.678119 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 38c9e64e1f5e4f2394afa876553357bf (table default.Table_57e91bade4c54372abe1dbba44434098 [id=91731a4895d4478fad68b1bcbdc45d5d]). Expected version 2 got 1
I20250115 03:06:17.679195 14523 tablet_service.cc:1467] Processing CreateTablet for tablet d643bd0947c54bcfa79f15d88e1e53e4 (DEFAULT_TABLE table=default.Table_c1f030f5a1ef49a8826714b1b730459a [id=570edeb30f8841ab8c07f4e78353990f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.679262 14669 tablet_service.cc:1514] Processing DeleteTablet for tablet fe8d94c8fecc4077ba052390810c7430 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.679400 14523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d643bd0947c54bcfa79f15d88e1e53e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.684507 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 3b9c09b6315345039cdea9eeee3f181a
I20250115 03:06:17.684891 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 398b93f1db0741a9b54837bca43b1bad
I20250115 03:06:17.685156 14168 catalog_manager.cc:6596] ReplaceTablet: tablet 398b93f1db0741a9b54837bca43b1bad deleted and replaced by tablet c1a81dda6ac44432927fa2c90b3e0002
I20250115 03:06:17.685357 14820 tablet_replica.cc:331] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:17.685508 14820 raft_consensus.cc:2238] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
W20250115 03:06:17.685503 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 7d894471f7b94aec8a19df043906d7e7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7d894471f7b94aec8a19df043906d7e7 already in progress: creating tablet
I20250115 03:06:17.685652 14820 raft_consensus.cc:2267] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.685752 14520 tablet_service.cc:1514] Processing DeleteTablet for tablet 398b93f1db0741a9b54837bca43b1bad with delete_type TABLET_DATA_DELETED (replaced by tablet c1a81dda6ac44432927fa2c90b3e0002) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.685878 14523 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b9c09b6315345039cdea9eeee3f181a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.686110 14820 ts_tablet_manager.cc:1905] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.687975 14820 ts_tablet_manager.cc:1918] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.688278 14820 log.cc:1198] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/33380937a3b7428995b06e477a896c02
I20250115 03:06:17.688997 14170 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet c1a81dda6ac44432927fa2c90b3e0002
I20250115 03:06:17.691552 14167 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_57e91bade4c54372abe1dbba44434098 [id=91731a4895d4478fad68b1bcbdc45d5d] alter complete (version 2)
I20250115 03:06:17.693347 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 87b234185fc64505b097f2ae7d52d813 (DEFAULT_TABLE table=default.Table_dca8868e4ea14ebeaefe647e2188876c [id=bd521742ddaa43d18ef24465d5ba313b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.693581 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87b234185fc64505b097f2ae7d52d813. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.693921 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 041a30d6ee3247df883d00b5896b0eb0 (DEFAULT_TABLE table=default.Table_b30b12a256e14787bb0f14692ea746ee [id=17f2b7f18af342278fa6a7a95d8ea85c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.694036 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 041a30d6ee3247df883d00b5896b0eb0 (DEFAULT_TABLE table=default.Table_b30b12a256e14787bb0f14692ea746ee [id=17f2b7f18af342278fa6a7a95d8ea85c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.694133 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 041a30d6ee3247df883d00b5896b0eb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.694377 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 041a30d6ee3247df883d00b5896b0eb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.696632 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 87b234185fc64505b097f2ae7d52d813 (DEFAULT_TABLE table=default.Table_dca8868e4ea14ebeaefe647e2188876c [id=bd521742ddaa43d18ef24465d5ba313b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.696734 14809 tablet_bootstrap.cc:492] T a1b8b5b7fe03496ea2639dbc9c920974 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.696820 14809 ts_tablet_manager.cc:1397] T a1b8b5b7fe03496ea2639dbc9c920974 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.037s user 0.001s sys 0.000s
I20250115 03:06:17.696844 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87b234185fc64505b097f2ae7d52d813. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.697490 14809 raft_consensus.cc:357] T a1b8b5b7fe03496ea2639dbc9c920974 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.697600 14809 raft_consensus.cc:383] T a1b8b5b7fe03496ea2639dbc9c920974 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.697634 14809 raft_consensus.cc:738] T a1b8b5b7fe03496ea2639dbc9c920974 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.697736 14809 consensus_queue.cc:260] T a1b8b5b7fe03496ea2639dbc9c920974 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.697516 15617 tablet.cc:1721] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.697993 14809 ts_tablet_manager.cc:1428] T a1b8b5b7fe03496ea2639dbc9c920974 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.698136 14809 tablet_bootstrap.cc:492] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.698979 14809 tablet_bootstrap.cc:654] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.700546 15426 tablet.cc:1721] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.702898 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 2e7cf4fc3be8446292b0c91141eeadcc (DEFAULT_TABLE table=default.Table_4b4d4f2b2a2d40b5b717b010b99c5c26 [id=cab546ca09ce4671a31dd9d260155528]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.703105 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e7cf4fc3be8446292b0c91141eeadcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.703188 14669 tablet_service.cc:1467] Processing CreateTablet for tablet ef0994b1e51b4c688354998730e7b743 (DEFAULT_TABLE table=default.Table_04c92b10d83444518cc1805f51e8babb [id=ecace87761f746999c8cca0525485c6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.703305 14397 tablet_service.cc:1467] Processing CreateTablet for tablet ef0994b1e51b4c688354998730e7b743 (DEFAULT_TABLE table=default.Table_04c92b10d83444518cc1805f51e8babb [id=ecace87761f746999c8cca0525485c6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.703500 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef0994b1e51b4c688354998730e7b743. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.703516 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 836541bcc82d4644b4c5eb2f3f25c030 (DEFAULT_TABLE table=default.Table_ab60aad92f7f48fb8519f90f108f57ad [id=5159afd6260c4871a21e6b2442795e9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.703694 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 836541bcc82d4644b4c5eb2f3f25c030. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.703680 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 836541bcc82d4644b4c5eb2f3f25c030 (DEFAULT_TABLE table=default.Table_ab60aad92f7f48fb8519f90f108f57ad [id=5159afd6260c4871a21e6b2442795e9b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.703886 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 836541bcc82d4644b4c5eb2f3f25c030. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.705808 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef0994b1e51b4c688354998730e7b743. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.706178 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 1202288f2442477badeb35667dd08dab (DEFAULT_TABLE table=default.Table_ba92f093e14445bebf601200eebd2b59 [id=8b4cf3ac45024f328efaa975fd6c6014]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.706327 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 1202288f2442477badeb35667dd08dab (DEFAULT_TABLE table=default.Table_ba92f093e14445bebf601200eebd2b59 [id=8b4cf3ac45024f328efaa975fd6c6014]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.706372 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1202288f2442477badeb35667dd08dab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.706511 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1202288f2442477badeb35667dd08dab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.708385 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 2e7cf4fc3be8446292b0c91141eeadcc (DEFAULT_TABLE table=default.Table_4b4d4f2b2a2d40b5b717b010b99c5c26 [id=cab546ca09ce4671a31dd9d260155528]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.708652 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e7cf4fc3be8446292b0c91141eeadcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.713240 14167 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_b900c4395de448a08b0285a5807f2fdf [id=adfd7e86aa0f48e5b5e36b4113517dba] alter complete (version 1)
I20250115 03:06:17.715775 14396 tablet_service.cc:1467] Processing CreateTablet for tablet b3334f58b90b4947af52150546ccaa28 (DEFAULT_TABLE table=default.Table_d58c19b62a194b46858ee8955ebf3f82 [id=6cba4fb2f7104c6e953340b04e5dc494]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.716001 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3334f58b90b4947af52150546ccaa28. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.718166 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3bf849316a4ebd841c3345ca17087e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.718330 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3bf849316a4ebd841c3345ca17087e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.719185 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 13ce129d8a9245baab7c772f0f5fbaa7 (table default.Table_d58c19b62a194b46858ee8955ebf3f82 [id=6cba4fb2f7104c6e953340b04e5dc494]) successfully deleted
W20250115 03:06:17.719305 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet fe8d94c8fecc4077ba052390810c7430 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe8d94c8fecc4077ba052390810c7430 already in progress: creating tablet
I20250115 03:06:17.719808 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 60997f625d074960ad75aef5ba14b9c2 (DEFAULT_TABLE table=default.Table_4db09fb126344de28b8a6b6baa54183a [id=bace8b6ea7894ccb96e0e31954917604]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.720038 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60997f625d074960ad75aef5ba14b9c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.718255 14669 tablet_service.cc:1467] Processing CreateTablet for tablet b3334f58b90b4947af52150546ccaa28 (DEFAULT_TABLE table=default.Table_d58c19b62a194b46858ee8955ebf3f82 [id=6cba4fb2f7104c6e953340b04e5dc494]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.720512 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3334f58b90b4947af52150546ccaa28. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.721704 14809 tablet_bootstrap.cc:492] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.721809 14809 ts_tablet_manager.cc:1397] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.000s
I20250115 03:06:17.721822 14820 ts_tablet_manager.cc:1939] T 33380937a3b7428995b06e477a896c02 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:17.722146 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 33380937a3b7428995b06e477a896c02 (table default.Table_90aedfcdb1274e1497cf36620b980768 [id=9e4f5de45eb24beabef550d15dcad8bf]) successfully deleted
I20250115 03:06:17.722344 14820 tablet_replica.cc:331] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:17.722299 14809 raft_consensus.cc:357] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.722466 14809 raft_consensus.cc:383] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.722481 14820 raft_consensus.cc:2238] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.722574 14820 raft_consensus.cc:2267] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.722803 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_7ae87c57c8ac4747bf90d2a1f4c9f9d1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:17.723114 14811 tablet_bootstrap.cc:492] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
W20250115 03:06:17.723101 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7ae87c57c8ac4747bf90d2a1f4c9f9d1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.723186 14811 ts_tablet_manager.cc:1397] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.082s user 0.001s sys 0.000s
I20250115 03:06:17.723331 14809 raft_consensus.cc:738] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.723536 14809 consensus_queue.cc:260] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.723541 14811 raft_consensus.cc:357] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.723625 14811 raft_consensus.cc:383] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.723660 14811 raft_consensus.cc:738] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.723764 14811 consensus_queue.cc:260] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.723992 14811 ts_tablet_manager.cc:1428] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.724135 14811 tablet_bootstrap.cc:492] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:17.725207 14811 tablet_bootstrap.cc:654] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.725417 14167 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5780da67ce984297ba33d3c6901c9e3a (table default.Table_551ca9fefcb74ed5aa29803f19de6ab6 [id=79eb8c9694b74a31a4253b33d064e230]). Expected version 1 got 0
W20250115 03:06:17.726438 14356 consensus_peers.cc:487] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:17.728159 14809 ts_tablet_manager.cc:1428] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:17.730634 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_bc39bcc6dcea45ec9bdac3e3e6926460"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:17.729504 14809 tablet_bootstrap.cc:492] T 3007a0506ce14c6c89c600b98a81b168 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
W20250115 03:06:17.730931 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc39bcc6dcea45ec9bdac3e3e6926460 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.731979 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_204aaf913d224b93adf8ce86999572c7" } new_table_name: "default.Table_a8a91cdcd75a40dea727b3c6890804c3" modify_external_catalogs: true
I20250115 03:06:17.732378 14811 tablet_bootstrap.cc:492] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:17.732456 14811 ts_tablet_manager.cc:1397] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20250115 03:06:17.733144 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_35de20a1cb6a488bbc86997188d7ce82" } new_table_name: "default.Table_867788c99c794aadbb2dc125669537dc" modify_external_catalogs: true
I20250115 03:06:17.733404 14811 raft_consensus.cc:357] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.733505 14811 raft_consensus.cc:383] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.733538 14811 raft_consensus.cc:738] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.733634 14811 consensus_queue.cc:260] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.733855 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_807e7c2704994c25aa03b1267a20f794" } modify_external_catalogs: true
I20250115 03:06:17.733937 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_807e7c2704994c25aa03b1267a20f794" } modify_external_catalogs: true
I20250115 03:06:17.735064 14811 ts_tablet_manager.cc:1428] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:17.735272 14811 tablet_bootstrap.cc:492] T 13f36a563f13403eab372deb10b67451 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:17.736188 14523 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d10731df02f4e9682b8de6962256635 with delete_type TABLET_DATA_DELETED (replaced by tablet 5780da67ce984297ba33d3c6901c9e3a) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.737483 14811 tablet_bootstrap.cc:654] T 13f36a563f13403eab372deb10b67451 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:17.736755 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 0f3bf849316a4ebd841c3345ca17087e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0f3bf849316a4ebd841c3345ca17087e already in progress: creating tablet
I20250115 03:06:17.738062 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e46e737b4c074b7a92903cf0f6a786ca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.738329 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e46e737b4c074b7a92903cf0f6a786ca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.738898 14665 tablet_service.cc:1467] Processing CreateTablet for tablet bf6199cdd6e940d1970dd43d8090100d (DEFAULT_TABLE table=default.Table_bf4a2c34cc7d42acabda873e7d47d3fb [id=0f03d804c7114afcb8d40e7e00025ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.739039 14664 tablet_service.cc:1467] Processing CreateTablet for tablet d643bd0947c54bcfa79f15d88e1e53e4 (DEFAULT_TABLE table=default.Table_c1f030f5a1ef49a8826714b1b730459a [id=570edeb30f8841ab8c07f4e78353990f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.739105 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf6199cdd6e940d1970dd43d8090100d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.739253 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d643bd0947c54bcfa79f15d88e1e53e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.739310 15390 raft_consensus.cc:491] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.739336 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b9c09b6315345039cdea9eeee3f181a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.739382 15390 raft_consensus.cc:513] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.739521 14397 tablet_service.cc:1467] Processing CreateTablet for tablet bf6199cdd6e940d1970dd43d8090100d (DEFAULT_TABLE table=default.Table_bf4a2c34cc7d42acabda873e7d47d3fb [id=0f03d804c7114afcb8d40e7e00025ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.739704 15390 leader_election.cc:290] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.739709 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf6199cdd6e940d1970dd43d8090100d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.739995 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5780da67ce984297ba33d3c6901c9e3a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:17.740429 14356 leader_election.cc:343] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.740691 14809 tablet_bootstrap.cc:654] T 3007a0506ce14c6c89c600b98a81b168 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.741796 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5780da67ce984297ba33d3c6901c9e3a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:17.741901 14687 raft_consensus.cc:2463] T 5780da67ce984297ba33d3c6901c9e3a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:17.742192 15325 raft_consensus.cc:491] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.742256 15325 raft_consensus.cc:513] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.739038 15970 tablet.cc:1721] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.742740 14396 tablet_service.cc:1467] Processing CreateTablet for tablet d643bd0947c54bcfa79f15d88e1e53e4 (DEFAULT_TABLE table=default.Table_c1f030f5a1ef49a8826714b1b730459a [id=570edeb30f8841ab8c07f4e78353990f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.742528 15325 leader_election.cc:290] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:17.742944 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d643bd0947c54bcfa79f15d88e1e53e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.743472 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1b8b5b7fe03496ea2639dbc9c920974" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:17.743873 14358 leader_election.cc:304] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
W20250115 03:06:17.744072 14626 leader_election.cc:343] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.744423 15390 raft_consensus.cc:2798] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:17.744486 15390 raft_consensus.cc:491] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:17.744524 15390 raft_consensus.cc:3054] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.745411 15390 raft_consensus.cc:513] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:17.745617 15834 log.cc:926] Time spent T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246: Append to log took a long time: real 0.053s user 0.003s sys 0.000s
I20250115 03:06:17.745721 15390 leader_election.cc:290] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.747409 15325 raft_consensus.cc:491] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.747475 15325 raft_consensus.cc:513] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.747743 15325 leader_election.cc:290] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:17.748019 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 60997f625d074960ad75aef5ba14b9c2 (DEFAULT_TABLE table=default.Table_4db09fb126344de28b8a6b6baa54183a [id=bace8b6ea7894ccb96e0e31954917604]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.748198 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60997f625d074960ad75aef5ba14b9c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.749300 14523 tablet_service.cc:1467] Processing CreateTablet for tablet 980b69b60a354519b0691ecb0bbd6c70 (DEFAULT_TABLE table=default.Table_ea8049593bb4406883df33d6635d42e5 [id=e13eae5b128b46a3bba1954f3b1f56c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.749516 14523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 980b69b60a354519b0691ecb0bbd6c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.751205 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b9c09b6315345039cdea9eeee3f181a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.751775 14811 tablet_bootstrap.cc:492] T 13f36a563f13403eab372deb10b67451 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:17.751850 14811 ts_tablet_manager.cc:1397] T 13f36a563f13403eab372deb10b67451 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
W20250115 03:06:17.751997 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet fe8d94c8fecc4077ba052390810c7430 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe8d94c8fecc4077ba052390810c7430 already in progress: creating tablet
I20250115 03:06:17.752144 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6784e353df554bb2aa10b667c8b488da" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:17.752256 14811 raft_consensus.cc:357] T 13f36a563f13403eab372deb10b67451 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.752348 14811 raft_consensus.cc:383] T 13f36a563f13403eab372deb10b67451 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.752398 14811 raft_consensus.cc:738] T 13f36a563f13403eab372deb10b67451 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
W20250115 03:06:17.752473 14626 leader_election.cc:343] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.752509 14811 consensus_queue.cc:260] T 13f36a563f13403eab372deb10b67451 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.752647 14812 tablet_bootstrap.cc:492] T 58130f2b27b745d08c0002ac96af6d72 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.752723 14812 ts_tablet_manager.cc:1397] T 58130f2b27b745d08c0002ac96af6d72 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.115s user 0.000s sys 0.001s
I20250115 03:06:17.752758 14811 ts_tablet_manager.cc:1428] T 13f36a563f13403eab372deb10b67451 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.752913 14811 tablet_bootstrap.cc:492] T dcdb5282e41c43dbb235ae78465d19f9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:17.753145 14812 raft_consensus.cc:357] T 58130f2b27b745d08c0002ac96af6d72 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.753233 14812 raft_consensus.cc:383] T 58130f2b27b745d08c0002ac96af6d72 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.753262 14812 raft_consensus.cc:738] T 58130f2b27b745d08c0002ac96af6d72 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.753369 14812 consensus_queue.cc:260] T 58130f2b27b745d08c0002ac96af6d72 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.753351 14809 tablet_bootstrap.cc:492] T 3007a0506ce14c6c89c600b98a81b168 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.753504 14809 ts_tablet_manager.cc:1397] T 3007a0506ce14c6c89c600b98a81b168 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.001s
I20250115 03:06:17.753600 14812 ts_tablet_manager.cc:1428] T 58130f2b27b745d08c0002ac96af6d72 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.753772 14812 tablet_bootstrap.cc:492] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:17.753770 14811 tablet_bootstrap.cc:654] T dcdb5282e41c43dbb235ae78465d19f9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.754144 14809 raft_consensus.cc:357] T 3007a0506ce14c6c89c600b98a81b168 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:17.754287 14809 raft_consensus.cc:383] T 3007a0506ce14c6c89c600b98a81b168 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.754354 14809 raft_consensus.cc:738] T 3007a0506ce14c6c89c600b98a81b168 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.754617 14812 tablet_bootstrap.cc:654] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.754583 14809 consensus_queue.cc:260] T 3007a0506ce14c6c89c600b98a81b168 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:17.754799 15401 raft_consensus.cc:491] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.754868 15401 raft_consensus.cc:513] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.755154 15401 leader_election.cc:290] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.755177 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5780da67ce984297ba33d3c6901c9e3a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:17.755213 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 398b93f1db0741a9b54837bca43b1bad with delete_type TABLET_DATA_DELETED (replaced by tablet c1a81dda6ac44432927fa2c90b3e0002) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.755262 14687 raft_consensus.cc:3054] T 5780da67ce984297ba33d3c6901c9e3a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.755968 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6175f691fec45bf9853ee90959a1c98" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:17.756068 14809 ts_tablet_manager.cc:1428] T 3007a0506ce14c6c89c600b98a81b168 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:17.756070 14687 raft_consensus.cc:2463] T 5780da67ce984297ba33d3c6901c9e3a P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:17.756300 14809 tablet_bootstrap.cc:492] T 30499e99ab7d4138a376dea40b768c32 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.756398 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5780da67ce984297ba33d3c6901c9e3a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:17.757064 14356 leader_election.cc:343] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:17.757189 14356 leader_election.cc:343] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.757797 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1b8b5b7fe03496ea2639dbc9c920974" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:17.757872 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6784e353df554bb2aa10b667c8b488da" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:17.757980 14421 raft_consensus.cc:2463] T 6784e353df554bb2aa10b667c8b488da P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:17.757896 14420 raft_consensus.cc:2463] T a1b8b5b7fe03496ea2639dbc9c920974 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:17.758937 14358 leader_election.cc:304] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.759045 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6175f691fec45bf9853ee90959a1c98" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:17.759124 14687 raft_consensus.cc:2463] T b6175f691fec45bf9853ee90959a1c98 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:17.759128 15401 raft_consensus.cc:2798] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:17.759198 15401 raft_consensus.cc:695] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:17.759299 15401 consensus_queue.cc:237] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.759786 14809 tablet_bootstrap.cc:654] T 30499e99ab7d4138a376dea40b768c32 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.761415 14812 tablet_bootstrap.cc:492] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.761483 14812 ts_tablet_manager.cc:1397] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20250115 03:06:17.761853 14812 raft_consensus.cc:357] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:17.761931 14812 raft_consensus.cc:383] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.761957 14812 raft_consensus.cc:738] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.762069 14812 consensus_queue.cc:260] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:17.762356 14812 ts_tablet_manager.cc:1428] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.762490 14812 tablet_bootstrap.cc:492] T aa7754f577af4ef6b6fca41e61e5965c P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:17.762727 14358 leader_election.cc:304] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.762907 14861 consensus_queue.cc:1035] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.763108 14861 consensus_queue.cc:1035] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.763265 14812 tablet_bootstrap.cc:654] T aa7754f577af4ef6b6fca41e61e5965c P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.763361 14861 raft_consensus.cc:2798] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:17.763406 14861 raft_consensus.cc:491] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:17.763438 14861 raft_consensus.cc:3054] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.764283 14861 raft_consensus.cc:513] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.764592 14861 leader_election.cc:290] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.765051 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6175f691fec45bf9853ee90959a1c98" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:17.766357 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 93fa227c0de74a3fa86778641fdcda3d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.766502 14819 tablet_replica.cc:331] T 93fa227c0de74a3fa86778641fdcda3d P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.766520 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 398b93f1db0741a9b54837bca43b1bad with delete_type TABLET_DATA_DELETED (replaced by tablet c1a81dda6ac44432927fa2c90b3e0002) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:17.766536 14356 leader_election.cc:343] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.766615 14819 raft_consensus.cc:2238] T 93fa227c0de74a3fa86778641fdcda3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.766695 14819 raft_consensus.cc:2267] T 93fa227c0de74a3fa86778641fdcda3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.766767 14628 leader_election.cc:304] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.766986 14628 leader_election.cc:304] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.767076 14819 ts_tablet_manager.cc:1905] T 93fa227c0de74a3fa86778641fdcda3d P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.767179 15153 raft_consensus.cc:2798] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:17.767225 15153 raft_consensus.cc:491] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:17.767259 15153 raft_consensus.cc:3054] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.768060 15153 raft_consensus.cc:513] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.768363 15153 leader_election.cc:290] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:17.768450 15153 raft_consensus.cc:2798] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:17.768494 15153 raft_consensus.cc:491] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:17.768527 15153 raft_consensus.cc:3054] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.769266 15153 raft_consensus.cc:513] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.769557 14819 ts_tablet_manager.cc:1918] T 93fa227c0de74a3fa86778641fdcda3d P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.769567 15153 leader_election.cc:290] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:17.769618 14819 log.cc:1198] T 93fa227c0de74a3fa86778641fdcda3d P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/93fa227c0de74a3fa86778641fdcda3d
I20250115 03:06:17.769955 14819 ts_tablet_manager.cc:1939] T 93fa227c0de74a3fa86778641fdcda3d P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.770054 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1b8b5b7fe03496ea2639dbc9c920974" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:17.770359 14626 leader_election.cc:343] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.770552 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6784e353df554bb2aa10b667c8b488da" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:17.770848 14626 leader_election.cc:343] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.772181 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_4b64655665164e00b508a19e84b9991f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.772519 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4b64655665164e00b508a19e84b9991f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.772182 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2653ad06e063469cbfda9ba1bada2951"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.773214 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2653ad06e063469cbfda9ba1bada2951 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.774173 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_04a5f895839e461eaa6017426eca96de"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.774416 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_04a5f895839e461eaa6017426eca96de in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.774863 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6784e353df554bb2aa10b667c8b488da" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:17.774955 14420 raft_consensus.cc:3054] T 6784e353df554bb2aa10b667c8b488da P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.775715 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6175f691fec45bf9853ee90959a1c98" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:17.775794 14420 raft_consensus.cc:2463] T 6784e353df554bb2aa10b667c8b488da P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:17.775817 14689 raft_consensus.cc:3054] T b6175f691fec45bf9853ee90959a1c98 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.775971 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1b8b5b7fe03496ea2639dbc9c920974" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:17.776059 14421 raft_consensus.cc:3054] T a1b8b5b7fe03496ea2639dbc9c920974 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.776603 14689 raft_consensus.cc:2463] T b6175f691fec45bf9853ee90959a1c98 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:17.776863 14421 raft_consensus.cc:2463] T a1b8b5b7fe03496ea2639dbc9c920974 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:17.777289 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 13f36a563f13403eab372deb10b67451 (table default.Table_70d0b700eac34892b4ef769d739481fd [id=61f3625fed2b4a70b5331d6c6ac73f1c]). Expected version 1 got 0
I20250115 03:06:17.777953 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_8ac3b54226f54715849ee4eb26e7a42f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:17.778136 14738 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
W20250115 03:06:17.778208 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8ac3b54226f54715849ee4eb26e7a42f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.780822 15325 raft_consensus.cc:491] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.780893 15325 raft_consensus.cc:513] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:17.782879 15325 leader_election.cc:290] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:17.783294 14628 leader_election.cc:304] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.783502 14628 leader_election.cc:304] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.783566 14853 raft_consensus.cc:491] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.783553 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3007a0506ce14c6c89c600b98a81b168" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:17.783648 14421 raft_consensus.cc:2463] T 3007a0506ce14c6c89c600b98a81b168 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:17.783635 14853 raft_consensus.cc:513] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.783862 15325 raft_consensus.cc:2798] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:17.783937 15325 raft_consensus.cc:695] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:17.783948 14853 leader_election.cc:290] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.784039 15453 raft_consensus.cc:2798] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:17.784052 15325 consensus_queue.cc:237] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.784178 15453 raft_consensus.cc:695] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:17.784276 15453 consensus_queue.cc:237] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:17.785995 14493 leader_election.cc:343] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 054251151c294e06905092f2a9306849
I20250115 03:06:17.786268 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3007a0506ce14c6c89c600b98a81b168" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:17.786556 14626 leader_election.cc:343] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:17.787438 14493 leader_election.cc:343] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 054251151c294e06905092f2a9306849
I20250115 03:06:17.787511 14493 leader_election.cc:304] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.787678 14853 raft_consensus.cc:2743] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.787951 14358 leader_election.cc:304] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.788314 14885 raft_consensus.cc:2798] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:17.788381 14885 raft_consensus.cc:695] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:17.788508 14885 consensus_queue.cc:237] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:17.790879 14356 consensus_peers.cc:487] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:17.791496 14628 leader_election.cc:304] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.791975 15153 raft_consensus.cc:2798] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:17.792028 15153 raft_consensus.cc:491] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:17.792062 15153 raft_consensus.cc:3054] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.792860 15153 raft_consensus.cc:513] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:17.793169 15153 leader_election.cc:290] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:17.793745 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3007a0506ce14c6c89c600b98a81b168" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:17.794073 14626 leader_election.cc:343] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.794512 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3007a0506ce14c6c89c600b98a81b168" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:17.794607 14421 raft_consensus.cc:3054] T 3007a0506ce14c6c89c600b98a81b168 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.794900 15390 consensus_queue.cc:1035] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.795348 14421 raft_consensus.cc:2463] T 3007a0506ce14c6c89c600b98a81b168 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:17.795831 14628 leader_election.cc:304] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.796023 15453 raft_consensus.cc:2798] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:17.796101 15453 raft_consensus.cc:695] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:17.796198 15453 consensus_queue.cc:237] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
W20250115 03:06:17.801115 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 0f3bf849316a4ebd841c3345ca17087e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0f3bf849316a4ebd841c3345ca17087e already in progress: creating tablet
W20250115 03:06:17.801476 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 3b9c09b6315345039cdea9eeee3f181a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b9c09b6315345039cdea9eeee3f181a already in progress: creating tablet
I20250115 03:06:17.801908 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 93fa227c0de74a3fa86778641fdcda3d (table default.Table_4f241c86102a4bbfb509a2d5d76953e9 [id=03b92c08d0b0473d83090c58e450c84b]) successfully deleted
I20250115 03:06:17.805220 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet d2220e7ec26b4708b2c7a2540c053dcf
W20250115 03:06:17.805611 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dda418ed197e4af2aa511d3f1851663e consensus state has no leader
W20250115 03:06:17.806044 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6132d15b17ab4d0aaa10a7ea04cda7ac consensus state has no leader
I20250115 03:06:17.807229 14523 tablet_service.cc:1514] Processing DeleteTablet for tablet d2220e7ec26b4708b2c7a2540c053dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.808454 15401 raft_consensus.cc:491] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.808545 15401 raft_consensus.cc:513] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.808862 15401 leader_election.cc:290] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.809063 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ec5f1644f3f6456f8d0d5ee407d7e7cd" } new_table_name: "default.Table_1ed9a3074bb7436797358399890084b7" modify_external_catalogs: true
I20250115 03:06:17.809491 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8efd181c42eb42628a9f36fad4fa8d64" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:17.809814 14356 leader_election.cc:343] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.813652 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8efd181c42eb42628a9f36fad4fa8d64" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:17.813776 14692 raft_consensus.cc:2463] T 8efd181c42eb42628a9f36fad4fa8d64 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:17.814002 14358 leader_election.cc:304] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.814190 15390 raft_consensus.cc:2798] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:17.814235 15390 raft_consensus.cc:491] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:17.814270 15390 raft_consensus.cc:3054] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.815151 15390 raft_consensus.cc:513] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.815471 15390 leader_election.cc:290] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.817006 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8efd181c42eb42628a9f36fad4fa8d64" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:17.817333 14356 leader_election.cc:343] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.817762 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8efd181c42eb42628a9f36fad4fa8d64" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:17.817862 14692 raft_consensus.cc:3054] T 8efd181c42eb42628a9f36fad4fa8d64 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.817929 14820 ts_tablet_manager.cc:1905] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.818642 14692 raft_consensus.cc:2463] T 8efd181c42eb42628a9f36fad4fa8d64 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:17.818914 14358 leader_election.cc:304] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
W20250115 03:06:17.819113 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cff95c69fb9045cf9ec0ebb128c9ef6b consensus state has no leader
I20250115 03:06:17.819154 14853 raft_consensus.cc:491] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.819221 14853 raft_consensus.cc:513] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.819514 14853 leader_election.cc:290] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.819648 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_511db1d50b7246e9b6dc2d6ed50da4d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.819921 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_511db1d50b7246e9b6dc2d6ed50da4d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:17.820086 14493 leader_election.cc:343] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 5986558350244e7aa4ddcd5cccb7d676
W20250115 03:06:17.820778 14493 leader_election.cc:343] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 5986558350244e7aa4ddcd5cccb7d676
I20250115 03:06:17.820840 14493 leader_election.cc:304] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.821291 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f3b64ddeafb3400583229c6698c2a5e5" } modify_external_catalogs: true
I20250115 03:06:17.821374 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f3b64ddeafb3400583229c6698c2a5e5" } modify_external_catalogs: true
I20250115 03:06:17.821571 14853 raft_consensus.cc:2743] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.821602 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_dca8868e4ea14ebeaefe647e2188876c" } new_table_name: "default.Table_9f9940dec82b49098561c9a2e90b3475" modify_external_catalogs: true
I20250115 03:06:17.821880 15401 raft_consensus.cc:2798] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:17.821957 15401 raft_consensus.cc:695] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:17.822060 15401 consensus_queue.cc:237] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.822844 14166 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 58130f2b27b745d08c0002ac96af6d72 (table default.Table_67f9f825d77c4874bb1d4a5c17dedde2 [id=6aa86d8861174b9091f29b0cfd39fa55]). Expected version 1 got 0
I20250115 03:06:17.824014 14523 tablet_service.cc:1467] Processing CreateTablet for tablet fdaf7c090ef642b8b339f61a5d3fb06e (DEFAULT_TABLE table=default.Table_bc39bcc6dcea45ec9bdac3e3e6926460 [id=698085e765e74439a7ec71523be0b499]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.824229 14523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdaf7c090ef642b8b339f61a5d3fb06e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.825258 14520 tablet_service.cc:1467] Processing CreateTablet for tablet 9bb0b7a8acfd40738abcc951936b1116 (DEFAULT_TABLE table=default.Table_7ae87c57c8ac4747bf90d2a1f4c9f9d1 [id=77a560c93816435894621976eaaf0d05]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.825464 14520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bb0b7a8acfd40738abcc951936b1116. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.827828 14166 catalog_manager.cc:5526] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:17.828060 14166 catalog_manager.cc:5526] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:17.828249 14166 catalog_manager.cc:5526] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:17.828440 14166 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet dcdb5282e41c43dbb235ae78465d19f9 (table default.Table_375bb4e2095c4675b734247fabacbec7 [id=b298dce7ab664327893633f821ed4d89]). Expected version 1 got 0
I20250115 03:06:17.828891 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_9c49fab0c4db42e1b04b2703b0f3851a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.829023 15834 log.cc:926] Time spent T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246: Append to log took a long time: real 0.060s user 0.001s sys 0.000s
W20250115 03:06:17.829177 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c49fab0c4db42e1b04b2703b0f3851a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.833120 14853 raft_consensus.cc:491] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.833196 14853 raft_consensus.cc:513] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.833500 14853 leader_election.cc:290] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:17.834167 14493 leader_election.cc:343] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 6d10731df02f4e9682b8de6962256635
I20250115 03:06:17.834398 15390 raft_consensus.cc:491] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.834473 15390 raft_consensus.cc:513] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.834879 15390 leader_election.cc:290] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:17.835232 14493 leader_election.cc:343] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 6d10731df02f4e9682b8de6962256635
I20250115 03:06:17.835299 14493 leader_election.cc:304] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.835474 14853 raft_consensus.cc:2743] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.835752 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58130f2b27b745d08c0002ac96af6d72" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:17.836040 14356 leader_election.cc:343] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.836305 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58130f2b27b745d08c0002ac96af6d72" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:17.836414 14692 raft_consensus.cc:2463] T 58130f2b27b745d08c0002ac96af6d72 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:17.836668 14358 leader_election.cc:304] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:17.838716 14885 raft_consensus.cc:2798] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:17.838783 14885 raft_consensus.cc:491] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:17.838821 14885 raft_consensus.cc:3054] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.839740 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_cbfbacbaeee4488aba1a8be104138404" } new_table_name: "default.Table_c9a7e51bb4ca41eba7b6a942eb649b81" modify_external_catalogs: true
I20250115 03:06:17.839715 14885 raft_consensus.cc:513] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.839818 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 980b69b60a354519b0691ecb0bbd6c70 (DEFAULT_TABLE table=default.Table_ea8049593bb4406883df33d6635d42e5 [id=e13eae5b128b46a3bba1954f3b1f56c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.840039 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 980b69b60a354519b0691ecb0bbd6c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.840045 14885 leader_election.cc:290] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:17.840595 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 3b9c09b6315345039cdea9eeee3f181a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b9c09b6315345039cdea9eeee3f181a already in progress: creating tablet
I20250115 03:06:17.840967 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58130f2b27b745d08c0002ac96af6d72" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:17.841058 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet d2220e7ec26b4708b2c7a2540c053dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:17.841290 14356 leader_election.cc:343] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.841401 14669 tablet_service.cc:1514] Processing DeleteTablet for tablet d2220e7ec26b4708b2c7a2540c053dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:17.841719 14356 consensus_peers.cc:487] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:17.841956 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet f70a69acbdc049beb9d98eada09d9eb8 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:17.842873 14397 tablet_service.cc:1467] Processing CreateTablet for tablet fdaf7c090ef642b8b339f61a5d3fb06e (DEFAULT_TABLE table=default.Table_bc39bcc6dcea45ec9bdac3e3e6926460 [id=698085e765e74439a7ec71523be0b499]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.842873 14665 tablet_service.cc:1467] Processing CreateTablet for tablet fdaf7c090ef642b8b339f61a5d3fb06e (DEFAULT_TABLE table=default.Table_bc39bcc6dcea45ec9bdac3e3e6926460 [id=698085e765e74439a7ec71523be0b499]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.843079 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdaf7c090ef642b8b339f61a5d3fb06e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.843079 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fdaf7c090ef642b8b339f61a5d3fb06e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.845288 14170 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 58130f2b27b745d08c0002ac96af6d72 (table default.Table_67f9f825d77c4874bb1d4a5c17dedde2 [id=6aa86d8861174b9091f29b0cfd39fa55]). Expected version 1 got 0
I20250115 03:06:17.845453 14170 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet a1c93bfd3da44f9f8a8d585f2d21f1be (table default.Table_40b1953a4a364eb1ae58e3387fc65a91 [id=64d20b1792044c5e8add735aaf75a6d9]). Expected version 1 got 0
I20250115 03:06:17.845819 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet a3eaec2fff184aad816349fc36fa3db1
I20250115 03:06:17.845943 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58130f2b27b745d08c0002ac96af6d72" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:17.846050 14692 raft_consensus.cc:3054] T 58130f2b27b745d08c0002ac96af6d72 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.846640 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 980b69b60a354519b0691ecb0bbd6c70 (DEFAULT_TABLE table=default.Table_ea8049593bb4406883df33d6635d42e5 [id=e13eae5b128b46a3bba1954f3b1f56c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.846848 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 980b69b60a354519b0691ecb0bbd6c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.846850 14692 raft_consensus.cc:2463] T 58130f2b27b745d08c0002ac96af6d72 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:17.847107 14358 leader_election.cc:304] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
W20250115 03:06:17.847316 14162 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 87b234185fc64505b097f2ae7d52d813 consensus state has no leader
I20250115 03:06:17.847877 14523 tablet_service.cc:1514] Processing DeleteTablet for tablet a3eaec2fff184aad816349fc36fa3db1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.848047 15390 raft_consensus.cc:2798] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:17.848160 15390 raft_consensus.cc:695] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:17.848302 15390 consensus_queue.cc:237] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.849515 14812 tablet_bootstrap.cc:492] T aa7754f577af4ef6b6fca41e61e5965c P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.849601 14812 ts_tablet_manager.cc:1397] T aa7754f577af4ef6b6fca41e61e5965c P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.087s user 0.002s sys 0.000s
I20250115 03:06:17.849999 14812 raft_consensus.cc:357] T aa7754f577af4ef6b6fca41e61e5965c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:17.850088 14812 raft_consensus.cc:383] T aa7754f577af4ef6b6fca41e61e5965c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.850118 14812 raft_consensus.cc:738] T aa7754f577af4ef6b6fca41e61e5965c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.850217 14812 consensus_queue.cc:260] T aa7754f577af4ef6b6fca41e61e5965c P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:17.850471 14812 ts_tablet_manager.cc:1428] T aa7754f577af4ef6b6fca41e61e5965c P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.851680 14812 tablet_bootstrap.cc:492] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:17.852591 14812 tablet_bootstrap.cc:654] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.854719 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet 103f78d0d9fc42308e1e23316895be33 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:17.857213 14820 ts_tablet_manager.cc:1918] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.857338 14820 log.cc:1198] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/f8b90f854a5e4221a24986f8c71ca300
I20250115 03:06:17.857390 14468 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:17.857599 14820 ts_tablet_manager.cc:1939] T f8b90f854a5e4221a24986f8c71ca300 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:17.857818 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 9bb0b7a8acfd40738abcc951936b1116 (DEFAULT_TABLE table=default.Table_7ae87c57c8ac4747bf90d2a1f4c9f9d1 [id=77a560c93816435894621976eaaf0d05]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.858081 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet f8b90f854a5e4221a24986f8c71ca300 (table default.Table_4e74d34a621f4b1898241f62a20ffc10 [id=d8084de64a264d28a3370971617a694c]) successfully deleted
I20250115 03:06:17.858201 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bb0b7a8acfd40738abcc951936b1116. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.858474 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 9bb0b7a8acfd40738abcc951936b1116 (DEFAULT_TABLE table=default.Table_7ae87c57c8ac4747bf90d2a1f4c9f9d1 [id=77a560c93816435894621976eaaf0d05]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.858759 14820 tablet_replica.cc:331] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:17.859148 14820 raft_consensus.cc:2238] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.859411 14820 raft_consensus.cc:2267] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.859076 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bb0b7a8acfd40738abcc951936b1116. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.859830 14820 ts_tablet_manager.cc:1905] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.861672 14820 ts_tablet_manager.cc:1918] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20250115 03:06:17.863233 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet d2220e7ec26b4708b2c7a2540c053dcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d2220e7ec26b4708b2c7a2540c053dcf already in progress: creating tablet
I20250115 03:06:17.863876 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet a3eaec2fff184aad816349fc36fa3db1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.864084 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet a3eaec2fff184aad816349fc36fa3db1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.870173 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d8f964febee84681a352ac34900c1e6c" } modify_external_catalogs: true
I20250115 03:06:17.870313 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d8f964febee84681a352ac34900c1e6c" } modify_external_catalogs: true
I20250115 03:06:17.874943 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_18deb1c4560441b8abc82d6709285a34"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.875252 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_18deb1c4560441b8abc82d6709285a34 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:17.876063 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet d2220e7ec26b4708b2c7a2540c053dcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d2220e7ec26b4708b2c7a2540c053dcf already in progress: creating tablet
I20250115 03:06:17.877432 14170 master_service.cc:946] ReplaceTablet: received request to replace tablet a73d45bf459c4137af6961b680d282ac from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:17.877770 15617 tablet.cc:1721] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.879865 15325 consensus_queue.cc:1035] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.880363 14168 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet dcdb5282e41c43dbb235ae78465d19f9 (table default.Table_375bb4e2095c4675b734247fabacbec7 [id=b298dce7ab664327893633f821ed4d89]). Expected version 1 got 0
W20250115 03:06:17.880507 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet a3eaec2fff184aad816349fc36fa3db1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a3eaec2fff184aad816349fc36fa3db1 already in progress: creating tablet
I20250115 03:06:17.881623 15970 tablet.cc:1721] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:17.882018 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 1c76ab180b26487896f4360efe47b2c7
I20250115 03:06:17.866497 14811 tablet_bootstrap.cc:492] T dcdb5282e41c43dbb235ae78465d19f9 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:17.883306 14811 ts_tablet_manager.cc:1397] T dcdb5282e41c43dbb235ae78465d19f9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.130s user 0.000s sys 0.002s
I20250115 03:06:17.883756 14811 raft_consensus.cc:357] T dcdb5282e41c43dbb235ae78465d19f9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.883853 14811 raft_consensus.cc:383] T dcdb5282e41c43dbb235ae78465d19f9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.883884 14811 raft_consensus.cc:738] T dcdb5282e41c43dbb235ae78465d19f9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.883988 14811 consensus_queue.cc:260] T dcdb5282e41c43dbb235ae78465d19f9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.884250 14811 ts_tablet_manager.cc:1428] T dcdb5282e41c43dbb235ae78465d19f9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:17.884421 14811 tablet_bootstrap.cc:492] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
W20250115 03:06:17.884510 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet a3eaec2fff184aad816349fc36fa3db1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a3eaec2fff184aad816349fc36fa3db1 already in progress: creating tablet
I20250115 03:06:17.885212 14812 tablet_bootstrap.cc:492] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.885280 14812 ts_tablet_manager.cc:1397] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.034s user 0.001s sys 0.000s
I20250115 03:06:17.885339 14811 tablet_bootstrap.cc:654] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.885666 14812 raft_consensus.cc:357] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:17.885766 14812 raft_consensus.cc:383] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.885792 14812 raft_consensus.cc:738] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.885910 14812 consensus_queue.cc:260] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:17.886334 14812 ts_tablet_manager.cc:1428] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:17.886991 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c76ab180b26487896f4360efe47b2c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.887148 14819 tablet_replica.cc:331] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.887130 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c76ab180b26487896f4360efe47b2c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.887257 14819 raft_consensus.cc:2238] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:17.887427 14819 raft_consensus.cc:2267] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.887445 14823 tablet_replica.cc:331] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:17.887552 14823 raft_consensus.cc:2238] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.887589 14523 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c76ab180b26487896f4360efe47b2c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.887634 14823 raft_consensus.cc:2267] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.887758 14819 ts_tablet_manager.cc:1905] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.887916 14823 ts_tablet_manager.cc:1905] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.889833 14819 ts_tablet_manager.cc:1918] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:17.889899 14819 log.cc:1198] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/1c76ab180b26487896f4360efe47b2c7
I20250115 03:06:17.889945 14823 ts_tablet_manager.cc:1918] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:17.890005 14823 log.cc:1198] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/1c76ab180b26487896f4360efe47b2c7
I20250115 03:06:17.890116 14819 ts_tablet_manager.cc:1939] T 1c76ab180b26487896f4360efe47b2c7 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.890230 14823 ts_tablet_manager.cc:1939] T 1c76ab180b26487896f4360efe47b2c7 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:17.890659 14166 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b175cba7965344979476c3cb6d789eca" } modify_external_catalogs: true
I20250115 03:06:17.890765 14166 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b175cba7965344979476c3cb6d789eca" } modify_external_catalogs: true
I20250115 03:06:17.891960 14171 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet dcdb5282e41c43dbb235ae78465d19f9 (table default.Table_375bb4e2095c4675b734247fabacbec7 [id=b298dce7ab664327893633f821ed4d89]). Expected version 1 got 0
I20250115 03:06:17.892421 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_48dcc3ce48c94335a128ff0ce47649ae" } modify_external_catalogs: true
I20250115 03:06:17.892536 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_48dcc3ce48c94335a128ff0ce47649ae" } modify_external_catalogs: true
I20250115 03:06:17.894119 14812 tablet_bootstrap.cc:492] T 30499e99ab7d4138a376dea40b768c32 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:17.894990 14812 tablet_bootstrap.cc:654] T 30499e99ab7d4138a376dea40b768c32 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.895888 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 1c76ab180b26487896f4360efe47b2c7 (table default.Table_d8f964febee84681a352ac34900c1e6c [id=dde72643e9ec41e290d4896e58742c70]) successfully deleted
I20250115 03:06:17.896046 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 1c76ab180b26487896f4360efe47b2c7 (table default.Table_d8f964febee84681a352ac34900c1e6c [id=dde72643e9ec41e290d4896e58742c70]) successfully deleted
I20250115 03:06:17.900847 14820 log.cc:1198] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/054251151c294e06905092f2a9306849
I20250115 03:06:17.901510 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet fa08518530ae4d5487af386a650c4672
I20250115 03:06:17.902688 15401 consensus_queue.cc:1035] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:17.902902 14523 tablet_service.cc:1514] Processing DeleteTablet for tablet fa08518530ae4d5487af386a650c4672 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.906121 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet fa08518530ae4d5487af386a650c4672 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.906224 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet fa08518530ae4d5487af386a650c4672 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.906423 14823 tablet_replica.cc:331] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:17.906387 14168 catalog_manager.cc:5526] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:17.906528 14823 raft_consensus.cc:2238] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.906580 14168 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 58130f2b27b745d08c0002ac96af6d72 (table default.Table_67f9f825d77c4874bb1d4a5c17dedde2 [id=6aa86d8861174b9091f29b0cfd39fa55]). Expected version 1 got 0
I20250115 03:06:17.906610 14823 raft_consensus.cc:2267] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.906699 14168 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet dcdb5282e41c43dbb235ae78465d19f9 (table default.Table_375bb4e2095c4675b734247fabacbec7 [id=b298dce7ab664327893633f821ed4d89]). Expected version 1 got 0
I20250115 03:06:17.906839 14819 tablet_replica.cc:331] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.906790 14168 catalog_manager.cc:5526] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:17.906951 14819 raft_consensus.cc:2238] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:17.906960 14823 ts_tablet_manager.cc:1905] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.907027 14168 catalog_manager.cc:5526] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:17.907109 14819 raft_consensus.cc:2267] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.907172 14168 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5780da67ce984297ba33d3c6901c9e3a (table default.Table_551ca9fefcb74ed5aa29803f19de6ab6 [id=79eb8c9694b74a31a4253b33d064e230]). Expected version 1 got 0
I20250115 03:06:17.907310 14168 catalog_manager.cc:5526] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:17.907433 14819 ts_tablet_manager.cc:1905] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.907932 14812 tablet_bootstrap.cc:492] T 30499e99ab7d4138a376dea40b768c32 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.908001 14812 ts_tablet_manager.cc:1397] T 30499e99ab7d4138a376dea40b768c32 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.001s
I20250115 03:06:17.908414 14812 raft_consensus.cc:357] T 30499e99ab7d4138a376dea40b768c32 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.908517 14812 raft_consensus.cc:383] T 30499e99ab7d4138a376dea40b768c32 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.908492 14523 tablet_service.cc:1467] Processing CreateTablet for tablet 820d5af6daa74cadb53d2a5229854a60 (DEFAULT_TABLE table=default.Table_2653ad06e063469cbfda9ba1bada2951 [id=8a3b8556dcae4ded8de4adcb19115d83]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.908551 14812 raft_consensus.cc:738] T 30499e99ab7d4138a376dea40b768c32 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.908659 14812 consensus_queue.cc:260] T 30499e99ab7d4138a376dea40b768c32 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.908819 15617 tablet.cc:1721] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:17.909013 14812 ts_tablet_manager.cc:1428] T 30499e99ab7d4138a376dea40b768c32 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.909168 14812 tablet_bootstrap.cc:492] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:17.909598 14819 ts_tablet_manager.cc:1918] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:17.909629 15970 tablet.cc:1721] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:17.909714 14819 log.cc:1198] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/fa08518530ae4d5487af386a650c4672
I20250115 03:06:17.910054 14819 ts_tablet_manager.cc:1939] T fa08518530ae4d5487af386a650c4672 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.910346 14812 tablet_bootstrap.cc:654] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.910876 14823 ts_tablet_manager.cc:1918] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:17.911005 14823 log.cc:1198] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/fa08518530ae4d5487af386a650c4672
I20250115 03:06:17.911327 14823 ts_tablet_manager.cc:1939] T fa08518530ae4d5487af386a650c4672 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:17.912415 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 103f78d0d9fc42308e1e23316895be33
I20250115 03:06:17.912657 14164 catalog_manager.cc:6596] ReplaceTablet: tablet 103f78d0d9fc42308e1e23316895be33 deleted and replaced by tablet 9c436975bd034068a6d58d5831db1896
I20250115 03:06:17.917140 14557 raft_consensus.cc:3054] T 13f36a563f13403eab372deb10b67451 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:17.917371 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet a73d45bf459c4137af6961b680d282ac
I20250115 03:06:17.917570 14170 catalog_manager.cc:6596] ReplaceTablet: tablet a73d45bf459c4137af6961b680d282ac deleted and replaced by tablet 15263a3b32be418288770ce348c082d7
I20250115 03:06:17.919859 15426 tablet.cc:1721] T 13f36a563f13403eab372deb10b67451 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20250115 03:06:17.920745 14626 consensus_peers.cc:487] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:17.921687 14812 tablet_bootstrap.cc:492] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.921860 14523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 820d5af6daa74cadb53d2a5229854a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.921880 14812 ts_tablet_manager.cc:1397] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
W20250115 03:06:17.924018 14356 consensus_peers.cc:487] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:17.924393 14812 raft_consensus.cc:357] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:17.924608 14812 raft_consensus.cc:383] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.924741 14812 raft_consensus.cc:738] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.924979 14812 consensus_queue.cc:260] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:17.925683 14812 ts_tablet_manager.cc:1428] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:17.925968 14812 tablet_bootstrap.cc:492] T d2220e7ec26b4708b2c7a2540c053dcf P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:17.926762 14858 raft_consensus.cc:491] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.926844 14858 raft_consensus.cc:513] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.927171 14858 leader_election.cc:290] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:17.927274 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet aa7754f577af4ef6b6fca41e61e5965c
I20250115 03:06:17.927376 14812 tablet_bootstrap.cc:654] T d2220e7ec26b4708b2c7a2540c053dcf P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.927739 14520 tablet_service.cc:1514] Processing DeleteTablet for tablet a73d45bf459c4137af6961b680d282ac with delete_type TABLET_DATA_DELETED (replaced by tablet 15263a3b32be418288770ce348c082d7) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:17.928412 14493 leader_election.cc:343] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 90277f7d279e4c7a83c43f40766cd11b
W20250115 03:06:17.928644 14493 leader_election.cc:343] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 90277f7d279e4c7a83c43f40766cd11b
I20250115 03:06:17.928699 14493 leader_election.cc:304] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.928859 14858 raft_consensus.cc:2743] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:17.929828 14523 tablet_service.cc:1514] Processing DeleteTablet for tablet 103f78d0d9fc42308e1e23316895be33 with delete_type TABLET_DATA_DELETED (replaced by tablet 9c436975bd034068a6d58d5831db1896) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.931350 14738 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:17.937340 14523 tablet_service.cc:1514] Processing DeleteTablet for tablet aa7754f577af4ef6b6fca41e61e5965c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.937352 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_562d28c67cc14d868fe4194eb7a4a05f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.937665 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_562d28c67cc14d868fe4194eb7a4a05f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.938392 14170 master_service.cc:946] ReplaceTablet: received request to replace tablet e43ce064a02347efa81ca274d76c6960 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:17.939790 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_a8dbae8770744751986252a5d31eb679"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:17.940075 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a8dbae8770744751986252a5d31eb679 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:17.940577 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 820d5af6daa74cadb53d2a5229854a60 (DEFAULT_TABLE table=default.Table_2653ad06e063469cbfda9ba1bada2951 [id=8a3b8556dcae4ded8de4adcb19115d83]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.940693 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 103f78d0d9fc42308e1e23316895be33 with delete_type TABLET_DATA_DELETED (replaced by tablet 9c436975bd034068a6d58d5831db1896) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.940693 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet 103f78d0d9fc42308e1e23316895be33 with delete_type TABLET_DATA_DELETED (replaced by tablet 9c436975bd034068a6d58d5831db1896) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.940146 14162 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_375bb4e2095c4675b734247fabacbec7 [id=b298dce7ab664327893633f821ed4d89] alter complete (version 1)
I20250115 03:06:17.940809 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 820d5af6daa74cadb53d2a5229854a60. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.940789 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet de14b1d525cc42808500279ae6b60e80 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:17.945106 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet a73d45bf459c4137af6961b680d282ac with delete_type TABLET_DATA_DELETED (replaced by tablet 15263a3b32be418288770ce348c082d7) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.945379 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet e43ce064a02347efa81ca274d76c6960
I20250115 03:06:17.945608 14170 catalog_manager.cc:6596] ReplaceTablet: tablet e43ce064a02347efa81ca274d76c6960 deleted and replaced by tablet b9f67c35591f4fdeb8a926a085171b49
I20250115 03:06:17.946091 14166 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 8b0f0539530643f78207e528c39530af (table default.Table_f7799772605a4284a062cd0e4ae15ed3 [id=fe357a2a64fc4531870b305eb2480df1]). Expected version 1 got 0
I20250115 03:06:17.946767 16001 raft_consensus.cc:491] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:17.946846 16001 raft_consensus.cc:513] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:17.947219 14520 tablet_service.cc:1514] Processing DeleteTablet for tablet e43ce064a02347efa81ca274d76c6960 with delete_type TABLET_DATA_DELETED (replaced by tablet b9f67c35591f4fdeb8a926a085171b49) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.947460 14518 tablet_service.cc:1467] Processing CreateTablet for tablet b0a7def7d9a7418381b2d5a93a9a9265 (DEFAULT_TABLE table=default.Table_04a5f895839e461eaa6017426eca96de [id=ef69fd1a3c5e49d0b73ac5da8551140c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.947690 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0a7def7d9a7418381b2d5a93a9a9265. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.948009 14812 tablet_bootstrap.cc:492] T d2220e7ec26b4708b2c7a2540c053dcf P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.948083 14812 ts_tablet_manager.cc:1397] T d2220e7ec26b4708b2c7a2540c053dcf P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s
I20250115 03:06:17.948467 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet fa08518530ae4d5487af386a650c4672 (table default.Table_48dcc3ce48c94335a128ff0ce47649ae [id=12b47ada2a2c48f2b6807fa62fcca1a6]) successfully deleted
I20250115 03:06:17.948506 14812 raft_consensus.cc:357] T d2220e7ec26b4708b2c7a2540c053dcf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.948602 14812 raft_consensus.cc:383] T d2220e7ec26b4708b2c7a2540c053dcf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.948637 14812 raft_consensus.cc:738] T d2220e7ec26b4708b2c7a2540c053dcf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.948654 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d27310f8433411a85bf88d05a81bcf0" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:17.948720 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 103f78d0d9fc42308e1e23316895be33 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 103f78d0d9fc42308e1e23316895be33 already in progress: creating tablet
I20250115 03:06:17.948755 14812 consensus_queue.cc:260] T d2220e7ec26b4708b2c7a2540c053dcf P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.948848 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet fa08518530ae4d5487af386a650c4672 (table default.Table_48dcc3ce48c94335a128ff0ce47649ae [id=12b47ada2a2c48f2b6807fa62fcca1a6]) successfully deleted
I20250115 03:06:17.948877 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet a73d45bf459c4137af6961b680d282ac with delete_type TABLET_DATA_DELETED (replaced by tablet 15263a3b32be418288770ce348c082d7) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:17.949064 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 103f78d0d9fc42308e1e23316895be33 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 103f78d0d9fc42308e1e23316895be33 already in progress: creating tablet
I20250115 03:06:17.949082 14812 ts_tablet_manager.cc:1428] T d2220e7ec26b4708b2c7a2540c053dcf P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20250115 03:06:17.949159 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet a73d45bf459c4137af6961b680d282ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a73d45bf459c4137af6961b680d282ac already in progress: creating tablet
I20250115 03:06:17.948994 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet aa7754f577af4ef6b6fca41e61e5965c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.949262 14812 tablet_bootstrap.cc:492] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:17.949404 14823 tablet_replica.cc:331] T aa7754f577af4ef6b6fca41e61e5965c P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:17.949510 14823 raft_consensus.cc:2238] T aa7754f577af4ef6b6fca41e61e5965c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.949596 14823 raft_consensus.cc:2267] T aa7754f577af4ef6b6fca41e61e5965c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.949774 14522 tablet_service.cc:1467] Processing CreateTablet for tablet 015ec42caa1b468ea10f25c8f26e278a (DEFAULT_TABLE table=default.Table_8ac3b54226f54715849ee4eb26e7a42f [id=e4e6540d82d94425a7175231b7e0aaaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.949982 14522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 015ec42caa1b468ea10f25c8f26e278a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.950088 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet aa7754f577af4ef6b6fca41e61e5965c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.950436 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 815c5e096e4d4b0cbfb7c15bac571b10 (DEFAULT_TABLE table=default.Table_e46e737b4c074b7a92903cf0f6a786ca [id=112f0d2601004b9b999a16b569f69d2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.950554 14397 tablet_service.cc:1467] Processing CreateTablet for tablet f66defba523f4df996929f98c848f95a (DEFAULT_TABLE table=default.Table_4b64655665164e00b508a19e84b9991f [id=c206de1bd1fa4e3db3d80167aca79165]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.950634 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 815c5e096e4d4b0cbfb7c15bac571b10. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.950675 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet e43ce064a02347efa81ca274d76c6960 with delete_type TABLET_DATA_DELETED (replaced by tablet b9f67c35591f4fdeb8a926a085171b49) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:17.950791 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66defba523f4df996929f98c848f95a. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:17.951088 14628 leader_election.cc:343] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.951264 14819 tablet_replica.cc:331] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:17.951406 14819 raft_consensus.cc:2238] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.951517 14819 raft_consensus.cc:2267] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.951864 14398 tablet_service.cc:1467] Processing CreateTablet for tablet b0a7def7d9a7418381b2d5a93a9a9265 (DEFAULT_TABLE table=default.Table_04a5f895839e461eaa6017426eca96de [id=ef69fd1a3c5e49d0b73ac5da8551140c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.952075 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0a7def7d9a7418381b2d5a93a9a9265. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.952241 14823 ts_tablet_manager.cc:1905] T aa7754f577af4ef6b6fca41e61e5965c P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.954351 14823 ts_tablet_manager.cc:1918] T aa7754f577af4ef6b6fca41e61e5965c P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:17.954365 14520 tablet_service.cc:1467] Processing CreateTablet for tablet f66defba523f4df996929f98c848f95a (DEFAULT_TABLE table=default.Table_4b64655665164e00b508a19e84b9991f [id=c206de1bd1fa4e3db3d80167aca79165]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.954397 14664 tablet_service.cc:1467] Processing CreateTablet for tablet b0a7def7d9a7418381b2d5a93a9a9265 (DEFAULT_TABLE table=default.Table_04a5f895839e461eaa6017426eca96de [id=ef69fd1a3c5e49d0b73ac5da8551140c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.954418 14823 log.cc:1198] T aa7754f577af4ef6b6fca41e61e5965c P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/aa7754f577af4ef6b6fca41e61e5965c
I20250115 03:06:17.954578 14520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66defba523f4df996929f98c848f95a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.954578 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0a7def7d9a7418381b2d5a93a9a9265. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.954641 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 815c5e096e4d4b0cbfb7c15bac571b10 (DEFAULT_TABLE table=default.Table_e46e737b4c074b7a92903cf0f6a786ca [id=112f0d2601004b9b999a16b569f69d2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.954793 14663 tablet_service.cc:1467] Processing CreateTablet for tablet f66defba523f4df996929f98c848f95a (DEFAULT_TABLE table=default.Table_4b64655665164e00b508a19e84b9991f [id=c206de1bd1fa4e3db3d80167aca79165]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.954888 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 815c5e096e4d4b0cbfb7c15bac571b10. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.954975 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f66defba523f4df996929f98c848f95a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.956305 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d27310f8433411a85bf88d05a81bcf0" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:17.956605 14626 leader_election.cc:343] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:17.956663 14626 leader_election.cc:304] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:17.956852 15325 raft_consensus.cc:2743] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:17.956998 14626 consensus_peers.cc:487] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:17.957098 14812 tablet_bootstrap.cc:654] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.957151 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 820d5af6daa74cadb53d2a5229854a60 (DEFAULT_TABLE table=default.Table_2653ad06e063469cbfda9ba1bada2951 [id=8a3b8556dcae4ded8de4adcb19115d83]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:17.957249 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet aa7754f577af4ef6b6fca41e61e5965c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa7754f577af4ef6b6fca41e61e5965c already in progress: creating tablet
I20250115 03:06:17.957367 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 820d5af6daa74cadb53d2a5229854a60. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:17.957743 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet a73d45bf459c4137af6961b680d282ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a73d45bf459c4137af6961b680d282ac already in progress: creating tablet
I20250115 03:06:17.959307 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet e43ce064a02347efa81ca274d76c6960 with delete_type TABLET_DATA_DELETED (replaced by tablet b9f67c35591f4fdeb8a926a085171b49) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:17.951897 14819 ts_tablet_manager.cc:1905] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:17.961047 16011 log.cc:926] Time spent T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1: Append to log took a long time: real 0.078s user 0.000s sys 0.000s
I20250115 03:06:17.961325 14170 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet dcdb5282e41c43dbb235ae78465d19f9 (table default.Table_375bb4e2095c4675b734247fabacbec7 [id=b298dce7ab664327893633f821ed4d89]). Expected version 1 got 0
I20250115 03:06:17.962183 14819 ts_tablet_manager.cc:1918] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:17.962244 14819 log.cc:1198] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/e43ce064a02347efa81ca274d76c6960
I20250115 03:06:17.962481 14819 ts_tablet_manager.cc:1939] T e43ce064a02347efa81ca274d76c6960 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:17.962867 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 015ec42caa1b468ea10f25c8f26e278a (DEFAULT_TABLE table=default.Table_8ac3b54226f54715849ee4eb26e7a42f [id=e4e6540d82d94425a7175231b7e0aaaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.963050 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 015ec42caa1b468ea10f25c8f26e278a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.964489 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 015ec42caa1b468ea10f25c8f26e278a (DEFAULT_TABLE table=default.Table_8ac3b54226f54715849ee4eb26e7a42f [id=e4e6540d82d94425a7175231b7e0aaaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.964720 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 015ec42caa1b468ea10f25c8f26e278a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.965929 14820 ts_tablet_manager.cc:1939] T 054251151c294e06905092f2a9306849 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:17.966503 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 054251151c294e06905092f2a9306849 (table default.Table_92b7f10774fe4db0ac9872d33fceb7a7 [id=54fd71a4b30d41b7b59b8dfc261a616d]) successfully deleted
W20250115 03:06:17.966879 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet c7e9a1d1ada941b68688e0ed6742aa99 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c7e9a1d1ada941b68688e0ed6742aa99 already in progress: creating tablet
I20250115 03:06:17.966912 14523 tablet_service.cc:1467] Processing CreateTablet for tablet 815c5e096e4d4b0cbfb7c15bac571b10 (DEFAULT_TABLE table=default.Table_e46e737b4c074b7a92903cf0f6a786ca [id=112f0d2601004b9b999a16b569f69d2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:17.967113 14523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 815c5e096e4d4b0cbfb7c15bac571b10. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:17.967142 14823 ts_tablet_manager.cc:1939] T aa7754f577af4ef6b6fca41e61e5965c P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
W20250115 03:06:17.967536 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet cc67fb818b8146629d92f9ac1944dba6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cc67fb818b8146629d92f9ac1944dba6 already in progress: creating tablet
I20250115 03:06:17.967603 14823 tablet_replica.cc:331] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
W20250115 03:06:17.967904 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 56c600fe18504e05bab2bcde3c89861d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 56c600fe18504e05bab2bcde3c89861d already in progress: creating tablet
I20250115 03:06:17.967926 14823 raft_consensus.cc:2238] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.968096 14820 tablet_replica.cc:331] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:17.968147 14823 raft_consensus.cc:2267] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.968206 14820 raft_consensus.cc:2238] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:17.968288 14820 raft_consensus.cc:2267] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:17.968570 14820 ts_tablet_manager.cc:1905] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.968695 14823 ts_tablet_manager.cc:1905] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:17.969038 16001 leader_election.cc:290] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:17.971614 14809 tablet_bootstrap.cc:492] T 30499e99ab7d4138a376dea40b768c32 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.971696 14809 ts_tablet_manager.cc:1397] T 30499e99ab7d4138a376dea40b768c32 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.215s user 0.000s sys 0.002s
I20250115 03:06:17.972054 14823 ts_tablet_manager.cc:1918] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:17.972138 14823 log.cc:1198] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/e43ce064a02347efa81ca274d76c6960
I20250115 03:06:17.972119 14809 raft_consensus.cc:357] T 30499e99ab7d4138a376dea40b768c32 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.972210 14809 raft_consensus.cc:383] T 30499e99ab7d4138a376dea40b768c32 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.972241 14809 raft_consensus.cc:738] T 30499e99ab7d4138a376dea40b768c32 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.972343 14809 consensus_queue.cc:260] T 30499e99ab7d4138a376dea40b768c32 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.972610 14809 ts_tablet_manager.cc:1428] T 30499e99ab7d4138a376dea40b768c32 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:17.972518 14823 ts_tablet_manager.cc:1939] T e43ce064a02347efa81ca274d76c6960 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:17.972783 14809 tablet_bootstrap.cc:492] T aa7754f577af4ef6b6fca41e61e5965c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.973637 14809 tablet_bootstrap.cc:654] T aa7754f577af4ef6b6fca41e61e5965c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.973721 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet de14b1d525cc42808500279ae6b60e80
I20250115 03:06:17.973958 14164 catalog_manager.cc:6596] ReplaceTablet: tablet de14b1d525cc42808500279ae6b60e80 deleted and replaced by tablet daea48ecc69d441ab66fcb40ea2e7a66
I20250115 03:06:17.977118 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_c9a7e51bb4ca41eba7b6a942eb649b81 [id=e441d6f5acea4837accbfb8e4856f82d] alter complete (version 2)
I20250115 03:06:17.977779 14812 tablet_bootstrap.cc:492] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.977851 14812 ts_tablet_manager.cc:1397] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20250115 03:06:17.978224 14812 raft_consensus.cc:357] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.978303 14812 raft_consensus.cc:383] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.978328 14812 raft_consensus.cc:738] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.978370 14523 tablet_service.cc:1514] Processing DeleteTablet for tablet de14b1d525cc42808500279ae6b60e80 with delete_type TABLET_DATA_DELETED (replaced by tablet daea48ecc69d441ab66fcb40ea2e7a66) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:17.978420 14812 consensus_queue.cc:260] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.978636 14812 ts_tablet_manager.cc:1428] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.978782 14812 tablet_bootstrap.cc:492] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:17.978782 14809 tablet_bootstrap.cc:492] T aa7754f577af4ef6b6fca41e61e5965c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.978849 14809 ts_tablet_manager.cc:1397] T aa7754f577af4ef6b6fca41e61e5965c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20250115 03:06:17.979231 14809 raft_consensus.cc:357] T aa7754f577af4ef6b6fca41e61e5965c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:17.979317 14809 raft_consensus.cc:383] T aa7754f577af4ef6b6fca41e61e5965c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.979349 14809 raft_consensus.cc:738] T aa7754f577af4ef6b6fca41e61e5965c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.979473 14809 consensus_queue.cc:260] T aa7754f577af4ef6b6fca41e61e5965c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:17.979564 14812 tablet_bootstrap.cc:654] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.979755 14809 ts_tablet_manager.cc:1428] T aa7754f577af4ef6b6fca41e61e5965c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.980486 14812 tablet_bootstrap.cc:492] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:17.980561 14812 ts_tablet_manager.cc:1397] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:17.981029 14812 raft_consensus.cc:357] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.981115 14812 raft_consensus.cc:383] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.981142 14812 raft_consensus.cc:738] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.981226 14812 consensus_queue.cc:260] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:17.982529 14812 ts_tablet_manager.cc:1428] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:17.982677 14812 tablet_bootstrap.cc:492] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:17.983534 14812 tablet_bootstrap.cc:654] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.984731 14809 tablet_bootstrap.cc:492] T 5d27310f8433411a85bf88d05a81bcf0 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.985579 14809 tablet_bootstrap.cc:654] T 5d27310f8433411a85bf88d05a81bcf0 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.987015 14809 tablet_bootstrap.cc:492] T 5d27310f8433411a85bf88d05a81bcf0 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.987243 14809 ts_tablet_manager.cc:1397] T 5d27310f8433411a85bf88d05a81bcf0 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:17.987686 14809 raft_consensus.cc:357] T 5d27310f8433411a85bf88d05a81bcf0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:17.987926 14809 raft_consensus.cc:383] T 5d27310f8433411a85bf88d05a81bcf0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.987996 14809 raft_consensus.cc:738] T 5d27310f8433411a85bf88d05a81bcf0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.988131 14809 consensus_queue.cc:260] T 5d27310f8433411a85bf88d05a81bcf0 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:17.988451 14809 ts_tablet_manager.cc:1428] T 5d27310f8433411a85bf88d05a81bcf0 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:17.988770 14809 tablet_bootstrap.cc:492] T 8b0f0539530643f78207e528c39530af P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.989786 14809 tablet_bootstrap.cc:654] T 8b0f0539530643f78207e528c39530af P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:17.991235 14809 tablet_bootstrap.cc:492] T 8b0f0539530643f78207e528c39530af P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:17.991438 14809 ts_tablet_manager.cc:1397] T 8b0f0539530643f78207e528c39530af P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:17.992316 14809 raft_consensus.cc:357] T 8b0f0539530643f78207e528c39530af P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:17.992519 14809 raft_consensus.cc:383] T 8b0f0539530643f78207e528c39530af P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:17.992646 14809 raft_consensus.cc:738] T 8b0f0539530643f78207e528c39530af P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:17.992877 14809 consensus_queue.cc:260] T 8b0f0539530643f78207e528c39530af P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:17.993656 14809 ts_tablet_manager.cc:1428] T 8b0f0539530643f78207e528c39530af P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:17.993944 14809 tablet_bootstrap.cc:492] T 7c503b29cb2843338fb1870a89aee443 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:17.995353 14809 tablet_bootstrap.cc:654] T 7c503b29cb2843338fb1870a89aee443 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.006922 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d1185d5f9bb6408585b742d6b55fec52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:18.006980 14885 consensus_queue.cc:1035] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:18.007279 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d1185d5f9bb6408585b742d6b55fec52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.007346 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_c3e03487af564b27a13203fb3ea3e1de"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.007684 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c3e03487af564b27a13203fb3ea3e1de in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.007966 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b0ca39935d224741840950333b63bcfe" } modify_external_catalogs: true
I20250115 03:06:18.008055 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b0ca39935d224741840950333b63bcfe" } modify_external_catalogs: true
I20250115 03:06:18.008437 15153 consensus_queue.cc:1035] T edf17b41198b4f8fbe5db22c58aa6bb2 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:18.008466 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ae52ad4b70d5436fac2aee0b178ee45e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.008757 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ae52ad4b70d5436fac2aee0b178ee45e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.008977 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_39e0d37b84fc42c0807a4cbffb7bae0a" } new_table_name: "default.Table_6bb3ca6745554454a044c329e6b7e8d3" modify_external_catalogs: true
I20250115 03:06:18.009516 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_69f57657cf714422b33241ef1f1951c7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.009804 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_69f57657cf714422b33241ef1f1951c7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.010668 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ea8049593bb4406883df33d6635d42e5" } new_table_name: "default.Table_62faf28a3d6b45698cdbb0e301130a90" modify_external_catalogs: true
I20250115 03:06:18.015199 14811 tablet_bootstrap.cc:492] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:18.015287 14811 ts_tablet_manager.cc:1397] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.131s user 0.000s sys 0.002s
I20250115 03:06:18.015722 14811 raft_consensus.cc:357] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.015831 14811 raft_consensus.cc:383] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.015861 14811 raft_consensus.cc:738] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.015969 14811 consensus_queue.cc:260] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.016019 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ba948e2cc56b4023a05595735e5a1434"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:18.016233 14811 ts_tablet_manager.cc:1428] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20250115 03:06:18.016321 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba948e2cc56b4023a05595735e5a1434 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.016412 14811 tablet_bootstrap.cc:492] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:18.017354 14811 tablet_bootstrap.cc:654] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.019937 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_09ddb549f5424be289e1ff6bdefbb152" } new_table_name: "default.Table_0b6cd33c30014981b78899a738418cbf" modify_external_catalogs: true
I20250115 03:06:18.019907 16021 tablet.cc:1721] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:18.020692 14523 tablet_service.cc:1514] Processing DeleteTablet for tablet 6110b6c4911d4932b4cac4d86598a5c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.026854 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet bf6f61dc685a48bdb367278773bf1d90
I20250115 03:06:18.027411 14523 tablet_service.cc:1514] Processing DeleteTablet for tablet bf6f61dc685a48bdb367278773bf1d90 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:18.029649 14493 consensus_peers.cc:487] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 -> Peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Couldn't send request to peer 5c0dae7aff4b44c699275b95ba639770. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e43ce064a02347efa81ca274d76c6960. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:18.030584 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet e43ce064a02347efa81ca274d76c6960 (table default.Table_4b95b36b8d744822b248adc43a061bb1 [id=0d8deaf903704c00a4a0576354ecfbe6]) successfully deleted
I20250115 03:06:18.031123 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet aa7754f577af4ef6b6fca41e61e5965c (table default.Table_b175cba7965344979476c3cb6d789eca [id=8ca6a98158114c7bb6bf49940d846cc6]) successfully deleted
I20250115 03:06:18.031322 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet e43ce064a02347efa81ca274d76c6960 (table default.Table_4b95b36b8d744822b248adc43a061bb1 [id=0d8deaf903704c00a4a0576354ecfbe6]) successfully deleted
I20250115 03:06:18.032405 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet de14b1d525cc42808500279ae6b60e80 with delete_type TABLET_DATA_DELETED (replaced by tablet daea48ecc69d441ab66fcb40ea2e7a66) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.032613 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet de14b1d525cc42808500279ae6b60e80 with delete_type TABLET_DATA_DELETED (replaced by tablet daea48ecc69d441ab66fcb40ea2e7a66) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.033098 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d27310f8433411a85bf88d05a81bcf0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.033509 14823 tablet_replica.cc:331] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.033629 14823 raft_consensus.cc:2238] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.033733 14823 raft_consensus.cc:2267] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.034076 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 30499e99ab7d4138a376dea40b768c32 with delete_type TABLET_DATA_DELETED (replaced by tablet 80ede4e9696d491aa385b375fcb5cf3b) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.034411 14823 ts_tablet_manager.cc:1905] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.035519 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet bf6f61dc685a48bdb367278773bf1d90 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:18.035900 14163 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 980b69b60a354519b0691ecb0bbd6c70 consensus state has no leader
W20250115 03:06:18.036533 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet de14b1d525cc42808500279ae6b60e80 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de14b1d525cc42808500279ae6b60e80 already in progress: creating tablet
I20250115 03:06:18.036623 14823 ts_tablet_manager.cc:1918] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.036691 14823 log.cc:1198] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/5d27310f8433411a85bf88d05a81bcf0
W20250115 03:06:18.036734 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet de14b1d525cc42808500279ae6b60e80 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de14b1d525cc42808500279ae6b60e80 already in progress: creating tablet
I20250115 03:06:18.036880 14823 ts_tablet_manager.cc:1939] T 5d27310f8433411a85bf88d05a81bcf0 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.037060 14819 tablet_replica.cc:331] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.037180 14819 raft_consensus.cc:2238] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:18.037354 14819 raft_consensus.cc:2267] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.037626 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet bf6f61dc685a48bdb367278773bf1d90 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.038149 14819 ts_tablet_manager.cc:1905] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:18.038291 14167 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 676ec8b9dc8241d4ae40488d3ce8f61e consensus state has no leader
I20250115 03:06:18.040230 14819 ts_tablet_manager.cc:1918] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.040292 14819 log.cc:1198] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/bf6f61dc685a48bdb367278773bf1d90
I20250115 03:06:18.040324 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e14dfeec5b704385b2947bebe118adae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:18.040503 14819 ts_tablet_manager.cc:1939] T bf6f61dc685a48bdb367278773bf1d90 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
W20250115 03:06:18.040642 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e14dfeec5b704385b2947bebe118adae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.041877 14820 ts_tablet_manager.cc:1918] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.042157 14820 log.cc:1198] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/420b7c8d43a04bae972eed2014d5634c
I20250115 03:06:18.043188 14820 ts_tablet_manager.cc:1939] T 420b7c8d43a04bae972eed2014d5634c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:18.043924 14823 tablet_replica.cc:331] T 30499e99ab7d4138a376dea40b768c32 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.044045 14823 raft_consensus.cc:2238] T 30499e99ab7d4138a376dea40b768c32 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.044144 14823 raft_consensus.cc:2267] T 30499e99ab7d4138a376dea40b768c32 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.044288 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet bf6f61dc685a48bdb367278773bf1d90 (table default.Table_b0ca39935d224741840950333b63bcfe [id=08996d7663c94ed1acb94b675012934c]) successfully deleted
I20250115 03:06:18.044405 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 5d27310f8433411a85bf88d05a81bcf0 (table default.Table_eec42b37d6de49ccad194b13edd928e3 [id=5d934a52e6704fa68f7c685ae3c72081]) successfully deleted
I20250115 03:06:18.044543 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 420b7c8d43a04bae972eed2014d5634c (table default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]) successfully deleted
W20250115 03:06:18.047142 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet f47da0e523d14388898a4be9c37bb809 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f47da0e523d14388898a4be9c37bb809 already in progress: creating tablet
I20250115 03:06:18.047166 14823 ts_tablet_manager.cc:1905] T 30499e99ab7d4138a376dea40b768c32 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.047904 14820 tablet_replica.cc:331] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:18.048019 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e6dfaa9c84d144e68a2d047436d0ecd2" } modify_external_catalogs: true
I20250115 03:06:18.048110 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e6dfaa9c84d144e68a2d047436d0ecd2" } modify_external_catalogs: true
I20250115 03:06:18.048297 14820 raft_consensus.cc:2238] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.048543 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f7de4dae6ab44600a02d5b8f92eafedd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:18.048492 14820 raft_consensus.cc:2267] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:18.048813 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f7de4dae6ab44600a02d5b8f92eafedd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.049130 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f1bb182509e64b589f4cb90482cec747" } new_table_name: "default.Table_4fbcf95abbb546dfb26e14668b42465f" modify_external_catalogs: true
I20250115 03:06:18.050137 14809 tablet_bootstrap.cc:492] T 7c503b29cb2843338fb1870a89aee443 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:18.051432 14809 ts_tablet_manager.cc:1397] T 7c503b29cb2843338fb1870a89aee443 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.058s user 0.002s sys 0.000s
I20250115 03:06:18.051941 14809 raft_consensus.cc:357] T 7c503b29cb2843338fb1870a89aee443 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.052095 14809 raft_consensus.cc:383] T 7c503b29cb2843338fb1870a89aee443 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.052171 14809 raft_consensus.cc:738] T 7c503b29cb2843338fb1870a89aee443 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.053084 14809 consensus_queue.cc:260] T 7c503b29cb2843338fb1870a89aee443 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:18.053208 14167 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f2b8bd783b614bf9b5526de197c51df3 consensus state has no leader
I20250115 03:06:18.054034 14823 ts_tablet_manager.cc:1918] T 30499e99ab7d4138a376dea40b768c32 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.054141 14823 log.cc:1198] T 30499e99ab7d4138a376dea40b768c32 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/30499e99ab7d4138a376dea40b768c32
I20250115 03:06:18.054369 14823 ts_tablet_manager.cc:1939] T 30499e99ab7d4138a376dea40b768c32 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.055009 14823 tablet_replica.cc:331] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.054911 14820 ts_tablet_manager.cc:1905] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.055140 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 30499e99ab7d4138a376dea40b768c32 (table default.Table_75cd7b01fc534652975eb593912b88bc [id=111fc67094164d3f99965c246ae12b24]) successfully deleted
I20250115 03:06:18.055174 14823 raft_consensus.cc:2238] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.055344 14823 raft_consensus.cc:2267] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.055380 14557 raft_consensus.cc:3054] T d28ed8538c4a44d18279371b826c22a5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.055879 14823 ts_tablet_manager.cc:1905] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.057186 14603 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:18.057521 14823 ts_tablet_manager.cc:1918] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.057575 14823 log.cc:1198] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/bf6f61dc685a48bdb367278773bf1d90
I20250115 03:06:18.057773 14823 ts_tablet_manager.cc:1939] T bf6f61dc685a48bdb367278773bf1d90 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.058259 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet bf6f61dc685a48bdb367278773bf1d90 (table default.Table_b0ca39935d224741840950333b63bcfe [id=08996d7663c94ed1acb94b675012934c]) successfully deleted
I20250115 03:06:18.062572 14170 master_service.cc:946] ReplaceTablet: received request to replace tablet 58161a322a3d469c8444d353689bed65 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:18.062901 14809 ts_tablet_manager.cc:1428] T 7c503b29cb2843338fb1870a89aee443 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.011s user 0.001s sys 0.000s
I20250115 03:06:18.063390 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_67f9f825d77c4874bb1d4a5c17dedde2" } new_table_name: "default.Table_fe2cfc8f0c564170b3c291cab294193f" modify_external_catalogs: true
I20250115 03:06:18.065158 14809 tablet_bootstrap.cc:492] T 074fdc09e4e24078ad0bb588b32ff24f P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:18.068912 14809 tablet_bootstrap.cc:654] T 074fdc09e4e24078ad0bb588b32ff24f P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.069511 14811 tablet_bootstrap.cc:492] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:18.069592 14811 ts_tablet_manager.cc:1397] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.053s user 0.001s sys 0.001s
I20250115 03:06:18.069947 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 58161a322a3d469c8444d353689bed65
I20250115 03:06:18.070019 14811 raft_consensus.cc:357] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.070104 14811 raft_consensus.cc:383] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.070134 14811 raft_consensus.cc:738] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.070214 14170 catalog_manager.cc:6596] ReplaceTablet: tablet 58161a322a3d469c8444d353689bed65 deleted and replaced by tablet d140be0fab694f59b8a535392970fad0
I20250115 03:06:18.070237 14811 consensus_queue.cc:260] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.070488 14811 ts_tablet_manager.cc:1428] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.070577 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 58161a322a3d469c8444d353689bed65 with delete_type TABLET_DATA_DELETED (replaced by tablet d140be0fab694f59b8a535392970fad0) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.070648 14811 tablet_bootstrap.cc:492] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:18.071193 14823 tablet_replica.cc:331] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.071317 14823 raft_consensus.cc:2238] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.071406 14823 raft_consensus.cc:2267] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.071532 14811 tablet_bootstrap.cc:654] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.071907 14823 ts_tablet_manager.cc:1905] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.073650 14823 ts_tablet_manager.cc:1918] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.073704 14823 log.cc:1198] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/58161a322a3d469c8444d353689bed65
I20250115 03:06:18.074011 14823 ts_tablet_manager.cc:1939] T 58161a322a3d469c8444d353689bed65 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.074508 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 58161a322a3d469c8444d353689bed65 (table default.Table_6df09ae8a979496abf73d384649b81b5 [id=e30f9e0f1aef45b7a1668a6d0f5f42bb]) successfully deleted
I20250115 03:06:18.075393 14688 raft_consensus.cc:1270] T 58130f2b27b745d08c0002ac96af6d72 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer e8df7247ee9c43e0aa18216da2661f75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:18.079183 14523 tablet_service.cc:1514] Processing DeleteTablet for tablet 58161a322a3d469c8444d353689bed65 with delete_type TABLET_DATA_DELETED (replaced by tablet d140be0fab694f59b8a535392970fad0) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.080509 14812 tablet_bootstrap.cc:492] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:18.072173 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 58161a322a3d469c8444d353689bed65 with delete_type TABLET_DATA_DELETED (replaced by tablet d140be0fab694f59b8a535392970fad0) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.080592 14812 ts_tablet_manager.cc:1397] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.098s user 0.002s sys 0.000s
I20250115 03:06:18.076597 15390 consensus_queue.cc:1035] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:18.081132 14812 raft_consensus.cc:357] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.081231 14812 raft_consensus.cc:383] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.081260 14812 raft_consensus.cc:738] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.081355 14812 consensus_queue.cc:260] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.081621 14812 ts_tablet_manager.cc:1428] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.081781 14812 tablet_bootstrap.cc:492] T c84127dd93bc47ada551f93295ead47b P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:18.082116 14356 consensus_peers.cc:487] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:18.082783 14812 tablet_bootstrap.cc:654] T c84127dd93bc47ada551f93295ead47b P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.086148 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_18deb1c4560441b8abc82d6709285a34" } modify_external_catalogs: true
I20250115 03:06:18.086243 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_18deb1c4560441b8abc82d6709285a34" } modify_external_catalogs: true
I20250115 03:06:18.088011 14166 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 7c503b29cb2843338fb1870a89aee443 (table default.Table_dfcccd3390fa4814a0a04bb1ff5e43fa [id=f0a7068682b9481a82fa34cb54ed528e]). Expected version 1 got 0
I20250115 03:06:18.088515 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 8b0f0539530643f78207e528c39530af (table default.Table_f7799772605a4284a062cd0e4ae15ed3 [id=fe357a2a64fc4531870b305eb2480df1]). Expected version 1 got 0
I20250115 03:06:18.089102 15617 tablet.cc:1721] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:18.090206 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet d2220e7ec26b4708b2c7a2540c053dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.090459 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 7340ac89e5524c35948f3acc6c941cb5 (DEFAULT_TABLE table=default.Table_562d28c67cc14d868fe4194eb7a4a05f [id=b03eeb2678e14625957a2f4acac474d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.090694 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7340ac89e5524c35948f3acc6c941cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.090883 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 4c4ce5c2921341ce837ee4f5f7d131b7 (DEFAULT_TABLE table=default.Table_18deb1c4560441b8abc82d6709285a34 [id=049731ed01874931b8bd1d2912361c43]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.091055 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 7340ac89e5524c35948f3acc6c941cb5 (DEFAULT_TABLE table=default.Table_562d28c67cc14d868fe4194eb7a4a05f [id=b03eeb2678e14625957a2f4acac474d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.091096 14393 tablet_service.cc:1467] Processing CreateTablet for tablet b9f67c35591f4fdeb8a926a085171b49 (DEFAULT_TABLE table=default.Table_4b95b36b8d744822b248adc43a061bb1 [id=0d8deaf903704c00a4a0576354ecfbe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.091317 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 9c436975bd034068a6d58d5831db1896 (DEFAULT_TABLE table=default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.091387 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 85fcafa24fa1448c995d30dd93b0737f (DEFAULT_TABLE table=default.Table_9c49fab0c4db42e1b04b2703b0f3851a [id=827543773ec041c0aded0ef6d7e8c49b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.091501 14395 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d27310f8433411a85bf88d05a81bcf0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.091552 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet aa7754f577af4ef6b6fca41e61e5965c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.091703 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 2b0e3df3c79c40ba958b6e2453fa666f (DEFAULT_TABLE table=default.Table_511db1d50b7246e9b6dc2d6ed50da4d0 [id=14cf5f7ea1964ed8ad8d000e12904c09]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.092144 14823 tablet_replica.cc:331] T d2220e7ec26b4708b2c7a2540c053dcf P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.092253 14823 raft_consensus.cc:2238] T d2220e7ec26b4708b2c7a2540c053dcf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.092334 14823 raft_consensus.cc:2267] T d2220e7ec26b4708b2c7a2540c053dcf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.092658 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 85fcafa24fa1448c995d30dd93b0737f (DEFAULT_TABLE table=default.Table_9c49fab0c4db42e1b04b2703b0f3851a [id=827543773ec041c0aded0ef6d7e8c49b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.092720 14823 ts_tablet_manager.cc:1905] T d2220e7ec26b4708b2c7a2540c053dcf P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.092864 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85fcafa24fa1448c995d30dd93b0737f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.094604 14823 ts_tablet_manager.cc:1918] T d2220e7ec26b4708b2c7a2540c053dcf P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.094668 14823 log.cc:1198] T d2220e7ec26b4708b2c7a2540c053dcf P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/d2220e7ec26b4708b2c7a2540c053dcf
I20250115 03:06:18.094787 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 4c4ce5c2921341ce837ee4f5f7d131b7 (DEFAULT_TABLE table=default.Table_18deb1c4560441b8abc82d6709285a34 [id=049731ed01874931b8bd1d2912361c43]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.094841 14823 ts_tablet_manager.cc:1939] T d2220e7ec26b4708b2c7a2540c053dcf P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.094971 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c4ce5c2921341ce837ee4f5f7d131b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.095119 14166 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 7c503b29cb2843338fb1870a89aee443 (table default.Table_dfcccd3390fa4814a0a04bb1ff5e43fa [id=f0a7068682b9481a82fa34cb54ed528e]). Expected version 1 got 0
I20250115 03:06:18.096777 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 30499e99ab7d4138a376dea40b768c32 with delete_type TABLET_DATA_DELETED (replaced by tablet 80ede4e9696d491aa385b375fcb5cf3b) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.097028 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 15263a3b32be418288770ce348c082d7 (DEFAULT_TABLE table=default.Table_70287f2d5a024cc88244993789eb747b [id=88146f846aa74ecfb3977f01ea45df2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.097061 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 78cc96de8b414ac19285338ccb1024e0
I20250115 03:06:18.097177 15970 tablet.cc:1721] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:18.097424 14819 tablet_replica.cc:331] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.097543 14819 raft_consensus.cc:2238] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:18.097710 14819 raft_consensus.cc:2267] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.098016 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c4ce5c2921341ce837ee4f5f7d131b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.098747 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d31b716d5bb04894865c704529827af0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:18.098968 14523 tablet_service.cc:1467] Processing CreateTablet for tablet 7340ac89e5524c35948f3acc6c941cb5 (DEFAULT_TABLE table=default.Table_562d28c67cc14d868fe4194eb7a4a05f [id=b03eeb2678e14625957a2f4acac474d1]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:18.099037 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d31b716d5bb04894865c704529827af0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.099196 14523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7340ac89e5524c35948f3acc6c941cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.099954 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 78cc96de8b414ac19285338ccb1024e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.100147 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7340ac89e5524c35948f3acc6c941cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.101099 14520 tablet_service.cc:1467] Processing CreateTablet for tablet 85fcafa24fa1448c995d30dd93b0737f (DEFAULT_TABLE table=default.Table_9c49fab0c4db42e1b04b2703b0f3851a [id=827543773ec041c0aded0ef6d7e8c49b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.101356 14522 tablet_service.cc:1467] Processing CreateTablet for tablet 4c4ce5c2921341ce837ee4f5f7d131b7 (DEFAULT_TABLE table=default.Table_18deb1c4560441b8abc82d6709285a34 [id=049731ed01874931b8bd1d2912361c43]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.101625 14518 tablet_service.cc:1467] Processing CreateTablet for tablet b9f67c35591f4fdeb8a926a085171b49 (DEFAULT_TABLE table=default.Table_4b95b36b8d744822b248adc43a061bb1 [id=0d8deaf903704c00a4a0576354ecfbe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.101856 14524 tablet_service.cc:1467] Processing CreateTablet for tablet 2b0e3df3c79c40ba958b6e2453fa666f (DEFAULT_TABLE table=default.Table_511db1d50b7246e9b6dc2d6ed50da4d0 [id=14cf5f7ea1964ed8ad8d000e12904c09]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.101917 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9f67c35591f4fdeb8a926a085171b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.102084 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 15263a3b32be418288770ce348c082d7 (DEFAULT_TABLE table=default.Table_70287f2d5a024cc88244993789eb747b [id=88146f846aa74ecfb3977f01ea45df2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.102303 14519 tablet_service.cc:1467] Processing CreateTablet for tablet 9c436975bd034068a6d58d5831db1896 (DEFAULT_TABLE table=default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.102449 14525 tablet_service.cc:1514] Processing DeleteTablet for tablet 78cc96de8b414ac19285338ccb1024e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.103729 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c436975bd034068a6d58d5831db1896. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.105379 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e0faf1e67748430cb2b830441fd58344" } modify_external_catalogs: true
I20250115 03:06:18.105466 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e0faf1e67748430cb2b830441fd58344" } modify_external_catalogs: true
I20250115 03:06:18.105926 15325 raft_consensus.cc:491] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.105995 15325 raft_consensus.cc:513] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.106083 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 78cc96de8b414ac19285338ccb1024e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.106145 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet d2220e7ec26b4708b2c7a2540c053dcf (table default.Table_807e7c2704994c25aa03b1267a20f794 [id=fb7018f8e1ac4d6db5c0c114b0e73a07]) successfully deleted
I20250115 03:06:18.106315 15325 leader_election.cc:290] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:18.106469 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 9c436975bd034068a6d58d5831db1896 (DEFAULT_TABLE table=default.Table_b20b6ea81915403f986c91997f637c86 [id=522c89cadb084de68368a97199ea66db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.106670 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c436975bd034068a6d58d5831db1896. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.108310 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c503b29cb2843338fb1870a89aee443" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:18.109015 14626 leader_election.cc:343] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:18.109427 14519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c436975bd034068a6d58d5831db1896. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.109769 14819 ts_tablet_manager.cc:1905] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.111435 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15263a3b32be418288770ce348c082d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.111539 14819 ts_tablet_manager.cc:1918] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.111593 14819 log.cc:1198] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/58161a322a3d469c8444d353689bed65
I20250115 03:06:18.111809 14819 ts_tablet_manager.cc:1939] T 58161a322a3d469c8444d353689bed65 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
W20250115 03:06:18.112429 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 78cc96de8b414ac19285338ccb1024e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78cc96de8b414ac19285338ccb1024e0 already in progress: creating tablet
I20250115 03:06:18.112974 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c503b29cb2843338fb1870a89aee443" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:18.112990 14669 tablet_service.cc:1467] Processing CreateTablet for tablet b9f67c35591f4fdeb8a926a085171b49 (DEFAULT_TABLE table=default.Table_4b95b36b8d744822b248adc43a061bb1 [id=0d8deaf903704c00a4a0576354ecfbe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.113080 14421 raft_consensus.cc:2463] T 7c503b29cb2843338fb1870a89aee443 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:18.113184 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9f67c35591f4fdeb8a926a085171b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.113443 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b0e3df3c79c40ba958b6e2453fa666f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.113483 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9f67c35591f4fdeb8a926a085171b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.115032 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 2b0e3df3c79c40ba958b6e2453fa666f (DEFAULT_TABLE table=default.Table_511db1d50b7246e9b6dc2d6ed50da4d0 [id=14cf5f7ea1964ed8ad8d000e12904c09]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.115227 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15263a3b32be418288770ce348c082d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.115227 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b0e3df3c79c40ba958b6e2453fa666f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.116609 14524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b0e3df3c79c40ba958b6e2453fa666f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.116904 14520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85fcafa24fa1448c995d30dd93b0737f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.117141 14628 leader_election.cc:304] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:18.117214 14522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c4ce5c2921341ce837ee4f5f7d131b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.117267 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85fcafa24fa1448c995d30dd93b0737f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.117480 14820 ts_tablet_manager.cc:1918] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.117904 15325 raft_consensus.cc:2798] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:18.117965 15325 raft_consensus.cc:491] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:18.118010 15325 raft_consensus.cc:3054] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.117952 14820 log.cc:1198] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/5986558350244e7aa4ddcd5cccb7d676
I20250115 03:06:18.118829 15325 raft_consensus.cc:513] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.119141 15325 leader_election.cc:290] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:18.119338 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 15263a3b32be418288770ce348c082d7 (DEFAULT_TABLE table=default.Table_70287f2d5a024cc88244993789eb747b [id=88146f846aa74ecfb3977f01ea45df2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.119601 14819 tablet_replica.cc:331] T 5d27310f8433411a85bf88d05a81bcf0 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.120071 14819 raft_consensus.cc:2238] T 5d27310f8433411a85bf88d05a81bcf0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.120168 14819 raft_consensus.cc:2267] T 5d27310f8433411a85bf88d05a81bcf0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.120265 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c503b29cb2843338fb1870a89aee443" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:18.120323 14820 ts_tablet_manager.cc:1939] T 5986558350244e7aa4ddcd5cccb7d676 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:18.120357 14421 raft_consensus.cc:3054] T 7c503b29cb2843338fb1870a89aee443 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.121289 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 58161a322a3d469c8444d353689bed65 (table default.Table_6df09ae8a979496abf73d384649b81b5 [id=e30f9e0f1aef45b7a1668a6d0f5f42bb]) successfully deleted
I20250115 03:06:18.121593 14421 raft_consensus.cc:2463] T 7c503b29cb2843338fb1870a89aee443 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:18.121889 14628 leader_election.cc:304] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:18.121941 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15263a3b32be418288770ce348c082d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.122435 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c503b29cb2843338fb1870a89aee443" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:18.122822 14626 leader_election.cc:343] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:18.123340 14819 ts_tablet_manager.cc:1905] T 5d27310f8433411a85bf88d05a81bcf0 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.123600 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 0d25aa6a292b454f8f451d6493d940c3
I20250115 03:06:18.123826 15325 raft_consensus.cc:2798] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:18.123900 15325 raft_consensus.cc:695] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:18.124006 15325 consensus_queue.cc:237] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.124223 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d25aa6a292b454f8f451d6493d940c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.124347 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d25aa6a292b454f8f451d6493d940c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.124471 14823 tablet_replica.cc:331] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.124536 15325 raft_consensus.cc:491] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.124573 14823 raft_consensus.cc:2238] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.124604 15325 raft_consensus.cc:513] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.124652 14823 raft_consensus.cc:2267] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.124892 15325 leader_election.cc:290] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:18.125170 14819 ts_tablet_manager.cc:1918] T 5d27310f8433411a85bf88d05a81bcf0 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.125227 14819 log.cc:1198] T 5d27310f8433411a85bf88d05a81bcf0 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/5d27310f8433411a85bf88d05a81bcf0
I20250115 03:06:18.125401 14819 ts_tablet_manager.cc:1939] T 5d27310f8433411a85bf88d05a81bcf0 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.126041 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b0f0539530643f78207e528c39530af" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:18.126137 14421 raft_consensus.cc:2463] T 8b0f0539530643f78207e528c39530af P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:18.126585 14819 tablet_replica.cc:331] T aa7754f577af4ef6b6fca41e61e5965c P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.126664 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 5d27310f8433411a85bf88d05a81bcf0 (table default.Table_eec42b37d6de49ccad194b13edd928e3 [id=5d934a52e6704fa68f7c685ae3c72081]) successfully deleted
I20250115 03:06:18.126698 14819 raft_consensus.cc:2238] T aa7754f577af4ef6b6fca41e61e5965c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.126783 14819 raft_consensus.cc:2267] T aa7754f577af4ef6b6fca41e61e5965c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.124984 14823 ts_tablet_manager.cc:1905] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.127111 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 5986558350244e7aa4ddcd5cccb7d676 (table default.Table_e72d1ee69400410c9da9082d4bd8bc9a [id=6ab343bee1254fb19a3024ebafa44464]) successfully deleted
I20250115 03:06:18.127787 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b0f0539530643f78207e528c39530af" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:18.128660 14823 ts_tablet_manager.cc:1918] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20250115 03:06:18.129134 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 85cceb5a3f234d85937ad0bb75348991 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 85cceb5a3f234d85937ad0bb75348991 already in progress: creating tablet
I20250115 03:06:18.128378 14820 tablet_replica.cc:331] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:18.130008 14819 ts_tablet_manager.cc:1905] T aa7754f577af4ef6b6fca41e61e5965c P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.130029 14823 log.cc:1198] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/0d25aa6a292b454f8f451d6493d940c3
I20250115 03:06:18.130370 14823 ts_tablet_manager.cc:1939] T 0d25aa6a292b454f8f451d6493d940c3 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.130570 14820 raft_consensus.cc:2238] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.130966 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 0d25aa6a292b454f8f451d6493d940c3 (table default.Table_e0faf1e67748430cb2b830441fd58344 [id=8698f51a556d4fcc8ed5ebe0bbc9feb0]) successfully deleted
I20250115 03:06:18.126411 14628 leader_election.cc:304] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
W20250115 03:06:18.131139 14626 leader_election.cc:343] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:18.128705 14520 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d25aa6a292b454f8f451d6493d940c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.132357 14819 ts_tablet_manager.cc:1918] T aa7754f577af4ef6b6fca41e61e5965c P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.132424 14819 log.cc:1198] T aa7754f577af4ef6b6fca41e61e5965c P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/aa7754f577af4ef6b6fca41e61e5965c
I20250115 03:06:18.132133 14820 raft_consensus.cc:2267] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.132624 14819 ts_tablet_manager.cc:1939] T aa7754f577af4ef6b6fca41e61e5965c P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.133137 14819 tablet_replica.cc:331] T 30499e99ab7d4138a376dea40b768c32 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.133268 14819 raft_consensus.cc:2238] T 30499e99ab7d4138a376dea40b768c32 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.133358 14819 raft_consensus.cc:2267] T 30499e99ab7d4138a376dea40b768c32 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.133772 14809 tablet_bootstrap.cc:492] T 074fdc09e4e24078ad0bb588b32ff24f P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:18.133822 14819 ts_tablet_manager.cc:1905] T 30499e99ab7d4138a376dea40b768c32 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.133942 14809 ts_tablet_manager.cc:1397] T 074fdc09e4e24078ad0bb588b32ff24f P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.069s user 0.001s sys 0.001s
I20250115 03:06:18.134387 14809 raft_consensus.cc:357] T 074fdc09e4e24078ad0bb588b32ff24f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.134570 14809 raft_consensus.cc:383] T 074fdc09e4e24078ad0bb588b32ff24f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.134723 14809 raft_consensus.cc:738] T 074fdc09e4e24078ad0bb588b32ff24f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.134887 14809 consensus_queue.cc:260] T 074fdc09e4e24078ad0bb588b32ff24f P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.135512 14819 ts_tablet_manager.cc:1918] T 30499e99ab7d4138a376dea40b768c32 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.135573 14819 log.cc:1198] T 30499e99ab7d4138a376dea40b768c32 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/30499e99ab7d4138a376dea40b768c32
I20250115 03:06:18.135751 14819 ts_tablet_manager.cc:1939] T 30499e99ab7d4138a376dea40b768c32 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.136253 14819 tablet_replica.cc:331] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.136353 14819 raft_consensus.cc:2238] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:18.137189 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet aa7754f577af4ef6b6fca41e61e5965c (table default.Table_b175cba7965344979476c3cb6d789eca [id=8ca6a98158114c7bb6bf49940d846cc6]) successfully deleted
I20250115 03:06:18.137306 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 30499e99ab7d4138a376dea40b768c32 (table default.Table_75cd7b01fc534652975eb593912b88bc [id=111fc67094164d3f99965c246ae12b24]) successfully deleted
W20250115 03:06:18.137414 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 78cc96de8b414ac19285338ccb1024e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78cc96de8b414ac19285338ccb1024e0 already in progress: creating tablet
I20250115 03:06:18.138341 14820 ts_tablet_manager.cc:1905] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.139442 15325 raft_consensus.cc:2798] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:18.139492 15325 raft_consensus.cc:491] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:18.139523 15325 raft_consensus.cc:3054] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.139951 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_67d007791e734b749420eb72d54c0c69" } modify_external_catalogs: true
I20250115 03:06:18.140055 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_67d007791e734b749420eb72d54c0c69" } modify_external_catalogs: true
I20250115 03:06:18.140233 15325 raft_consensus.cc:513] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.140558 15325 leader_election.cc:290] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:18.140759 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b0f0539530643f78207e528c39530af" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:18.140821 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b0f0539530643f78207e528c39530af" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:18.140904 14421 raft_consensus.cc:3054] T 8b0f0539530643f78207e528c39530af P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.141649 14421 raft_consensus.cc:2463] T 8b0f0539530643f78207e528c39530af P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:18.141935 14628 leader_election.cc:304] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:18.142076 15325 raft_consensus.cc:2798] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:18.142141 15325 raft_consensus.cc:695] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:18.142241 15325 consensus_queue.cc:237] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.143522 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 41b22aa85ce34b38b0d6a5e2d38f0a1c
I20250115 03:06:18.143921 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 41b22aa85ce34b38b0d6a5e2d38f0a1c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.144098 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet 41b22aa85ce34b38b0d6a5e2d38f0a1c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:18.145123 14626 leader_election.cc:343] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:18.145862 14520 tablet_service.cc:1514] Processing DeleteTablet for tablet 41b22aa85ce34b38b0d6a5e2d38f0a1c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.146107 14823 tablet_replica.cc:331] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.146246 14823 raft_consensus.cc:2238] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.146335 14823 raft_consensus.cc:2267] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.146636 14823 ts_tablet_manager.cc:1905] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.148355 14823 ts_tablet_manager.cc:1918] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.148413 14823 log.cc:1198] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/41b22aa85ce34b38b0d6a5e2d38f0a1c
I20250115 03:06:18.148636 14823 ts_tablet_manager.cc:1939] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.149013 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 41b22aa85ce34b38b0d6a5e2d38f0a1c (table default.Table_67d007791e734b749420eb72d54c0c69 [id=588a068527c446e9aa2d1910ac9dca64]) successfully deleted
I20250115 03:06:18.149412 14468 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:18.150488 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_397a360e045b4b5fabed16c0feff8112"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.150820 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_397a360e045b4b5fabed16c0feff8112 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.154780 14170 master_service.cc:946] ReplaceTablet: received request to replace tablet 4d4b8d4ed9f44112b83c6ff94aee2206 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:18.157740 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 4d4b8d4ed9f44112b83c6ff94aee2206
I20250115 03:06:18.157992 14170 catalog_manager.cc:6596] ReplaceTablet: tablet 4d4b8d4ed9f44112b83c6ff94aee2206 deleted and replaced by tablet 3724ff14f535416e859339a137c05050
I20250115 03:06:18.158228 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d4b8d4ed9f44112b83c6ff94aee2206 with delete_type TABLET_DATA_DELETED (replaced by tablet 3724ff14f535416e859339a137c05050) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.158365 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d4b8d4ed9f44112b83c6ff94aee2206 with delete_type TABLET_DATA_DELETED (replaced by tablet 3724ff14f535416e859339a137c05050) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.158504 14823 tablet_replica.cc:331] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.158552 14170 master_service.cc:946] ReplaceTablet: received request to replace tablet 9bb0b7a8acfd40738abcc951936b1116 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:18.158623 14823 raft_consensus.cc:2238] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.158728 14823 raft_consensus.cc:2267] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.158819 14520 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d4b8d4ed9f44112b83c6ff94aee2206 with delete_type TABLET_DATA_DELETED (replaced by tablet 3724ff14f535416e859339a137c05050) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.159170 14823 ts_tablet_manager.cc:1905] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.160785 14809 ts_tablet_manager.cc:1428] T 074fdc09e4e24078ad0bb588b32ff24f P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.027s user 0.000s sys 0.001s
I20250115 03:06:18.161036 14823 ts_tablet_manager.cc:1918] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:18.161108 14823 log.cc:1198] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/4d4b8d4ed9f44112b83c6ff94aee2206
I20250115 03:06:18.161339 14823 ts_tablet_manager.cc:1939] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.161893 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 4d4b8d4ed9f44112b83c6ff94aee2206 (table default.Table_b900c4395de448a08b0285a5807f2fdf [id=adfd7e86aa0f48e5b5e36b4113517dba]) successfully deleted
I20250115 03:06:18.163755 14811 tablet_bootstrap.cc:492] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:18.163842 14811 ts_tablet_manager.cc:1397] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.093s user 0.001s sys 0.000s
I20250115 03:06:18.164285 14811 raft_consensus.cc:357] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.164376 14811 raft_consensus.cc:383] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.164410 14811 raft_consensus.cc:738] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.164531 14811 consensus_queue.cc:260] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.164688 14809 tablet_bootstrap.cc:492] T d2220e7ec26b4708b2c7a2540c053dcf P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:18.164721 14819 raft_consensus.cc:2267] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.164759 14820 ts_tablet_manager.cc:1918] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:18.166220 14820 log.cc:1198] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/f70a69acbdc049beb9d98eada09d9eb8
I20250115 03:06:18.166554 14819 ts_tablet_manager.cc:1905] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.164789 14811 ts_tablet_manager.cc:1428] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.168259 14811 tablet_bootstrap.cc:492] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:18.170197 14885 consensus_queue.cc:1035] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:18.169315 14820 ts_tablet_manager.cc:1939] T f70a69acbdc049beb9d98eada09d9eb8 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:18.172331 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2e466ac2b74a4a1b8a5b12182123fd7c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.172637 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e466ac2b74a4a1b8a5b12182123fd7c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.178361 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet f70a69acbdc049beb9d98eada09d9eb8 (table default.Table_db8a2673977247e0b14b7995f7e09661 [id=f7e4534c1f6f48f09e10b84c1d5c458c]) successfully deleted
I20250115 03:06:18.181123 15617 tablet.cc:1721] T 58130f2b27b745d08c0002ac96af6d72 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:18.181866 15152 mvcc.cc:204] Tried to move back new op lower bound from 7114384908592398336 to 7114384907668267008. Current Snapshot: MvccSnapshot[applied={T|T < 7114384908592398336}]
I20250115 03:06:18.182024 15970 tablet.cc:1721] T 58130f2b27b745d08c0002ac96af6d72 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:18.177618 14820 tablet_replica.cc:331] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
W20250115 03:06:18.186885 14356 consensus_peers.cc:487] T b6175f691fec45bf9853ee90959a1c98 P e8df7247ee9c43e0aa18216da2661f75 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:18.179153 14520 tablet_service.cc:1514] Processing DeleteTablet for tablet d0c3536533e94174bb1e3888a3458395 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.179862 14811 tablet_bootstrap.cc:654] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.190459 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet ae0efe52ac6c40fbae45c725b07b03a3 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:18.193892 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet ae0efe52ac6c40fbae45c725b07b03a3
I20250115 03:06:18.195127 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet 3007a0506ce14c6c89c600b98a81b168 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:18.195493 14820 raft_consensus.cc:2238] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.195698 14820 raft_consensus.cc:2267] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.196491 14820 ts_tablet_manager.cc:1905] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.198702 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 3007a0506ce14c6c89c600b98a81b168
I20250115 03:06:18.199553 16001 consensus_queue.cc:1035] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:18.203590 14885 consensus_queue.cc:1035] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:18.205497 14809 tablet_bootstrap.cc:654] T d2220e7ec26b4708b2c7a2540c053dcf P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.206641 14812 tablet_bootstrap.cc:492] T c84127dd93bc47ada551f93295ead47b P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:18.206714 14812 ts_tablet_manager.cc:1397] T c84127dd93bc47ada551f93295ead47b P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.125s user 0.002s sys 0.000s
I20250115 03:06:18.207105 14812 raft_consensus.cc:357] T c84127dd93bc47ada551f93295ead47b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.207186 14812 raft_consensus.cc:383] T c84127dd93bc47ada551f93295ead47b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.207212 14812 raft_consensus.cc:738] T c84127dd93bc47ada551f93295ead47b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.207310 14812 consensus_queue.cc:260] T c84127dd93bc47ada551f93295ead47b P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.207541 14812 ts_tablet_manager.cc:1428] T c84127dd93bc47ada551f93295ead47b P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.207679 14812 tablet_bootstrap.cc:492] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:18.208495 14812 tablet_bootstrap.cc:654] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.211519 14168 catalog_manager.cc:6596] ReplaceTablet: tablet 3007a0506ce14c6c89c600b98a81b168 deleted and replaced by tablet 1fd348b3ae2a45dfa05c52f02536a201
I20250115 03:06:18.211768 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 3007a0506ce14c6c89c600b98a81b168 with delete_type TABLET_DATA_DELETED (replaced by tablet 1fd348b3ae2a45dfa05c52f02536a201) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.211908 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet 3007a0506ce14c6c89c600b98a81b168 with delete_type TABLET_DATA_DELETED (replaced by tablet 1fd348b3ae2a45dfa05c52f02536a201) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.212097 14823 tablet_replica.cc:331] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.212129 14171 catalog_manager.cc:6596] ReplaceTablet: tablet ae0efe52ac6c40fbae45c725b07b03a3 deleted and replaced by tablet 20d71366a4a747bcaa18d9ce14d05b91
I20250115 03:06:18.212205 14823 raft_consensus.cc:2238] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:18.212327 14823 pending_rounds.cc:62] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 1 pending ops.
I20250115 03:06:18.212357 14823 pending_rounds.cc:69] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384907454324736 op_type: NO_OP noop_request { }
I20250115 03:06:18.212402 14823 raft_consensus.cc:2883] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:18.212446 14823 raft_consensus.cc:2267] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.212584 14520 tablet_service.cc:1514] Processing DeleteTablet for tablet 3007a0506ce14c6c89c600b98a81b168 with delete_type TABLET_DATA_DELETED (replaced by tablet 1fd348b3ae2a45dfa05c52f02536a201) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.212705 14524 tablet_service.cc:1514] Processing DeleteTablet for tablet ae0efe52ac6c40fbae45c725b07b03a3 with delete_type TABLET_DATA_DELETED (replaced by tablet 20d71366a4a747bcaa18d9ce14d05b91) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.212756 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet ae0efe52ac6c40fbae45c725b07b03a3 with delete_type TABLET_DATA_DELETED (replaced by tablet 20d71366a4a747bcaa18d9ce14d05b91) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.212898 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet ae0efe52ac6c40fbae45c725b07b03a3 with delete_type TABLET_DATA_DELETED (replaced by tablet 20d71366a4a747bcaa18d9ce14d05b91) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.213407 14823 ts_tablet_manager.cc:1905] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.214783 14819 ts_tablet_manager.cc:1918] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.214946 14819 log.cc:1198] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/0d25aa6a292b454f8f451d6493d940c3
I20250115 03:06:18.215308 14823 ts_tablet_manager.cc:1918] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.215369 14823 log.cc:1198] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/3007a0506ce14c6c89c600b98a81b168
I20250115 03:06:18.215425 14819 ts_tablet_manager.cc:1939] T 0d25aa6a292b454f8f451d6493d940c3 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.215605 14823 ts_tablet_manager.cc:1939] T 3007a0506ce14c6c89c600b98a81b168 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.216041 14823 tablet_replica.cc:331] T ae0efe52ac6c40fbae45c725b07b03a3 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.216138 14823 raft_consensus.cc:2238] T ae0efe52ac6c40fbae45c725b07b03a3 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.216223 14823 raft_consensus.cc:2267] T ae0efe52ac6c40fbae45c725b07b03a3 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.216264 14819 tablet_replica.cc:331] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.216487 14823 ts_tablet_manager.cc:1905] T ae0efe52ac6c40fbae45c725b07b03a3 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.216459 14819 raft_consensus.cc:2238] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:18.216720 14819 raft_consensus.cc:2267] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.217021 16001 raft_consensus.cc:491] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.217103 16001 raft_consensus.cc:513] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.217361 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 0d25aa6a292b454f8f451d6493d940c3 (table default.Table_e0faf1e67748430cb2b830441fd58344 [id=8698f51a556d4fcc8ed5ebe0bbc9feb0]) successfully deleted
I20250115 03:06:18.217439 16001 leader_election.cc:290] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:18.217816 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "074fdc09e4e24078ad0bb588b32ff24f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:18.217911 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 3007a0506ce14c6c89c600b98a81b168 (table default.Table_cf7f3fbd11804e94848a7c5357169a86 [id=d64515775f264a419768662b8ce0a854]) successfully deleted
I20250115 03:06:18.217933 14420 raft_consensus.cc:2463] T 074fdc09e4e24078ad0bb588b32ff24f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:18.218068 14819 ts_tablet_manager.cc:1905] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.218554 14628 leader_election.cc:304] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:18.218712 16001 raft_consensus.cc:2798] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:18.218777 16001 raft_consensus.cc:491] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20250115 03:06:18.218796 14358 consensus_peers.cc:487] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 -> Peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Couldn't send request to peer 5c0dae7aff4b44c699275b95ba639770. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4d4b8d4ed9f44112b83c6ff94aee2206. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:18.218814 16001 raft_consensus.cc:3054] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.219645 16001 raft_consensus.cc:513] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.219970 16001 leader_election.cc:290] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:18.220239 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "074fdc09e4e24078ad0bb588b32ff24f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:18.220328 14420 raft_consensus.cc:3054] T 074fdc09e4e24078ad0bb588b32ff24f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.220428 14823 ts_tablet_manager.cc:1918] T ae0efe52ac6c40fbae45c725b07b03a3 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.220492 14823 log.cc:1198] T ae0efe52ac6c40fbae45c725b07b03a3 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/ae0efe52ac6c40fbae45c725b07b03a3
I20250115 03:06:18.220628 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 4c4ce5c2921341ce837ee4f5f7d131b7
I20250115 03:06:18.220718 14823 ts_tablet_manager.cc:1939] T ae0efe52ac6c40fbae45c725b07b03a3 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.221191 14420 raft_consensus.cc:2463] T 074fdc09e4e24078ad0bb588b32ff24f P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:18.221223 14811 tablet_bootstrap.cc:492] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:18.221333 14811 ts_tablet_manager.cc:1397] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.053s user 0.002s sys 0.000s
I20250115 03:06:18.221350 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c4ce5c2921341ce837ee4f5f7d131b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.221465 14628 leader_election.cc:304] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:18.221482 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c4ce5c2921341ce837ee4f5f7d131b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.221778 14811 raft_consensus.cc:357] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.221886 14811 raft_consensus.cc:383] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.221920 14811 raft_consensus.cc:738] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.221969 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet ae0efe52ac6c40fbae45c725b07b03a3 (table default.Table_6b67142e0bf14518ac39cdebdd31f278 [id=6492de6e91c74ac6b2896a3cc81061dc]) successfully deleted
I20250115 03:06:18.222024 14811 consensus_queue.cc:260] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:18.222288 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 4c4ce5c2921341ce837ee4f5f7d131b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4c4ce5c2921341ce837ee4f5f7d131b7 already in progress: creating tablet
I20250115 03:06:18.222302 14811 ts_tablet_manager.cc:1428] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:18.222474 14811 tablet_bootstrap.cc:492] T d895f75911b7456a88e7a5159db7d60b P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:18.223330 14811 tablet_bootstrap.cc:654] T d895f75911b7456a88e7a5159db7d60b P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.224040 16001 raft_consensus.cc:2798] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:18.224125 16001 raft_consensus.cc:695] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:18.224223 16001 consensus_queue.cc:237] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.226370 14809 tablet_bootstrap.cc:492] T d2220e7ec26b4708b2c7a2540c053dcf P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:18.226555 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f889454d62ab418fabd1240a45fa959f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:18.226492 14809 ts_tablet_manager.cc:1397] T d2220e7ec26b4708b2c7a2540c053dcf P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.062s user 0.001s sys 0.001s
W20250115 03:06:18.226876 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f889454d62ab418fabd1240a45fa959f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.227306 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_174d0415fa854efd9c7dd3405d04a414"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.227582 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_174d0415fa854efd9c7dd3405d04a414 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.228206 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e73230225df744a2a11f246bea89b8b0" } modify_external_catalogs: true
I20250115 03:06:18.228300 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e73230225df744a2a11f246bea89b8b0" } modify_external_catalogs: true
I20250115 03:06:18.228327 14811 tablet_bootstrap.cc:492] T d895f75911b7456a88e7a5159db7d60b P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:18.228397 14811 ts_tablet_manager.cc:1397] T d895f75911b7456a88e7a5159db7d60b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20250115 03:06:18.228782 14811 raft_consensus.cc:357] T d895f75911b7456a88e7a5159db7d60b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.228950 14811 raft_consensus.cc:383] T d895f75911b7456a88e7a5159db7d60b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.228993 14811 raft_consensus.cc:738] T d895f75911b7456a88e7a5159db7d60b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.228876 14809 raft_consensus.cc:357] T d2220e7ec26b4708b2c7a2540c053dcf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.229131 14811 consensus_queue.cc:260] T d895f75911b7456a88e7a5159db7d60b P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.229387 14738 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:18.229383 14811 ts_tablet_manager.cc:1428] T d895f75911b7456a88e7a5159db7d60b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.229557 14811 tablet_bootstrap.cc:492] T b6175f691fec45bf9853ee90959a1c98 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:18.229350 14809 raft_consensus.cc:383] T d2220e7ec26b4708b2c7a2540c053dcf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.229871 14809 raft_consensus.cc:738] T d2220e7ec26b4708b2c7a2540c053dcf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.230015 14853 raft_consensus.cc:491] T dcdb5282e41c43dbb235ae78465d19f9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.230086 14853 raft_consensus.cc:513] T dcdb5282e41c43dbb235ae78465d19f9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.230387 14853 leader_election.cc:290] T dcdb5282e41c43dbb235ae78465d19f9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:18.230422 14811 tablet_bootstrap.cc:654] T b6175f691fec45bf9853ee90959a1c98 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.230473 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "074fdc09e4e24078ad0bb588b32ff24f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:18.230793 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcdb5282e41c43dbb235ae78465d19f9" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:18.231581 14812 tablet_bootstrap.cc:492] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:18.231671 14812 ts_tablet_manager.cc:1397] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20250115 03:06:18.230425 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "074fdc09e4e24078ad0bb588b32ff24f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:18.232091 14524 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c4ce5c2921341ce837ee4f5f7d131b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.232077 14812 raft_consensus.cc:357] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.232160 14812 raft_consensus.cc:383] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.232189 14812 raft_consensus.cc:738] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.232287 14812 consensus_queue.cc:260] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.232553 14812 ts_tablet_manager.cc:1428] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.232602 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ccc15a33d119445e810777ee918a6541"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:18.232708 14812 tablet_bootstrap.cc:492] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:18.232874 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ccc15a33d119445e810777ee918a6541 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.233634 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcdb5282e41c43dbb235ae78465d19f9" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:18.233995 14493 leader_election.cc:304] T dcdb5282e41c43dbb235ae78465d19f9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:18.235451 14853 raft_consensus.cc:2743] T dcdb5282e41c43dbb235ae78465d19f9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:18.237053 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2cb3931be82e48859ff5afc497e38338"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.237350 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2cb3931be82e48859ff5afc497e38338 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.237460 14809 consensus_queue.cc:260] T d2220e7ec26b4708b2c7a2540c053dcf P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.237846 14811 tablet_bootstrap.cc:492] T b6175f691fec45bf9853ee90959a1c98 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:18.237918 14811 ts_tablet_manager.cc:1397] T b6175f691fec45bf9853ee90959a1c98 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20250115 03:06:18.238030 14809 ts_tablet_manager.cc:1428] T d2220e7ec26b4708b2c7a2540c053dcf P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.011s user 0.001s sys 0.000s
I20250115 03:06:18.238276 14811 raft_consensus.cc:357] T b6175f691fec45bf9853ee90959a1c98 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.238361 14811 raft_consensus.cc:383] T b6175f691fec45bf9853ee90959a1c98 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.238389 14811 raft_consensus.cc:738] T b6175f691fec45bf9853ee90959a1c98 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.238497 14811 consensus_queue.cc:260] T b6175f691fec45bf9853ee90959a1c98 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.238736 14811 ts_tablet_manager.cc:1428] T b6175f691fec45bf9853ee90959a1c98 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.238896 14811 tablet_bootstrap.cc:492] T 38c9e64e1f5e4f2394afa876553357bf P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
W20250115 03:06:18.239648 14626 leader_election.cc:343] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:18.239750 14811 tablet_bootstrap.cc:654] T 38c9e64e1f5e4f2394afa876553357bf P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:18.239866 14626 leader_election.cc:343] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:18.240309 14524 tablet_service.cc:1514] Processing DeleteTablet for tablet 341180618b4748eabc568888e2e6a7df with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.241230 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_9c49fab0c4db42e1b04b2703b0f3851a" } new_table_name: "default.Table_ec52a51cbc9e40f9bd1f983f932c4698" modify_external_catalogs: true
I20250115 03:06:18.241386 14809 tablet_bootstrap.cc:492] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:18.241647 14811 tablet_bootstrap.cc:492] T 38c9e64e1f5e4f2394afa876553357bf P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:18.241724 14811 ts_tablet_manager.cc:1397] T 38c9e64e1f5e4f2394afa876553357bf P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:18.242111 14811 raft_consensus.cc:357] T 38c9e64e1f5e4f2394afa876553357bf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.242203 14811 raft_consensus.cc:383] T 38c9e64e1f5e4f2394afa876553357bf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.242231 14811 raft_consensus.cc:738] T 38c9e64e1f5e4f2394afa876553357bf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.242332 14811 consensus_queue.cc:260] T 38c9e64e1f5e4f2394afa876553357bf P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.242599 14811 ts_tablet_manager.cc:1428] T 38c9e64e1f5e4f2394afa876553357bf P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.242743 14811 tablet_bootstrap.cc:492] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:18.243286 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e69034f6c27e4ba8bccd3bfa8d47428e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.243553 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e69034f6c27e4ba8bccd3bfa8d47428e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.243574 14811 tablet_bootstrap.cc:654] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.244225 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e14dfeec5b704385b2947bebe118adae" } new_table_name: "default.Table_019ba03032a745be836deba6e091013e" modify_external_catalogs: true
W20250115 03:06:18.246573 14356 consensus_peers.cc:487] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:18.246743 14812 tablet_bootstrap.cc:654] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.248217 14812 tablet_bootstrap.cc:492] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:18.248294 14812 ts_tablet_manager.cc:1397] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20250115 03:06:18.248637 14812 raft_consensus.cc:357] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.248723 14812 raft_consensus.cc:383] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.248765 14812 raft_consensus.cc:738] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.248867 14812 consensus_queue.cc:260] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.249132 14812 ts_tablet_manager.cc:1428] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.249328 14812 tablet_bootstrap.cc:492] T 69533c30b57e45dab0b00239dfd90f13 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:18.250334 14171 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_169e728924b749e4a9118c12d9b56eab" } new_table_name: "default.Table_941290b0882541e185286447fca362b9" modify_external_catalogs: true
W20250115 03:06:18.252422 14626 consensus_peers.cc:487] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:18.253576 14812 tablet_bootstrap.cc:654] T 69533c30b57e45dab0b00239dfd90f13 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.255386 14168 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet d895f75911b7456a88e7a5159db7d60b (table default.Table_6bb3ca6745554454a044c329e6b7e8d3 [id=4b6ce1381c024498a78d91e346f488f2]). Expected version 1 got 0
I20250115 03:06:18.260594 14820 ts_tablet_manager.cc:1918] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20250115 03:06:18.261765 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 85fcafa24fa1448c995d30dd93b0737f consensus state has no leader
W20250115 03:06:18.262009 14162 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4535ac88f78f48da83d9d2f951afdfa6 consensus state has no leader
I20250115 03:06:18.261973 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_8688ed8444804f65b144ec7ac48c5e73"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.262230 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8688ed8444804f65b144ec7ac48c5e73 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.261200 14820 log.cc:1198] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/a75b9401cb4b4f8b92a4779c3704eb46
I20250115 03:06:18.263238 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_9194fcb0d915415ea9220cc06cb2a37d" } new_table_name: "default.Table_68d93433744a4618ba532d361af70f09" modify_external_catalogs: true
I20250115 03:06:18.263612 14809 tablet_bootstrap.cc:654] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.265102 14820 ts_tablet_manager.cc:1939] T a75b9401cb4b4f8b92a4779c3704eb46 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
W20250115 03:06:18.266788 14171 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 238e68300f304266aaed94bc3b76a01d consensus state has no leader
I20250115 03:06:18.268659 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_4e4608a627b54f5e9d0acf5b1105de52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.268956 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4e4608a627b54f5e9d0acf5b1105de52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.271098 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet a75b9401cb4b4f8b92a4779c3704eb46 (table default.Table_a0c249d69b904390952a780b10e1369b [id=22d05297fb9545aeab39b5b62248d0d4]) successfully deleted
I20250115 03:06:18.272234 14820 tablet_replica.cc:331] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
W20250115 03:06:18.273226 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bfeafbf380df41738ebbcc11c19a7f85 consensus state has no leader
I20250115 03:06:18.274871 14820 raft_consensus.cc:2238] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.275663 14820 raft_consensus.cc:2267] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.277828 14165 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 58130f2b27b745d08c0002ac96af6d72 (table default.Table_fe2cfc8f0c564170b3c291cab294193f [id=6aa86d8861174b9091f29b0cfd39fa55]). Expected version 2 got 0
I20250115 03:06:18.277935 14165 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet d895f75911b7456a88e7a5159db7d60b (table default.Table_6bb3ca6745554454a044c329e6b7e8d3 [id=4b6ce1381c024498a78d91e346f488f2]). Expected version 1 got 0
I20250115 03:06:18.278012 14165 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 074fdc09e4e24078ad0bb588b32ff24f (table default.Table_87c206116f514479ab9de8cd4a05817f [id=eaddac40b10648b1933b121cc70d68e8]). Expected version 1 got 0
I20250115 03:06:18.278133 14165 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 7c503b29cb2843338fb1870a89aee443 (table default.Table_dfcccd3390fa4814a0a04bb1ff5e43fa [id=f0a7068682b9481a82fa34cb54ed528e]). Expected version 1 got 0
I20250115 03:06:18.278522 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_6bb3ca6745554454a044c329e6b7e8d3 [id=4b6ce1381c024498a78d91e346f488f2] alter complete (version 1)
I20250115 03:06:18.278257 14820 ts_tablet_manager.cc:1905] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.279479 14819 ts_tablet_manager.cc:1918] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.279932 14819 log.cc:1198] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/41b22aa85ce34b38b0d6a5e2d38f0a1c
I20250115 03:06:18.280910 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_bf4a2c34cc7d42acabda873e7d47d3fb" } new_table_name: "default.Table_28d056349ee04f339900b307997bca9a" modify_external_catalogs: true
I20250115 03:06:18.281438 14853 raft_consensus.cc:491] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.281517 14853 raft_consensus.cc:513] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.281836 14853 leader_election.cc:290] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:18.282527 14493 leader_election.cc:343] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 6110b6c4911d4932b4cac4d86598a5c6
W20250115 03:06:18.282655 14493 leader_election.cc:343] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 6110b6c4911d4932b4cac4d86598a5c6
I20250115 03:06:18.282691 14493 leader_election.cc:304] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:18.282785 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 074fdc09e4e24078ad0bb588b32ff24f (table default.Table_87c206116f514479ab9de8cd4a05817f [id=eaddac40b10648b1933b121cc70d68e8]). Expected version 1 got 0
I20250115 03:06:18.282959 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5780da67ce984297ba33d3c6901c9e3a (table default.Table_551ca9fefcb74ed5aa29803f19de6ab6 [id=79eb8c9694b74a31a4253b33d064e230]). Expected version 1 got 0
I20250115 03:06:18.283010 14853 raft_consensus.cc:2743] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:18.287364 15617 tablet.cc:1721] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:18.288355 15970 tablet.cc:1721] T 5780da67ce984297ba33d3c6901c9e3a P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20250115 03:06:18.289151 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bf6199cdd6e940d1970dd43d8090100d consensus state has no leader
I20250115 03:06:18.289592 14819 ts_tablet_manager.cc:1939] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.291652 14885 consensus_queue.cc:1035] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:18.296867 14168 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 38c9e64e1f5e4f2394afa876553357bf (table default.Table_57e91bade4c54372abe1dbba44434098 [id=91731a4895d4478fad68b1bcbdc45d5d]). Expected version 2 got 0
I20250115 03:06:18.297777 14167 catalog_manager.cc:5526] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:18.297956 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 7c503b29cb2843338fb1870a89aee443 (table default.Table_dfcccd3390fa4814a0a04bb1ff5e43fa [id=f0a7068682b9481a82fa34cb54ed528e]). Expected version 1 got 0
I20250115 03:06:18.298063 14167 catalog_manager.cc:5526] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:18.298202 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 074fdc09e4e24078ad0bb588b32ff24f (table default.Table_87c206116f514479ab9de8cd4a05817f [id=eaddac40b10648b1933b121cc70d68e8]). Expected version 1 got 0
I20250115 03:06:18.298383 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 5780da67ce984297ba33d3c6901c9e3a (table default.Table_551ca9fefcb74ed5aa29803f19de6ab6 [id=79eb8c9694b74a31a4253b33d064e230]). Expected version 1 got 0
I20250115 03:06:18.298492 14167 catalog_manager.cc:5526] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:18.298633 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 8b0f0539530643f78207e528c39530af (table default.Table_f7799772605a4284a062cd0e4ae15ed3 [id=fe357a2a64fc4531870b305eb2480df1]). Expected version 1 got 0
I20250115 03:06:18.299311 14165 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_fe2cfc8f0c564170b3c291cab294193f [id=6aa86d8861174b9091f29b0cfd39fa55] alter complete (version 2)
I20250115 03:06:18.299964 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet d2220e7ec26b4708b2c7a2540c053dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.300480 14820 ts_tablet_manager.cc:1918] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.300633 14820 log.cc:1198] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/90277f7d279e4c7a83c43f40766cd11b
I20250115 03:06:18.301012 14820 ts_tablet_manager.cc:1939] T 90277f7d279e4c7a83c43f40766cd11b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:18.301501 14820 tablet_replica.cc:331] T b5e7672ebc4442369156d12695261269 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:18.301514 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 90277f7d279e4c7a83c43f40766cd11b (table default.Table_7a132f92dbad4a868f47b81f0107d42e [id=a888fcc40089456fba95b6e634ae85f7]) successfully deleted
I20250115 03:06:18.301771 14820 raft_consensus.cc:2238] T b5e7672ebc4442369156d12695261269 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.302505 14819 tablet_replica.cc:331] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.302595 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 41b22aa85ce34b38b0d6a5e2d38f0a1c (table default.Table_67d007791e734b749420eb72d54c0c69 [id=588a068527c446e9aa2d1910ac9dca64]) successfully deleted
I20250115 03:06:18.302690 14819 raft_consensus.cc:2238] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:18.302866 14819 raft_consensus.cc:2267] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.306988 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0e5754dbfde84156b74d86bba62a0046" } modify_external_catalogs: true
I20250115 03:06:18.307093 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0e5754dbfde84156b74d86bba62a0046" } modify_external_catalogs: true
I20250115 03:06:18.308843 14853 raft_consensus.cc:491] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.308962 14853 raft_consensus.cc:513] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.309309 14853 leader_election.cc:290] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:18.309799 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bffef32339a4aa6b2ef7fb5661d035c" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:18.310035 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bffef32339a4aa6b2ef7fb5661d035c" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:18.311147 15325 consensus_queue.cc:1035] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:18.311693 14493 leader_election.cc:304] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:18.312319 14853 raft_consensus.cc:2743] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:18.313287 14626 consensus_peers.cc:487] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:18.314424 14809 tablet_bootstrap.cc:492] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:18.314559 14809 ts_tablet_manager.cc:1397] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.075s user 0.002s sys 0.000s
I20250115 03:06:18.314846 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet c84127dd93bc47ada551f93295ead47b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.315004 14823 tablet_replica.cc:331] T c84127dd93bc47ada551f93295ead47b P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.315131 14823 raft_consensus.cc:2238] T c84127dd93bc47ada551f93295ead47b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.315233 14823 raft_consensus.cc:2267] T c84127dd93bc47ada551f93295ead47b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.315204 14809 raft_consensus.cc:357] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.315305 14809 raft_consensus.cc:383] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.315340 14809 raft_consensus.cc:738] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.315735 14809 consensus_queue.cc:260] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.315979 14823 ts_tablet_manager.cc:1905] T c84127dd93bc47ada551f93295ead47b P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:18.316005 14356 consensus_peers.cc:487] T 8efd181c42eb42628a9f36fad4fa8d64 P e8df7247ee9c43e0aa18216da2661f75 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:18.317595 14809 ts_tablet_manager.cc:1428] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:18.317775 14809 tablet_bootstrap.cc:492] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:18.318784 14809 tablet_bootstrap.cc:654] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.319859 14819 ts_tablet_manager.cc:1905] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.321986 14166 master_service.cc:946] ReplaceTablet: received request to replace tablet 41b22aa85ce34b38b0d6a5e2d38f0a1c from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:18.322608 14557 raft_consensus.cc:3054] T dcdb5282e41c43dbb235ae78465d19f9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.323197 14823 ts_tablet_manager.cc:1918] T c84127dd93bc47ada551f93295ead47b P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.323251 14823 log.cc:1198] T c84127dd93bc47ada551f93295ead47b P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/c84127dd93bc47ada551f93295ead47b
I20250115 03:06:18.323438 14823 ts_tablet_manager.cc:1939] T c84127dd93bc47ada551f93295ead47b P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.324498 16021 tablet.cc:1721] T dcdb5282e41c43dbb235ae78465d19f9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:18.324553 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet c84127dd93bc47ada551f93295ead47b (table default.Table_121a1b9da2674287a03165a67467717f [id=9391b8f0b8a144f592e4aeaef70fbeef]) successfully deleted
I20250115 03:06:18.325862 14820 raft_consensus.cc:2267] T b5e7672ebc4442369156d12695261269 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.328414 14166 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_098c9884c0494d4592f142b4a197f38c" } modify_external_catalogs: true
I20250115 03:06:18.328529 14166 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_098c9884c0494d4592f142b4a197f38c" } modify_external_catalogs: true
I20250115 03:06:18.328645 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_42bcad362d394386967a7e1d627bd77c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.328998 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_42bcad362d394386967a7e1d627bd77c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.330134 14820 ts_tablet_manager.cc:1905] T b5e7672ebc4442369156d12695261269 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.330411 14164 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet dcdb5282e41c43dbb235ae78465d19f9 (table default.Table_375bb4e2095c4675b734247fabacbec7 [id=b298dce7ab664327893633f821ed4d89]). Expected version 1 got 0
I20250115 03:06:18.340646 14820 ts_tablet_manager.cc:1918] T b5e7672ebc4442369156d12695261269 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.341156 14819 ts_tablet_manager.cc:1918] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:18.343410 14165 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_551ca9fefcb74ed5aa29803f19de6ab6 [id=79eb8c9694b74a31a4253b33d064e230] alter complete (version 1)
I20250115 03:06:18.343577 14819 log.cc:1198] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/4d4b8d4ed9f44112b83c6ff94aee2206
W20250115 03:06:18.346179 14358 consensus_peers.cc:487] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 -> Peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Couldn't send request to peer 5c0dae7aff4b44c699275b95ba639770. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ae0efe52ac6c40fbae45c725b07b03a3. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:18.346398 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_1ed9a3074bb7436797358399890084b7" } modify_external_catalogs: true
I20250115 03:06:18.346565 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_1ed9a3074bb7436797358399890084b7" } modify_external_catalogs: true
I20250115 03:06:18.346995 14819 ts_tablet_manager.cc:1939] T 4d4b8d4ed9f44112b83c6ff94aee2206 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.348583 14820 log.cc:1198] T b5e7672ebc4442369156d12695261269 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/b5e7672ebc4442369156d12695261269
I20250115 03:06:18.348707 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet c84127dd93bc47ada551f93295ead47b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.350519 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 4d4b8d4ed9f44112b83c6ff94aee2206 (table default.Table_b900c4395de448a08b0285a5807f2fdf [id=adfd7e86aa0f48e5b5e36b4113517dba]) successfully deleted
I20250115 03:06:18.350773 14819 tablet_replica.cc:331] T 3007a0506ce14c6c89c600b98a81b168 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.352890 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 12b78b3abfe84714a97ecc89f2fb9e33 (DEFAULT_TABLE table=default.Table_f7de4dae6ab44600a02d5b8f92eafedd [id=21f87ba57c0c4cba844d3c4b883513c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.353173 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b78b3abfe84714a97ecc89f2fb9e33. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.353183 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 12b78b3abfe84714a97ecc89f2fb9e33 (DEFAULT_TABLE table=default.Table_f7de4dae6ab44600a02d5b8f92eafedd [id=21f87ba57c0c4cba844d3c4b883513c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.353389 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b78b3abfe84714a97ecc89f2fb9e33. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.353744 14524 tablet_service.cc:1467] Processing CreateTablet for tablet 12b78b3abfe84714a97ecc89f2fb9e33 (DEFAULT_TABLE table=default.Table_f7de4dae6ab44600a02d5b8f92eafedd [id=21f87ba57c0c4cba844d3c4b883513c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.354000 14524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b78b3abfe84714a97ecc89f2fb9e33. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.354496 14664 tablet_service.cc:1467] Processing CreateTablet for tablet cc01ad473e404e8ba26fd4cf7b0799f6 (DEFAULT_TABLE table=default.Table_69f57657cf714422b33241ef1f1951c7 [id=a23108fe271c46a1a3a4f4f785bc420d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.354734 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc01ad473e404e8ba26fd4cf7b0799f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.355309 14400 tablet_service.cc:1467] Processing CreateTablet for tablet ec0f3424ebcb442597a94ccdbb25eeb6 (DEFAULT_TABLE table=default.Table_ba948e2cc56b4023a05595735e5a1434 [id=c80c26973339459e826e9ecc70343fe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.355309 14669 tablet_service.cc:1467] Processing CreateTablet for tablet ec0f3424ebcb442597a94ccdbb25eeb6 (DEFAULT_TABLE table=default.Table_ba948e2cc56b4023a05595735e5a1434 [id=c80c26973339459e826e9ecc70343fe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.355509 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec0f3424ebcb442597a94ccdbb25eeb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.355509 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec0f3424ebcb442597a94ccdbb25eeb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.356088 14522 tablet_service.cc:1467] Processing CreateTablet for tablet 9ab3c40051ad42f692cebbd96144b60a (DEFAULT_TABLE table=default.Table_a8dbae8770744751986252a5d31eb679 [id=294a043ab6c9421fba79d1f04d3be93b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.356312 14522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ab3c40051ad42f692cebbd96144b60a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.357399 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 9ab3c40051ad42f692cebbd96144b60a (DEFAULT_TABLE table=default.Table_a8dbae8770744751986252a5d31eb679 [id=294a043ab6c9421fba79d1f04d3be93b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.357399 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 9ab3c40051ad42f692cebbd96144b60a (DEFAULT_TABLE table=default.Table_a8dbae8770744751986252a5d31eb679 [id=294a043ab6c9421fba79d1f04d3be93b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.357601 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ab3c40051ad42f692cebbd96144b60a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.357601 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ab3c40051ad42f692cebbd96144b60a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.358381 14518 tablet_service.cc:1467] Processing CreateTablet for tablet b9430861457247d9a324b43bd83fbae0 (DEFAULT_TABLE table=default.Table_d1185d5f9bb6408585b742d6b55fec52 [id=8e25d7d7bd99482788f22e3697de3929]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.358621 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9430861457247d9a324b43bd83fbae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.359449 14400 tablet_service.cc:1467] Processing CreateTablet for tablet b9430861457247d9a324b43bd83fbae0 (DEFAULT_TABLE table=default.Table_d1185d5f9bb6408585b742d6b55fec52 [id=8e25d7d7bd99482788f22e3697de3929]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.359442 14663 tablet_service.cc:1467] Processing CreateTablet for tablet b9430861457247d9a324b43bd83fbae0 (DEFAULT_TABLE table=default.Table_d1185d5f9bb6408585b742d6b55fec52 [id=8e25d7d7bd99482788f22e3697de3929]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.359647 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9430861457247d9a324b43bd83fbae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.359647 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9430861457247d9a324b43bd83fbae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.360615 14521 tablet_service.cc:1467] Processing CreateTablet for tablet cc01ad473e404e8ba26fd4cf7b0799f6 (DEFAULT_TABLE table=default.Table_69f57657cf714422b33241ef1f1951c7 [id=a23108fe271c46a1a3a4f4f785bc420d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.360769 14819 raft_consensus.cc:2238] T 3007a0506ce14c6c89c600b98a81b168 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.360854 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc01ad473e404e8ba26fd4cf7b0799f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.361009 14819 raft_consensus.cc:2267] T 3007a0506ce14c6c89c600b98a81b168 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.362915 14520 tablet_service.cc:1467] Processing CreateTablet for tablet ec0f3424ebcb442597a94ccdbb25eeb6 (DEFAULT_TABLE table=default.Table_ba948e2cc56b4023a05595735e5a1434 [id=c80c26973339459e826e9ecc70343fe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.363132 14520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec0f3424ebcb442597a94ccdbb25eeb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.365005 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 4535ac88f78f48da83d9d2f951afdfa6 (DEFAULT_TABLE table=default.Table_019ba03032a745be836deba6e091013e [id=1f8b8805819040a9935767a2f7b49fdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.365214 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4535ac88f78f48da83d9d2f951afdfa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.365458 14520 tablet_service.cc:1467] Processing CreateTablet for tablet 4535ac88f78f48da83d9d2f951afdfa6 (DEFAULT_TABLE table=default.Table_019ba03032a745be836deba6e091013e [id=1f8b8805819040a9935767a2f7b49fdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.365641 14520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4535ac88f78f48da83d9d2f951afdfa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.367292 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet df44c5f5a9ae4d1aa7647dcbe2303a39
I20250115 03:06:18.368108 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet df44c5f5a9ae4d1aa7647dcbe2303a39 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.368235 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet df44c5f5a9ae4d1aa7647dcbe2303a39 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.368414 14823 tablet_replica.cc:331] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.368539 14823 raft_consensus.cc:2238] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:18.368629 14819 ts_tablet_manager.cc:1905] T 3007a0506ce14c6c89c600b98a81b168 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.368714 14823 raft_consensus.cc:2267] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.369279 14823 ts_tablet_manager.cc:1905] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.368136 14820 ts_tablet_manager.cc:1939] T b5e7672ebc4442369156d12695261269 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:18.370718 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_30324f936db349899fc29c4213990c43"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.371057 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_30324f936db349899fc29c4213990c43 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.371263 14823 ts_tablet_manager.cc:1918] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:18.371340 14823 log.cc:1198] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/df44c5f5a9ae4d1aa7647dcbe2303a39
I20250115 03:06:18.371574 14823 ts_tablet_manager.cc:1939] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.372756 14819 ts_tablet_manager.cc:1918] T 3007a0506ce14c6c89c600b98a81b168 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.372822 14819 log.cc:1198] T 3007a0506ce14c6c89c600b98a81b168 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/3007a0506ce14c6c89c600b98a81b168
I20250115 03:06:18.373039 14819 ts_tablet_manager.cc:1939] T 3007a0506ce14c6c89c600b98a81b168 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.373613 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 3007a0506ce14c6c89c600b98a81b168 (table default.Table_cf7f3fbd11804e94848a7c5357169a86 [id=d64515775f264a419768662b8ce0a854]) successfully deleted
I20250115 03:06:18.373749 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet df44c5f5a9ae4d1aa7647dcbe2303a39 (table default.Table_098c9884c0494d4592f142b4a197f38c [id=8bf1da8d633940258affd0e11c334786]) successfully deleted
I20250115 03:06:18.373811 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet b5e7672ebc4442369156d12695261269 (table default.Table_c28b01bcd9784e5ead9f2735dec17850 [id=ab8607ff8b974b59b7a168cf5897ed9f]) successfully deleted
I20250115 03:06:18.374145 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 4535ac88f78f48da83d9d2f951afdfa6 (DEFAULT_TABLE table=default.Table_019ba03032a745be836deba6e091013e [id=1f8b8805819040a9935767a2f7b49fdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.374348 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4535ac88f78f48da83d9d2f951afdfa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.374840 14820 tablet_replica.cc:331] T 13ce129d8a9245baab7c772f0f5fbaa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:18.374969 14820 raft_consensus.cc:2238] T 13ce129d8a9245baab7c772f0f5fbaa7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.375070 14820 raft_consensus.cc:2267] T 13ce129d8a9245baab7c772f0f5fbaa7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.375420 14820 ts_tablet_manager.cc:1905] T 13ce129d8a9245baab7c772f0f5fbaa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.376171 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet df44c5f5a9ae4d1aa7647dcbe2303a39 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.376827 14400 tablet_service.cc:1467] Processing CreateTablet for tablet cc01ad473e404e8ba26fd4cf7b0799f6 (DEFAULT_TABLE table=default.Table_69f57657cf714422b33241ef1f1951c7 [id=a23108fe271c46a1a3a4f4f785bc420d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.377135 14520 tablet_service.cc:1467] Processing CreateTablet for tablet b142fa43a7734d1fb6e3d04bc720a74f (DEFAULT_TABLE table=default.Table_ae52ad4b70d5436fac2aee0b178ee45e [id=d02d602c3d444de2bb2f98c2d3c693ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.377406 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc01ad473e404e8ba26fd4cf7b0799f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.378459 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 9bb0b7a8acfd40738abcc951936b1116
I20250115 03:06:18.378764 14170 catalog_manager.cc:6596] ReplaceTablet: tablet 9bb0b7a8acfd40738abcc951936b1116 deleted and replaced by tablet 00aad81f4ca3427cb683de71148cff5c
I20250115 03:06:18.379015 14520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b142fa43a7734d1fb6e3d04bc720a74f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.379006 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet adbb0ae719f24dcc96709a83981e213d
I20250115 03:06:18.379101 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bb0b7a8acfd40738abcc951936b1116 with delete_type TABLET_DATA_DELETED (replaced by tablet 00aad81f4ca3427cb683de71148cff5c) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.379969 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bb0b7a8acfd40738abcc951936b1116 with delete_type TABLET_DATA_DELETED (replaced by tablet 00aad81f4ca3427cb683de71148cff5c) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.380098 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet adbb0ae719f24dcc96709a83981e213d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.380453 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e419aadbc76442dd857e70a764ec3c44"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.380782 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e419aadbc76442dd857e70a764ec3c44 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.381003 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet adbb0ae719f24dcc96709a83981e213d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.381181 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet cff95c69fb9045cf9ec0ebb128c9ef6b
I20250115 03:06:18.381332 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bb0b7a8acfd40738abcc951936b1116 with delete_type TABLET_DATA_DELETED (replaced by tablet 00aad81f4ca3427cb683de71148cff5c) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.381879 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet cff95c69fb9045cf9ec0ebb128c9ef6b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.382093 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet cff95c69fb9045cf9ec0ebb128c9ef6b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:18.382138 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 9bb0b7a8acfd40738abcc951936b1116 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9bb0b7a8acfd40738abcc951936b1116 already in progress: creating tablet
W20250115 03:06:18.382273 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet adbb0ae719f24dcc96709a83981e213d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet adbb0ae719f24dcc96709a83981e213d already in progress: creating tablet
I20250115 03:06:18.382480 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet cff95c69fb9045cf9ec0ebb128c9ef6b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.382694 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet d12d1f964d2b486bb703ed98cb2e7949
W20250115 03:06:18.382881 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet cff95c69fb9045cf9ec0ebb128c9ef6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cff95c69fb9045cf9ec0ebb128c9ef6b already in progress: creating tablet
I20250115 03:06:18.383561 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet d12d1f964d2b486bb703ed98cb2e7949 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.383631 14552 raft_consensus.cc:3054] T 38c9e64e1f5e4f2394afa876553357bf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.383731 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet adbb0ae719f24dcc96709a83981e213d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.384050 15834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246: Bootstrap replayed 1/1 log segments. Stats: ops{read=491 overwritten=0 applied=490 ignored=0} inserts{seen=362 ignored=0} mutations{seen=542 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20250115 03:06:18.384409 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet d12d1f964d2b486bb703ed98cb2e7949 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.384750 15834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246: Bootstrap complete.
W20250115 03:06:18.384963 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet d12d1f964d2b486bb703ed98cb2e7949 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d12d1f964d2b486bb703ed98cb2e7949 already in progress: creating tablet
I20250115 03:06:18.386018 15426 tablet.cc:1721] T 38c9e64e1f5e4f2394afa876553357bf P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:18.387033 14861 raft_consensus.cc:491] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.387128 14861 raft_consensus.cc:513] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.387356 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet d12d1f964d2b486bb703ed98cb2e7949 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.387517 14861 leader_election.cc:290] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:18.388127 14358 leader_election.cc:343] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: c84127dd93bc47ada551f93295ead47b
I20250115 03:06:18.392397 14163 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 38c9e64e1f5e4f2394afa876553357bf (table default.Table_57e91bade4c54372abe1dbba44434098 [id=91731a4895d4478fad68b1bcbdc45d5d]). Expected version 2 got 1
I20250115 03:06:18.393182 14819 tablet_replica.cc:331] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.393481 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84127dd93bc47ada551f93295ead47b" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:18.393585 14493 consensus_peers.cc:487] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 -> Peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Couldn't send request to peer e8df7247ee9c43e0aa18216da2661f75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e43ce064a02347efa81ca274d76c6960. This is attempt 1: this message will repeat every 5th retry.
W20250115 03:06:18.393797 14356 leader_election.cc:343] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:18.394196 14356 leader_election.cc:304] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:18.394335 14861 raft_consensus.cc:2743] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:18.394765 14819 raft_consensus.cc:2238] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:18.394814 14554 raft_consensus.cc:3054] T d895f75911b7456a88e7a5159db7d60b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.395076 14819 raft_consensus.cc:2267] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.396039 14820 ts_tablet_manager.cc:1918] T 13ce129d8a9245baab7c772f0f5fbaa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:18.398021 14665 tablet_service.cc:1467] Processing CreateTablet for tablet b142fa43a7734d1fb6e3d04bc720a74f (DEFAULT_TABLE table=default.Table_ae52ad4b70d5436fac2aee0b178ee45e [id=d02d602c3d444de2bb2f98c2d3c693ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.398247 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b142fa43a7734d1fb6e3d04bc720a74f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.398599 14401 tablet_service.cc:1467] Processing CreateTablet for tablet daea48ecc69d441ab66fcb40ea2e7a66 (DEFAULT_TABLE table=default.Table_b922b874a412477f8cf1a410e26beb1d [id=76a2384a834c40a2a8ec940808a94290]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.398826 14820 log.cc:1198] T 13ce129d8a9245baab7c772f0f5fbaa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/13ce129d8a9245baab7c772f0f5fbaa7
I20250115 03:06:18.399019 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daea48ecc69d441ab66fcb40ea2e7a66. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.399369 14820 ts_tablet_manager.cc:1939] T 13ce129d8a9245baab7c772f0f5fbaa7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:18.399539 14819 ts_tablet_manager.cc:1905] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.400194 14663 tablet_service.cc:1467] Processing CreateTablet for tablet daea48ecc69d441ab66fcb40ea2e7a66 (DEFAULT_TABLE table=default.Table_b922b874a412477f8cf1a410e26beb1d [id=76a2384a834c40a2a8ec940808a94290]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.400403 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daea48ecc69d441ab66fcb40ea2e7a66. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.399750 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 293a76d0d49e4a12ab951cbc7311d0f2 (DEFAULT_TABLE table=default.Table_c3e03487af564b27a13203fb3ea3e1de [id=4980aca16b2e49788da70eac4a4b265b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.402237 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 293a76d0d49e4a12ab951cbc7311d0f2 (DEFAULT_TABLE table=default.Table_c3e03487af564b27a13203fb3ea3e1de [id=4980aca16b2e49788da70eac4a4b265b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.402443 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293a76d0d49e4a12ab951cbc7311d0f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.402613 14400 tablet_service.cc:1467] Processing CreateTablet for tablet b142fa43a7734d1fb6e3d04bc720a74f (DEFAULT_TABLE table=default.Table_ae52ad4b70d5436fac2aee0b178ee45e [id=d02d602c3d444de2bb2f98c2d3c693ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.402751 14520 tablet_service.cc:1467] Processing CreateTablet for tablet daea48ecc69d441ab66fcb40ea2e7a66 (DEFAULT_TABLE table=default.Table_b922b874a412477f8cf1a410e26beb1d [id=76a2384a834c40a2a8ec940808a94290]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.403908 15426 tablet.cc:1721] T 38c9e64e1f5e4f2394afa876553357bf P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:18.404074 14165 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet d895f75911b7456a88e7a5159db7d60b (table default.Table_6bb3ca6745554454a044c329e6b7e8d3 [id=4b6ce1381c024498a78d91e346f488f2]). Expected version 1 got 0
I20250115 03:06:18.404227 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b142fa43a7734d1fb6e3d04bc720a74f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.407740 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 13ce129d8a9245baab7c772f0f5fbaa7 (table default.Table_d58c19b62a194b46858ee8955ebf3f82 [id=6cba4fb2f7104c6e953340b04e5dc494]) successfully deleted
I20250115 03:06:18.407883 14820 tablet_replica.cc:331] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:18.408001 14820 raft_consensus.cc:2238] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.408089 14820 raft_consensus.cc:2267] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:18.407994 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet fe8d94c8fecc4077ba052390810c7430 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fe8d94c8fecc4077ba052390810c7430 already in progress: creating tablet
I20250115 03:06:18.408402 14820 ts_tablet_manager.cc:1905] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:18.408646 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 0f3bf849316a4ebd841c3345ca17087e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0f3bf849316a4ebd841c3345ca17087e already in progress: creating tablet
W20250115 03:06:18.408867 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 3b9c09b6315345039cdea9eeee3f181a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3b9c09b6315345039cdea9eeee3f181a already in progress: creating tablet
I20250115 03:06:18.409526 15426 tablet.cc:1721] T d895f75911b7456a88e7a5159db7d60b P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:18.410364 15834 raft_consensus.cc:357] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [term 2 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } }
I20250115 03:06:18.410775 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293a76d0d49e4a12ab951cbc7311d0f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.411370 14603 heartbeater.cc:346] Connected to a master server at 127.13.186.252:41855
I20250115 03:06:18.411726 14819 ts_tablet_manager.cc:1918] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.411803 14819 log.cc:1198] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/ae0efe52ac6c40fbae45c725b07b03a3
I20250115 03:06:18.412025 14819 ts_tablet_manager.cc:1939] T ae0efe52ac6c40fbae45c725b07b03a3 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.412565 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet ae0efe52ac6c40fbae45c725b07b03a3 (table default.Table_6b67142e0bf14518ac39cdebdd31f278 [id=6492de6e91c74ac6b2896a3cc81061dc]) successfully deleted
I20250115 03:06:18.413875 14809 tablet_bootstrap.cc:492] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:18.413956 14809 ts_tablet_manager.cc:1397] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.096s user 0.002s sys 0.000s
I20250115 03:06:18.414355 14809 raft_consensus.cc:357] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.414446 14809 raft_consensus.cc:383] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.414465 14820 ts_tablet_manager.cc:1918] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.414477 14809 raft_consensus.cc:738] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.414520 14820 log.cc:1198] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/6d10731df02f4e9682b8de6962256635
I20250115 03:06:18.414585 14809 consensus_queue.cc:260] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.414860 14809 ts_tablet_manager.cc:1428] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.415019 14809 tablet_bootstrap.cc:492] T 01433c76a398462bb6708a02997047fa P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:18.415730 14809 tablet_bootstrap.cc:654] T 01433c76a398462bb6708a02997047fa P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.416011 14858 raft_consensus.cc:491] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.416078 14858 raft_consensus.cc:513] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.416378 14858 leader_election.cc:290] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:18.417596 14493 leader_election.cc:343] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 341180618b4748eabc568888e2e6a7df
W20250115 03:06:18.417986 14493 leader_election.cc:343] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 341180618b4748eabc568888e2e6a7df
I20250115 03:06:18.418053 14493 leader_election.cc:304] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:18.418344 14858 raft_consensus.cc:2743] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:18.418417 14520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet daea48ecc69d441ab66fcb40ea2e7a66. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.420460 15834 raft_consensus.cc:738] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2db8cff032af46db986bf52b21b52246, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.421061 15834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 490, Last appended: 2.491, Last appended by leader: 491, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } }
I20250115 03:06:18.425993 15834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [sys.catalog]: configured and running, proceeding with master startup.
I20250115 03:06:18.426829 14819 tablet_replica.cc:331] T d2220e7ec26b4708b2c7a2540c053dcf P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.426964 14819 raft_consensus.cc:2238] T d2220e7ec26b4708b2c7a2540c053dcf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.427057 14819 raft_consensus.cc:2267] T d2220e7ec26b4708b2c7a2540c053dcf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:18.427372 14158 catalog_manager.cc:4670] Async tablet task AlterTable RPC for tablet 238e68300f304266aaed94bc3b76a01d (table default.Table_941290b0882541e185286447fca362b9 [id=a2689bcea23843c9bb9e1f08e335e58f], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 238e68300f304266aaed94bc3b76a01d consensus state has no leader
W20250115 03:06:18.427482 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 4c4ce5c2921341ce837ee4f5f7d131b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4c4ce5c2921341ce837ee4f5f7d131b7 already in progress: creating tablet
I20250115 03:06:18.427860 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6b67142e0bf14518ac39cdebdd31f278" } modify_external_catalogs: true
I20250115 03:06:18.427960 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6b67142e0bf14518ac39cdebdd31f278" } modify_external_catalogs: true
I20250115 03:06:18.429348 15792 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" instance_seqno: 1736910370552147) as {username='slave'} at 127.13.186.193:44383; Asking this server to re-register.
I20250115 03:06:18.429996 14468 heartbeater.cc:463] Registering TS with master...
I20250115 03:06:18.431715 15792 ts_manager.cc:194] Registered new tserver with Master: e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:18.432662 14163 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet 20d71366a4a747bcaa18d9ce14d05b91
I20250115 03:06:18.433296 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet cef4442bfcfd4ad4b343a2785c3a9a74 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:18.434952 14820 ts_tablet_manager.cc:1939] T 6d10731df02f4e9682b8de6962256635 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:18.435433 14819 ts_tablet_manager.cc:1905] T d2220e7ec26b4708b2c7a2540c053dcf P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.437335 14819 ts_tablet_manager.cc:1918] T d2220e7ec26b4708b2c7a2540c053dcf P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.437403 14819 log.cc:1198] T d2220e7ec26b4708b2c7a2540c053dcf P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/d2220e7ec26b4708b2c7a2540c053dcf
I20250115 03:06:18.437882 14819 ts_tablet_manager.cc:1939] T d2220e7ec26b4708b2c7a2540c053dcf P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.438477 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet d2220e7ec26b4708b2c7a2540c053dcf (table default.Table_807e7c2704994c25aa03b1267a20f794 [id=fb7018f8e1ac4d6db5c0c114b0e73a07]) successfully deleted
I20250115 03:06:18.438638 14812 tablet_bootstrap.cc:492] T 69533c30b57e45dab0b00239dfd90f13 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:18.438711 14819 tablet_replica.cc:331] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.438737 14812 ts_tablet_manager.cc:1397] T 69533c30b57e45dab0b00239dfd90f13 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.189s user 0.001s sys 0.000s
I20250115 03:06:18.438817 14819 raft_consensus.cc:2238] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.438891 14819 raft_consensus.cc:2267] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.439149 14812 raft_consensus.cc:357] T 69533c30b57e45dab0b00239dfd90f13 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.439247 14812 raft_consensus.cc:383] T 69533c30b57e45dab0b00239dfd90f13 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.439278 14812 raft_consensus.cc:738] T 69533c30b57e45dab0b00239dfd90f13 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.439390 14812 consensus_queue.cc:260] T 69533c30b57e45dab0b00239dfd90f13 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.439618 14819 ts_tablet_manager.cc:1905] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.439664 14812 ts_tablet_manager.cc:1428] T 69533c30b57e45dab0b00239dfd90f13 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.439851 14812 tablet_bootstrap.cc:492] T 0f3bf849316a4ebd841c3345ca17087e P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:18.440709 14812 tablet_bootstrap.cc:654] T 0f3bf849316a4ebd841c3345ca17087e P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.442981 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet cef4442bfcfd4ad4b343a2785c3a9a74
I20250115 03:06:18.443225 14163 catalog_manager.cc:6596] ReplaceTablet: tablet cef4442bfcfd4ad4b343a2785c3a9a74 deleted and replaced by tablet c164db2cf23149258bdc14e68def00b3
I20250115 03:06:18.444367 14858 raft_consensus.cc:491] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.444459 14858 raft_consensus.cc:513] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.444590 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet cef4442bfcfd4ad4b343a2785c3a9a74 with delete_type TABLET_DATA_DELETED (replaced by tablet c164db2cf23149258bdc14e68def00b3) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.444773 14858 leader_election.cc:290] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:18.445151 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_59c02322b76b4ad18a2c6b0ac7438110"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.445477 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_59c02322b76b4ad18a2c6b0ac7438110 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.446136 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet cef4442bfcfd4ad4b343a2785c3a9a74 with delete_type TABLET_DATA_DELETED (replaced by tablet c164db2cf23149258bdc14e68def00b3) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:18.446501 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet cef4442bfcfd4ad4b343a2785c3a9a74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cef4442bfcfd4ad4b343a2785c3a9a74 already in progress: creating tablet
W20250115 03:06:18.447114 14493 leader_election.cc:343] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: d0c3536533e94174bb1e3888a3458395
W20250115 03:06:18.447257 14493 leader_election.cc:343] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: d0c3536533e94174bb1e3888a3458395
I20250115 03:06:18.447307 14493 leader_election.cc:304] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:18.447525 14858 raft_consensus.cc:2743] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:18.447690 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet a1b8b5b7fe03496ea2639dbc9c920974 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:18.449280 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet b8ec24d2937d44e79b839276bf615fa0 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:18.452039 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet a1b8b5b7fe03496ea2639dbc9c920974
I20250115 03:06:18.452293 14168 catalog_manager.cc:6596] ReplaceTablet: tablet a1b8b5b7fe03496ea2639dbc9c920974 deleted and replaced by tablet c07e0a0d439041cdbe089b3de4597098
I20250115 03:06:18.452555 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet a1b8b5b7fe03496ea2639dbc9c920974 with delete_type TABLET_DATA_DELETED (replaced by tablet c07e0a0d439041cdbe089b3de4597098) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.452564 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet a1b8b5b7fe03496ea2639dbc9c920974 with delete_type TABLET_DATA_DELETED (replaced by tablet c07e0a0d439041cdbe089b3de4597098) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.452569 14518 tablet_service.cc:1514] Processing DeleteTablet for tablet cef4442bfcfd4ad4b343a2785c3a9a74 with delete_type TABLET_DATA_DELETED (replaced by tablet c164db2cf23149258bdc14e68def00b3) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.452695 14823 tablet_replica.cc:331] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.452702 14522 tablet_service.cc:1514] Processing DeleteTablet for tablet a1b8b5b7fe03496ea2639dbc9c920974 with delete_type TABLET_DATA_DELETED (replaced by tablet c07e0a0d439041cdbe089b3de4597098) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.452821 14823 raft_consensus.cc:2238] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:18.453015 14823 raft_consensus.cc:2267] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.453763 14823 ts_tablet_manager.cc:1905] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.454942 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet b8ec24d2937d44e79b839276bf615fa0
I20250115 03:06:18.455169 14163 catalog_manager.cc:6596] ReplaceTablet: tablet b8ec24d2937d44e79b839276bf615fa0 deleted and replaced by tablet 4febfae2f2794669b5997a1fa7ef7412
I20250115 03:06:18.455559 14522 tablet_service.cc:1514] Processing DeleteTablet for tablet b8ec24d2937d44e79b839276bf615fa0 with delete_type TABLET_DATA_DELETED (replaced by tablet 4febfae2f2794669b5997a1fa7ef7412) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.455971 14823 ts_tablet_manager.cc:1918] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.456017 14823 log.cc:1198] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/a1b8b5b7fe03496ea2639dbc9c920974
I20250115 03:06:18.456404 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d4d372b6bd40428090e1280d674a8b20"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:18.456643 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet b8ec24d2937d44e79b839276bf615fa0 with delete_type TABLET_DATA_DELETED (replaced by tablet 4febfae2f2794669b5997a1fa7ef7412) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:18.456719 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d4d372b6bd40428090e1280d674a8b20 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.457149 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet b8ec24d2937d44e79b839276bf615fa0 with delete_type TABLET_DATA_DELETED (replaced by tablet 4febfae2f2794669b5997a1fa7ef7412) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.457434 14823 ts_tablet_manager.cc:1939] T a1b8b5b7fe03496ea2639dbc9c920974 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.458148 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet a1b8b5b7fe03496ea2639dbc9c920974 (table default.Table_2773b577ce554a45834f200dca2c2987 [id=8005446109b64adabae59e8612c35f58]) successfully deleted
I20250115 03:06:18.458261 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0c00b03c9b824693aa27df8649e12c7b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.458312 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet b8ec24d2937d44e79b839276bf615fa0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b8ec24d2937d44e79b839276bf615fa0 already in progress: creating tablet
W20250115 03:06:18.458535 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c00b03c9b824693aa27df8649e12c7b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.460531 14819 ts_tablet_manager.cc:1918] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.460783 14819 log.cc:1198] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/c84127dd93bc47ada551f93295ead47b
I20250115 03:06:18.462431 14819 ts_tablet_manager.cc:1939] T c84127dd93bc47ada551f93295ead47b P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.463392 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet c84127dd93bc47ada551f93295ead47b (table default.Table_121a1b9da2674287a03165a67467717f [id=9391b8f0b8a144f592e4aeaef70fbeef]) successfully deleted
I20250115 03:06:18.463631 14819 tablet_replica.cc:331] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.463800 14819 raft_consensus.cc:2238] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.463958 14819 raft_consensus.cc:2267] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.464433 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f2fdb3ede19849979bf289913d7698fe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:18.464557 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d0c0059d665043bda7f63aaae45c079e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.464720 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f2fdb3ede19849979bf289913d7698fe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.464895 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6faeecc09aca49888fb4ccd071352ab7" } new_table_name: "default.Table_91fae283e01841f2a274e89f22ebc882" modify_external_catalogs: true
I20250115 03:06:18.465201 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_c9a7e51bb4ca41eba7b6a942eb649b81" } new_table_name: "default.Table_9e42df4c569c47b698c9bbb6dcf39736" modify_external_catalogs: true
I20250115 03:06:18.465551 16001 raft_consensus.cc:491] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.465627 16001 raft_consensus.cc:513] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.465965 16001 leader_election.cc:290] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
W20250115 03:06:18.466365 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d0c0059d665043bda7f63aaae45c079e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.466702 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3efc2fb8427f493ea933347bec34f88f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:18.466807 14421 raft_consensus.cc:2463] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:18.467000 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3efc2fb8427f493ea933347bec34f88f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:18.467149 14628 leader_election.cc:304] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
W20250115 03:06:18.467334 14626 leader_election.cc:343] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:18.467358 16001 raft_consensus.cc:2798] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:18.467396 16001 raft_consensus.cc:491] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:18.467440 16001 raft_consensus.cc:3054] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.468366 16001 raft_consensus.cc:513] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.468690 16001 leader_election.cc:290] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:18.469339 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3efc2fb8427f493ea933347bec34f88f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:18.469441 14421 raft_consensus.cc:3054] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.469552 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3efc2fb8427f493ea933347bec34f88f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:18.470295 14421 raft_consensus.cc:2463] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:18.470463 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f7799772605a4284a062cd0e4ae15ed3" } new_table_name: "default.Table_d7d37f44d845457e8a3656b4f7deabf2" modify_external_catalogs: true
W20250115 03:06:18.471338 14626 leader_election.cc:343] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:18.471549 14628 leader_election.cc:304] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:18.471851 16001 raft_consensus.cc:2798] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:18.471928 16001 raft_consensus.cc:695] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:18.472038 16001 consensus_queue.cc:237] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.474586 14819 ts_tablet_manager.cc:1905] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.476511 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d407111658854f02aa8387a067f33f37" } new_table_name: "default.Table_1f92c58a90374663a4c6368b14f71d43" modify_external_catalogs: true
I20250115 03:06:18.476629 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_a0656bab78454d5aa134b66287b01eb0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:18.476794 14819 ts_tablet_manager.cc:1918] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:18.476848 14819 log.cc:1198] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/df44c5f5a9ae4d1aa7647dcbe2303a39
I20250115 03:06:18.477401 14819 ts_tablet_manager.cc:1939] T df44c5f5a9ae4d1aa7647dcbe2303a39 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.477828 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0138c3ede76d425b9b0950b0e328c7a6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:18.481738 15970 tablet.cc:1721] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20250115 03:06:18.482100 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0b5fd91a0ea64b3b85b7571e5daa08b2 consensus state has no leader
I20250115 03:06:18.480901 15617 tablet.cc:1721] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:18.486084 16077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "147c86b98f604d518e71f2ba3121fc21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:18.486215 16077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [sys.catalog]: This master's current role is: FOLLOWER
I20250115 03:06:18.486229 16021 tablet.cc:1721] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:18.486959 15791 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" instance_seqno: 1736910370903500) as {username='slave'} at 127.13.186.195:49863; Asking this server to re-register.
I20250115 03:06:18.487663 14738 heartbeater.cc:463] Registering TS with master...
I20250115 03:06:18.489370 15792 ts_manager.cc:194] Registered new tserver with Master: 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:18.490509 14811 tablet_bootstrap.cc:492] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:18.490475 16063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 147c86b98f604d518e71f2ba3121fc21. Latest consensus state: current_term: 2 leader_uuid: "147c86b98f604d518e71f2ba3121fc21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:18.490561 16063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246 [sys.catalog]: This master's current role is: FOLLOWER
I20250115 03:06:18.490604 14811 ts_tablet_manager.cc:1397] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.248s user 0.001s sys 0.000s
I20250115 03:06:18.491055 14811 raft_consensus.cc:357] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.491149 14811 raft_consensus.cc:383] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.491176 14811 raft_consensus.cc:738] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.491278 14811 consensus_queue.cc:260] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.491568 14811 ts_tablet_manager.cc:1428] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.492280 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet df44c5f5a9ae4d1aa7647dcbe2303a39 (table default.Table_098c9884c0494d4592f142b4a197f38c [id=8bf1da8d633940258affd0e11c334786]) successfully deleted
W20250115 03:06:18.492429 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 9bb0b7a8acfd40738abcc951936b1116 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9bb0b7a8acfd40738abcc951936b1116 already in progress: creating tablet
W20250115 03:06:18.492547 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet cff95c69fb9045cf9ec0ebb128c9ef6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cff95c69fb9045cf9ec0ebb128c9ef6b already in progress: creating tablet
I20250115 03:06:18.492653 14819 tablet_replica.cc:331] T a1b8b5b7fe03496ea2639dbc9c920974 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
W20250115 03:06:18.492655 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet d12d1f964d2b486bb703ed98cb2e7949 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d12d1f964d2b486bb703ed98cb2e7949 already in progress: creating tablet
W20250115 03:06:18.492766 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet adbb0ae719f24dcc96709a83981e213d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet adbb0ae719f24dcc96709a83981e213d already in progress: creating tablet
I20250115 03:06:18.492761 14819 raft_consensus.cc:2238] T a1b8b5b7fe03496ea2639dbc9c920974 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.492861 14819 raft_consensus.cc:2267] T a1b8b5b7fe03496ea2639dbc9c920974 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:18.492879 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet cef4442bfcfd4ad4b343a2785c3a9a74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cef4442bfcfd4ad4b343a2785c3a9a74 already in progress: creating tablet
I20250115 03:06:18.493314 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a7ee880814fc40f98518c5149b8b5b4b" } new_table_name: "default.Table_9f442bb6e9024025b7fa97a7ddd069f5" modify_external_catalogs: true
W20250115 03:06:18.494942 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a0656bab78454d5aa134b66287b01eb0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:18.495216 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0138c3ede76d425b9b0950b0e328c7a6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.494868 14164 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_9e42df4c569c47b698c9bbb6dcf39736 [id=e441d6f5acea4837accbfb8e4856f82d] alter complete (version 3)
I20250115 03:06:18.498198 14819 ts_tablet_manager.cc:1905] T a1b8b5b7fe03496ea2639dbc9c920974 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.498306 14853 raft_consensus.cc:491] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.498385 14853 raft_consensus.cc:513] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.498713 14853 leader_election.cc:290] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:18.498936 14422 raft_consensus.cc:1270] T 8b0f0539530643f78207e528c39530af P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 5c0dae7aff4b44c699275b95ba639770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250115 03:06:18.499538 14163 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c6fa98c6fb304dc49b8e0ced989cc936 consensus state has no leader
I20250115 03:06:18.499641 15325 consensus_queue.cc:1035] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:18.499920 14493 leader_election.cc:343] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: bf6f61dc685a48bdb367278773bf1d90
W20250115 03:06:18.500540 14493 leader_election.cc:343] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: bf6f61dc685a48bdb367278773bf1d90
I20250115 03:06:18.500602 14493 leader_election.cc:304] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:18.500856 14858 raft_consensus.cc:2743] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:18.502028 14626 consensus_peers.cc:487] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:18.504489 15617 tablet.cc:1721] T 6784e353df554bb2aa10b667c8b488da P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:18.505756 14819 ts_tablet_manager.cc:1918] T a1b8b5b7fe03496ea2639dbc9c920974 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.506691 14819 log.cc:1198] T a1b8b5b7fe03496ea2639dbc9c920974 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/a1b8b5b7fe03496ea2639dbc9c920974
I20250115 03:06:18.507578 14819 ts_tablet_manager.cc:1939] T a1b8b5b7fe03496ea2639dbc9c920974 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.508028 14557 raft_consensus.cc:3054] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.508846 15970 tablet.cc:1721] T 6784e353df554bb2aa10b667c8b488da P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:18.510993 14811 tablet_bootstrap.cc:492] T 58161a322a3d469c8444d353689bed65 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:18.511329 14820 tablet_replica.cc:331] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:18.511462 14820 raft_consensus.cc:2238] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.511550 14820 raft_consensus.cc:2267] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.511844 14820 ts_tablet_manager.cc:1905] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.511919 14811 tablet_bootstrap.cc:654] T 58161a322a3d469c8444d353689bed65 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.514339 15791 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" instance_seqno: 1736910370740004) as {username='slave'} at 127.13.186.194:49341; Asking this server to re-register.
I20250115 03:06:18.516680 14820 ts_tablet_manager.cc:1918] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.516749 14820 log.cc:1198] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/1c76ab180b26487896f4360efe47b2c7
I20250115 03:06:18.517103 14603 heartbeater.cc:463] Registering TS with master...
I20250115 03:06:18.517182 14820 ts_tablet_manager.cc:1939] T 1c76ab180b26487896f4360efe47b2c7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:18.517706 14853 raft_consensus.cc:491] T b6175f691fec45bf9853ee90959a1c98 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.517774 14853 raft_consensus.cc:513] T b6175f691fec45bf9853ee90959a1c98 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.518067 14853 leader_election.cc:290] T b6175f691fec45bf9853ee90959a1c98 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:18.518139 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet a1b8b5b7fe03496ea2639dbc9c920974 (table default.Table_2773b577ce554a45834f200dca2c2987 [id=8005446109b64adabae59e8612c35f58]) successfully deleted
W20250115 03:06:18.518250 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet b8ec24d2937d44e79b839276bf615fa0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b8ec24d2937d44e79b839276bf615fa0 already in progress: creating tablet
I20250115 03:06:18.518417 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6175f691fec45bf9853ee90959a1c98" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:18.518535 14820 tablet_replica.cc:331] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:18.518643 14820 raft_consensus.cc:2238] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.518743 14820 raft_consensus.cc:2267] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.518985 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 293a76d0d49e4a12ab951cbc7311d0f2 (DEFAULT_TABLE table=default.Table_c3e03487af564b27a13203fb3ea3e1de [id=4980aca16b2e49788da70eac4a4b265b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.519052 14820 ts_tablet_manager.cc:1905] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.519197 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 293a76d0d49e4a12ab951cbc7311d0f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.519322 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 8b0f0539530643f78207e528c39530af (table default.Table_d7d37f44d845457e8a3656b4f7deabf2 [id=fe357a2a64fc4531870b305eb2480df1]). Expected version 2 got 0
I20250115 03:06:18.519815 14397 tablet_service.cc:1467] Processing CreateTablet for tablet d140be0fab694f59b8a535392970fad0 (DEFAULT_TABLE table=default.Table_6df09ae8a979496abf73d384649b81b5 [id=e30f9e0f1aef45b7a1668a6d0f5f42bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.520018 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d140be0fab694f59b8a535392970fad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.520728 14820 ts_tablet_manager.cc:1918] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.520787 14820 log.cc:1198] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/fa08518530ae4d5487af386a650c4672
I20250115 03:06:18.521034 14820 ts_tablet_manager.cc:1939] T fa08518530ae4d5487af386a650c4672 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:18.521335 14663 tablet_service.cc:1467] Processing CreateTablet for tablet d140be0fab694f59b8a535392970fad0 (DEFAULT_TABLE table=default.Table_6df09ae8a979496abf73d384649b81b5 [id=e30f9e0f1aef45b7a1668a6d0f5f42bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.521560 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d140be0fab694f59b8a535392970fad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.521703 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 6d10731df02f4e9682b8de6962256635 (table default.Table_551ca9fefcb74ed5aa29803f19de6ab6 [id=79eb8c9694b74a31a4253b33d064e230]) successfully deleted
I20250115 03:06:18.521764 14820 tablet_replica.cc:331] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
W20250115 03:06:18.521823 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet d2220e7ec26b4708b2c7a2540c053dcf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d2220e7ec26b4708b2c7a2540c053dcf already in progress: creating tablet
I20250115 03:06:18.521867 14820 raft_consensus.cc:2238] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:18.522029 14820 raft_consensus.cc:2267] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:18.522323 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet a3eaec2fff184aad816349fc36fa3db1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a3eaec2fff184aad816349fc36fa3db1 already in progress: creating tablet
I20250115 03:06:18.522323 14520 tablet_service.cc:1467] Processing CreateTablet for tablet d140be0fab694f59b8a535392970fad0 (DEFAULT_TABLE table=default.Table_6df09ae8a979496abf73d384649b81b5 [id=e30f9e0f1aef45b7a1668a6d0f5f42bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.522446 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 1c76ab180b26487896f4360efe47b2c7 (table default.Table_d8f964febee84681a352ac34900c1e6c [id=dde72643e9ec41e290d4896e58742c70]) successfully deleted
I20250115 03:06:18.522532 14520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d140be0fab694f59b8a535392970fad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.522545 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet fa08518530ae4d5487af386a650c4672 (table default.Table_48dcc3ce48c94335a128ff0ce47649ae [id=12b47ada2a2c48f2b6807fa62fcca1a6]) successfully deleted
W20250115 03:06:18.522635 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet a73d45bf459c4137af6961b680d282ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a73d45bf459c4137af6961b680d282ac already in progress: creating tablet
W20250115 03:06:18.522847 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 103f78d0d9fc42308e1e23316895be33 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 103f78d0d9fc42308e1e23316895be33 already in progress: creating tablet
W20250115 03:06:18.522984 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet aa7754f577af4ef6b6fca41e61e5965c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aa7754f577af4ef6b6fca41e61e5965c already in progress: creating tablet
I20250115 03:06:18.523733 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6175f691fec45bf9853ee90959a1c98" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:18.524049 14493 leader_election.cc:304] T b6175f691fec45bf9853ee90959a1c98 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:18.516775 14811 tablet_bootstrap.cc:492] T 58161a322a3d469c8444d353689bed65 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:18.524236 14811 ts_tablet_manager.cc:1397] T 58161a322a3d469c8444d353689bed65 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20250115 03:06:18.524294 14853 raft_consensus.cc:2743] T b6175f691fec45bf9853ee90959a1c98 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:18.524662 14811 raft_consensus.cc:357] T 58161a322a3d469c8444d353689bed65 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.524749 14811 raft_consensus.cc:383] T 58161a322a3d469c8444d353689bed65 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.524775 14811 raft_consensus.cc:738] T 58161a322a3d469c8444d353689bed65 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.524859 14811 consensus_queue.cc:260] T 58161a322a3d469c8444d353689bed65 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.525275 14820 ts_tablet_manager.cc:1905] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.527114 14820 ts_tablet_manager.cc:1918] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:18.527228 14820 log.cc:1198] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/e43ce064a02347efa81ca274d76c6960
I20250115 03:06:18.527493 14820 ts_tablet_manager.cc:1939] T e43ce064a02347efa81ca274d76c6960 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:18.527879 14820 tablet_replica.cc:331] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:18.528031 14820 raft_consensus.cc:2238] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.530412 14811 ts_tablet_manager.cc:1428] T 58161a322a3d469c8444d353689bed65 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
I20250115 03:06:18.530584 14811 tablet_bootstrap.cc:492] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:18.532476 14820 raft_consensus.cc:2267] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.532820 14820 ts_tablet_manager.cc:1905] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.534147 15791 ts_manager.cc:194] Registered new tserver with Master: 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:18.535053 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet e43ce064a02347efa81ca274d76c6960 (table default.Table_4b95b36b8d744822b248adc43a061bb1 [id=0d8deaf903704c00a4a0576354ecfbe6]) successfully deleted
W20250115 03:06:18.535202 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet de14b1d525cc42808500279ae6b60e80 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet de14b1d525cc42808500279ae6b60e80 already in progress: creating tablet
I20250115 03:06:18.536352 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 69533c30b57e45dab0b00239dfd90f13 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.536512 14823 tablet_replica.cc:331] T 69533c30b57e45dab0b00239dfd90f13 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.536623 14823 raft_consensus.cc:2238] T 69533c30b57e45dab0b00239dfd90f13 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.536703 14823 raft_consensus.cc:2267] T 69533c30b57e45dab0b00239dfd90f13 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.538002 14823 ts_tablet_manager.cc:1905] T 69533c30b57e45dab0b00239dfd90f13 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.539618 14820 ts_tablet_manager.cc:1918] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.539685 14820 log.cc:1198] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/6110b6c4911d4932b4cac4d86598a5c6
I20250115 03:06:18.539964 14823 ts_tablet_manager.cc:1918] T 69533c30b57e45dab0b00239dfd90f13 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.540023 14823 log.cc:1198] T 69533c30b57e45dab0b00239dfd90f13 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/69533c30b57e45dab0b00239dfd90f13
I20250115 03:06:18.540153 14820 ts_tablet_manager.cc:1939] T 6110b6c4911d4932b4cac4d86598a5c6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:18.540205 14823 ts_tablet_manager.cc:1939] T 69533c30b57e45dab0b00239dfd90f13 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.540463 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 6110b6c4911d4932b4cac4d86598a5c6 (table default.Table_3e911fd952934f79bee759f184cbcce0 [id=0cbd9b74bdc84d9291bd5abee773d650]) successfully deleted
I20250115 03:06:18.540649 14820 tablet_replica.cc:331] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:18.541029 14820 raft_consensus.cc:2238] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.541173 14820 raft_consensus.cc:2267] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.541949 14809 tablet_bootstrap.cc:492] T 01433c76a398462bb6708a02997047fa P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:18.542024 14809 ts_tablet_manager.cc:1397] T 01433c76a398462bb6708a02997047fa P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.127s user 0.001s sys 0.000s
I20250115 03:06:18.542373 14809 raft_consensus.cc:357] T 01433c76a398462bb6708a02997047fa P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.542451 14809 raft_consensus.cc:383] T 01433c76a398462bb6708a02997047fa P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.542476 14809 raft_consensus.cc:738] T 01433c76a398462bb6708a02997047fa P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.542569 14809 consensus_queue.cc:260] T 01433c76a398462bb6708a02997047fa P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.542795 14809 ts_tablet_manager.cc:1428] T 01433c76a398462bb6708a02997047fa P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:18.542948 14809 tablet_bootstrap.cc:492] T 69533c30b57e45dab0b00239dfd90f13 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:18.542927 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 69533c30b57e45dab0b00239dfd90f13 (table default.Table_d76368b5971a4d08a167c7e7bac8144e [id=3968ef424a944c9c83aacaea3ccc7c0d]) successfully deleted
I20250115 03:06:18.544127 14809 tablet_bootstrap.cc:654] T 69533c30b57e45dab0b00239dfd90f13 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.545310 14811 tablet_bootstrap.cc:654] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.546662 14812 tablet_bootstrap.cc:492] T 0f3bf849316a4ebd841c3345ca17087e P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:18.546754 14812 ts_tablet_manager.cc:1397] T 0f3bf849316a4ebd841c3345ca17087e P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.107s user 0.002s sys 0.000s
I20250115 03:06:18.547181 14812 raft_consensus.cc:357] T 0f3bf849316a4ebd841c3345ca17087e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.547272 14812 raft_consensus.cc:383] T 0f3bf849316a4ebd841c3345ca17087e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.547304 14812 raft_consensus.cc:738] T 0f3bf849316a4ebd841c3345ca17087e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.547403 14812 consensus_queue.cc:260] T 0f3bf849316a4ebd841c3345ca17087e P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.547652 14812 ts_tablet_manager.cc:1428] T 0f3bf849316a4ebd841c3345ca17087e P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.547813 14812 tablet_bootstrap.cc:492] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:18.548650 14812 tablet_bootstrap.cc:654] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.550345 14170 master_service.cc:946] ReplaceTablet: received request to replace tablet c84127dd93bc47ada551f93295ead47b from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:18.553439 14809 tablet_bootstrap.cc:492] T 69533c30b57e45dab0b00239dfd90f13 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:18.554050 14809 ts_tablet_manager.cc:1397] T 69533c30b57e45dab0b00239dfd90f13 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.001s
I20250115 03:06:18.554795 14809 raft_consensus.cc:357] T 69533c30b57e45dab0b00239dfd90f13 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.554941 14809 raft_consensus.cc:383] T 69533c30b57e45dab0b00239dfd90f13 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.555006 14809 raft_consensus.cc:738] T 69533c30b57e45dab0b00239dfd90f13 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.555166 14809 consensus_queue.cc:260] T 69533c30b57e45dab0b00239dfd90f13 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.557163 14809 ts_tablet_manager.cc:1428] T 69533c30b57e45dab0b00239dfd90f13 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:18.557355 14809 tablet_bootstrap.cc:492] T 31446d60b85d4a878e9afe142011aeb1 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:18.558249 14809 tablet_bootstrap.cc:654] T 31446d60b85d4a878e9afe142011aeb1 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.559114 14809 tablet_bootstrap.cc:492] T 31446d60b85d4a878e9afe142011aeb1 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:18.559229 14809 ts_tablet_manager.cc:1397] T 31446d60b85d4a878e9afe142011aeb1 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:18.559639 14809 raft_consensus.cc:357] T 31446d60b85d4a878e9afe142011aeb1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.559777 14809 raft_consensus.cc:383] T 31446d60b85d4a878e9afe142011aeb1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.559839 14809 raft_consensus.cc:738] T 31446d60b85d4a878e9afe142011aeb1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.559995 14809 consensus_queue.cc:260] T 31446d60b85d4a878e9afe142011aeb1 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.560326 14809 ts_tablet_manager.cc:1428] T 31446d60b85d4a878e9afe142011aeb1 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.560514 14809 tablet_bootstrap.cc:492] T 0f3bf849316a4ebd841c3345ca17087e P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:18.560995 15970 tablet.cc:1721] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:18.561371 14809 tablet_bootstrap.cc:654] T 0f3bf849316a4ebd841c3345ca17087e P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.561892 15617 tablet.cc:1721] T 8b0f0539530643f78207e528c39530af P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:18.562117 16001 raft_consensus.cc:491] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.562186 16001 raft_consensus.cc:513] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.562367 14809 tablet_bootstrap.cc:492] T 0f3bf849316a4ebd841c3345ca17087e P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:18.562430 14809 ts_tablet_manager.cc:1397] T 0f3bf849316a4ebd841c3345ca17087e P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:18.562482 16001 leader_election.cc:290] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:18.562784 14809 raft_consensus.cc:357] T 0f3bf849316a4ebd841c3345ca17087e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.562834 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01433c76a398462bb6708a02997047fa" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:18.562871 14809 raft_consensus.cc:383] T 0f3bf849316a4ebd841c3345ca17087e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.562906 14809 raft_consensus.cc:738] T 0f3bf849316a4ebd841c3345ca17087e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.563004 14809 consensus_queue.cc:260] T 0f3bf849316a4ebd841c3345ca17087e P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:18.563153 14626 leader_election.cc:343] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:18.564252 14809 ts_tablet_manager.cc:1428] T 0f3bf849316a4ebd841c3345ca17087e P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:18.564460 14809 tablet_bootstrap.cc:492] T 5cb34cb2006f4716b87bef67f3a6dd8e P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:18.565387 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01433c76a398462bb6708a02997047fa" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:18.565487 14421 raft_consensus.cc:2463] T 01433c76a398462bb6708a02997047fa P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:18.566001 14628 leader_election.cc:304] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:18.566293 15153 raft_consensus.cc:2798] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:18.566351 15153 raft_consensus.cc:491] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:18.566385 15153 raft_consensus.cc:3054] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.566538 14809 tablet_bootstrap.cc:654] T 5cb34cb2006f4716b87bef67f3a6dd8e P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.567199 15153 raft_consensus.cc:513] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.567480 15153 leader_election.cc:290] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:18.567826 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01433c76a398462bb6708a02997047fa" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:18.568646 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_bc39bcc6dcea45ec9bdac3e3e6926460" } modify_external_catalogs: true
I20250115 03:06:18.568763 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_bc39bcc6dcea45ec9bdac3e3e6926460" } modify_external_catalogs: true
W20250115 03:06:18.568758 14626 leader_election.cc:343] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:18.569933 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01433c76a398462bb6708a02997047fa" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:18.570030 14421 raft_consensus.cc:3054] T 01433c76a398462bb6708a02997047fa P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.570856 14421 raft_consensus.cc:2463] T 01433c76a398462bb6708a02997047fa P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:18.571372 14628 leader_election.cc:304] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:18.571578 16001 raft_consensus.cc:2798] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:18.571642 16001 raft_consensus.cc:695] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:18.571743 16001 consensus_queue.cc:237] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.572228 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet fdaf7c090ef642b8b339f61a5d3fb06e
I20250115 03:06:18.572540 14811 tablet_bootstrap.cc:492] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:18.572618 14811 ts_tablet_manager.cc:1397] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.042s user 0.000s sys 0.002s
I20250115 03:06:18.573037 14811 raft_consensus.cc:357] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.573125 14811 raft_consensus.cc:383] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.573153 14811 raft_consensus.cc:738] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.573253 14811 consensus_queue.cc:260] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.573386 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet fdaf7c090ef642b8b339f61a5d3fb06e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.573496 14811 ts_tablet_manager.cc:1428] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.573661 14811 tablet_bootstrap.cc:492] T e63e854c49b34c1ab286f1f97fe443ba P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:18.573921 14520 tablet_service.cc:1514] Processing DeleteTablet for tablet fdaf7c090ef642b8b339f61a5d3fb06e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:18.574052 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet fdaf7c090ef642b8b339f61a5d3fb06e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fdaf7c090ef642b8b339f61a5d3fb06e already in progress: creating tablet
I20250115 03:06:18.574294 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet fdaf7c090ef642b8b339f61a5d3fb06e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.574503 14811 tablet_bootstrap.cc:654] T e63e854c49b34c1ab286f1f97fe443ba P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:18.574663 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet fdaf7c090ef642b8b339f61a5d3fb06e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fdaf7c090ef642b8b339f61a5d3fb06e already in progress: creating tablet
I20250115 03:06:18.575644 14809 tablet_bootstrap.cc:492] T 5cb34cb2006f4716b87bef67f3a6dd8e P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:18.575711 14809 ts_tablet_manager.cc:1397] T 5cb34cb2006f4716b87bef67f3a6dd8e P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20250115 03:06:18.576094 14809 raft_consensus.cc:357] T 5cb34cb2006f4716b87bef67f3a6dd8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.576180 14809 raft_consensus.cc:383] T 5cb34cb2006f4716b87bef67f3a6dd8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.576205 14809 raft_consensus.cc:738] T 5cb34cb2006f4716b87bef67f3a6dd8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.576295 14809 consensus_queue.cc:260] T 5cb34cb2006f4716b87bef67f3a6dd8e P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.576509 14809 ts_tablet_manager.cc:1428] T 5cb34cb2006f4716b87bef67f3a6dd8e P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:18.576637 14809 tablet_bootstrap.cc:492] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:18.577442 14809 tablet_bootstrap.cc:654] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.578514 14520 tablet_service.cc:1514] Processing DeleteTablet for tablet 858bbbb62fc249d5a915697c12362d0f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.579237 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_506e223627794e8bafd5f796b12b6049"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.579536 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_506e223627794e8bafd5f796b12b6049 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.581725 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_7eacd8191bf44769b6371caa6d4772f1" } modify_external_catalogs: true
I20250115 03:06:18.581816 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_7eacd8191bf44769b6371caa6d4772f1" } modify_external_catalogs: true
I20250115 03:06:18.582499 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_605aff02064949c1b48d374a50c910b2" } modify_external_catalogs: true
I20250115 03:06:18.582588 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_605aff02064949c1b48d374a50c910b2" } modify_external_catalogs: true
I20250115 03:06:18.584031 14811 tablet_bootstrap.cc:492] T e63e854c49b34c1ab286f1f97fe443ba P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:18.584107 14811 ts_tablet_manager.cc:1397] T e63e854c49b34c1ab286f1f97fe443ba P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20250115 03:06:18.584514 14811 raft_consensus.cc:357] T e63e854c49b34c1ab286f1f97fe443ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.584602 14811 raft_consensus.cc:383] T e63e854c49b34c1ab286f1f97fe443ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.584632 14811 raft_consensus.cc:738] T e63e854c49b34c1ab286f1f97fe443ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.584739 14811 consensus_queue.cc:260] T e63e854c49b34c1ab286f1f97fe443ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.585009 14811 ts_tablet_manager.cc:1428] T e63e854c49b34c1ab286f1f97fe443ba P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.585165 14811 tablet_bootstrap.cc:492] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:18.585973 14811 tablet_bootstrap.cc:654] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.586571 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 9adb11e56c394461a91c924dc1eb404b
I20250115 03:06:18.587397 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 9adb11e56c394461a91c924dc1eb404b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.587404 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 9adb11e56c394461a91c924dc1eb404b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.587493 14520 tablet_service.cc:1514] Processing DeleteTablet for tablet 9adb11e56c394461a91c924dc1eb404b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.587553 14823 tablet_replica.cc:331] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.587670 14823 raft_consensus.cc:2238] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:18.587842 14823 raft_consensus.cc:2267] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.588092 14823 ts_tablet_manager.cc:1905] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.589547 14823 ts_tablet_manager.cc:1918] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:18.589591 14823 log.cc:1198] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/9adb11e56c394461a91c924dc1eb404b
I20250115 03:06:18.589751 14823 ts_tablet_manager.cc:1939] T 9adb11e56c394461a91c924dc1eb404b P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.589879 14819 tablet_replica.cc:331] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.589996 14819 raft_consensus.cc:2238] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.590082 14819 raft_consensus.cc:2267] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.590378 14819 ts_tablet_manager.cc:1905] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.591889 14812 tablet_bootstrap.cc:492] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:18.591951 14812 ts_tablet_manager.cc:1397] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.044s user 0.002s sys 0.000s
I20250115 03:06:18.592041 14819 ts_tablet_manager.cc:1918] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:18.592094 14819 log.cc:1198] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/9adb11e56c394461a91c924dc1eb404b
I20250115 03:06:18.592291 14819 ts_tablet_manager.cc:1939] T 9adb11e56c394461a91c924dc1eb404b P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.592438 14812 raft_consensus.cc:357] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.592519 14812 raft_consensus.cc:383] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.592546 14812 raft_consensus.cc:738] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.592550 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 9adb11e56c394461a91c924dc1eb404b (table default.Table_605aff02064949c1b48d374a50c910b2 [id=d0fc70d8af9944099f8e7ee9a416399d]) successfully deleted
I20250115 03:06:18.592639 14812 consensus_queue.cc:260] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.592869 14812 ts_tablet_manager.cc:1428] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.593412 14812 tablet_bootstrap.cc:492] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:18.593539 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_1440f76edbb04d508d0304afb1dd436a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.593852 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1440f76edbb04d508d0304afb1dd436a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.594305 14812 tablet_bootstrap.cc:654] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.594875 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_40b1953a4a364eb1ae58e3387fc65a91" } modify_external_catalogs: true
I20250115 03:06:18.594965 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_40b1953a4a364eb1ae58e3387fc65a91" } modify_external_catalogs: true
I20250115 03:06:18.596377 14820 ts_tablet_manager.cc:1905] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.596904 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet a2fc04950cfe4c608a4712afd257ca76 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:18.598664 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_05584d1988f6439180926449a68a7198"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.598948 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_05584d1988f6439180926449a68a7198 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.599474 14820 ts_tablet_manager.cc:1918] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.599534 14820 log.cc:1198] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/bf6f61dc685a48bdb367278773bf1d90
I20250115 03:06:18.600481 14557 raft_consensus.cc:3054] T b6175f691fec45bf9853ee90959a1c98 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.600867 14812 tablet_bootstrap.cc:492] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:18.601007 14812 ts_tablet_manager.cc:1397] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20250115 03:06:18.601003 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 9adb11e56c394461a91c924dc1eb404b (table default.Table_605aff02064949c1b48d374a50c910b2 [id=d0fc70d8af9944099f8e7ee9a416399d]) successfully deleted
I20250115 03:06:18.601416 14812 raft_consensus.cc:357] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.601524 14812 raft_consensus.cc:383] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.601585 14812 raft_consensus.cc:738] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.601751 14812 consensus_queue.cc:260] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.602299 14812 ts_tablet_manager.cc:1428] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.602294 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet a1c93bfd3da44f9f8a8d585f2d21f1be
I20250115 03:06:18.602902 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet a1c93bfd3da44f9f8a8d585f2d21f1be with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.603175 14812 tablet_bootstrap.cc:492] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:18.603279 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet a1c93bfd3da44f9f8a8d585f2d21f1be with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.603678 14823 tablet_replica.cc:331] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.603790 14823 raft_consensus.cc:2238] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.603799 14520 tablet_service.cc:1514] Processing DeleteTablet for tablet a1c93bfd3da44f9f8a8d585f2d21f1be with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.603871 14823 raft_consensus.cc:2267] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.604192 14823 ts_tablet_manager.cc:1905] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.604245 14819 tablet_replica.cc:331] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.604343 14819 raft_consensus.cc:2238] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:18.604494 14819 raft_consensus.cc:2267] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.604727 14820 ts_tablet_manager.cc:1939] T bf6f61dc685a48bdb367278773bf1d90 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:18.604826 14819 ts_tablet_manager.cc:1905] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.605281 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet bf6f61dc685a48bdb367278773bf1d90 (table default.Table_b0ca39935d224741840950333b63bcfe [id=08996d7663c94ed1acb94b675012934c]) successfully deleted
I20250115 03:06:18.605463 14820 tablet_replica.cc:331] T 58161a322a3d469c8444d353689bed65 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:18.605567 14820 raft_consensus.cc:2238] T 58161a322a3d469c8444d353689bed65 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.605638 14820 raft_consensus.cc:2267] T 58161a322a3d469c8444d353689bed65 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.606068 14823 ts_tablet_manager.cc:1918] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.606124 14823 log.cc:1198] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/a1c93bfd3da44f9f8a8d585f2d21f1be
I20250115 03:06:18.606333 14823 ts_tablet_manager.cc:1939] T a1c93bfd3da44f9f8a8d585f2d21f1be P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.606678 14819 ts_tablet_manager.cc:1918] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.606891 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet a1c93bfd3da44f9f8a8d585f2d21f1be (table default.Table_40b1953a4a364eb1ae58e3387fc65a91 [id=64d20b1792044c5e8add735aaf75a6d9]) successfully deleted
I20250115 03:06:18.607903 14812 tablet_bootstrap.cc:654] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.607967 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet a2fc04950cfe4c608a4712afd257ca76
I20250115 03:06:18.608232 14163 catalog_manager.cc:6596] ReplaceTablet: tablet a2fc04950cfe4c608a4712afd257ca76 deleted and replaced by tablet 0fa5ce72e20b4e0092b39c64160f0995
I20250115 03:06:18.608403 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet a2fc04950cfe4c608a4712afd257ca76 with delete_type TABLET_DATA_DELETED (replaced by tablet 0fa5ce72e20b4e0092b39c64160f0995) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.608842 14520 tablet_service.cc:1514] Processing DeleteTablet for tablet a2fc04950cfe4c608a4712afd257ca76 with delete_type TABLET_DATA_DELETED (replaced by tablet 0fa5ce72e20b4e0092b39c64160f0995) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.609054 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet a2fc04950cfe4c608a4712afd257ca76 with delete_type TABLET_DATA_DELETED (replaced by tablet 0fa5ce72e20b4e0092b39c64160f0995) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.609475 14823 tablet_replica.cc:331] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.609634 14823 raft_consensus.cc:2238] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.609769 14823 raft_consensus.cc:2267] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.610110 14823 ts_tablet_manager.cc:1905] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.606741 14819 log.cc:1198] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/a1c93bfd3da44f9f8a8d585f2d21f1be
I20250115 03:06:18.611143 14819 ts_tablet_manager.cc:1939] T a1c93bfd3da44f9f8a8d585f2d21f1be P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.611832 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet a1c93bfd3da44f9f8a8d585f2d21f1be (table default.Table_40b1953a4a364eb1ae58e3387fc65a91 [id=64d20b1792044c5e8add735aaf75a6d9]) successfully deleted
I20250115 03:06:18.611932 14823 ts_tablet_manager.cc:1918] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:18.612027 14823 log.cc:1198] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/a2fc04950cfe4c608a4712afd257ca76
I20250115 03:06:18.612282 14823 ts_tablet_manager.cc:1939] T a2fc04950cfe4c608a4712afd257ca76 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.612876 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet a2fc04950cfe4c608a4712afd257ca76 (table default.Table_6b1f81ee12e04acda4bcf7d7f289a67a [id=8f811bda53fc4d76a88a3dcf69aa3dc6]) successfully deleted
I20250115 03:06:18.613001 14809 tablet_bootstrap.cc:492] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:18.613047 14819 tablet_replica.cc:331] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.613080 14809 ts_tablet_manager.cc:1397] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.036s user 0.002s sys 0.000s
I20250115 03:06:18.613161 14819 raft_consensus.cc:2238] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:18.613312 14819 raft_consensus.cc:2267] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.613492 14809 raft_consensus.cc:357] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.613606 14809 raft_consensus.cc:383] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.613646 14809 raft_consensus.cc:738] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.613777 14809 consensus_queue.cc:260] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.614030 14819 ts_tablet_manager.cc:1905] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.614421 14809 ts_tablet_manager.cc:1428] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.614666 14809 tablet_bootstrap.cc:492] T 51b412277e634ea39bb6d2b90a43b875 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:18.615770 14819 ts_tablet_manager.cc:1918] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:18.615823 14819 log.cc:1198] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/a2fc04950cfe4c608a4712afd257ca76
I20250115 03:06:18.616173 14819 ts_tablet_manager.cc:1939] T a2fc04950cfe4c608a4712afd257ca76 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.616344 14809 tablet_bootstrap.cc:654] T 51b412277e634ea39bb6d2b90a43b875 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.616832 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet a2fc04950cfe4c608a4712afd257ca76 (table default.Table_6b1f81ee12e04acda4bcf7d7f289a67a [id=8f811bda53fc4d76a88a3dcf69aa3dc6]) successfully deleted
I20250115 03:06:18.617305 14820 ts_tablet_manager.cc:1905] T 58161a322a3d469c8444d353689bed65 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.619191 14820 ts_tablet_manager.cc:1918] T 58161a322a3d469c8444d353689bed65 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.619740 14812 tablet_bootstrap.cc:492] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:18.619817 14812 ts_tablet_manager.cc:1397] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20250115 03:06:18.620198 14812 raft_consensus.cc:357] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.620282 14812 raft_consensus.cc:383] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.620309 14812 raft_consensus.cc:738] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.620409 14812 consensus_queue.cc:260] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.620656 14812 ts_tablet_manager.cc:1428] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.620812 14812 tablet_bootstrap.cc:492] T 907301ca08e0467f9f29c6dcb36bdd5e P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:18.621642 14812 tablet_bootstrap.cc:654] T 907301ca08e0467f9f29c6dcb36bdd5e P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.625335 14809 tablet_bootstrap.cc:492] T 51b412277e634ea39bb6d2b90a43b875 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:18.624814 14811 tablet_bootstrap.cc:492] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:18.625447 14809 ts_tablet_manager.cc:1397] T 51b412277e634ea39bb6d2b90a43b875 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20250115 03:06:18.625496 14811 ts_tablet_manager.cc:1397] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.000s
I20250115 03:06:18.625870 14809 raft_consensus.cc:357] T 51b412277e634ea39bb6d2b90a43b875 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.625957 14809 raft_consensus.cc:383] T 51b412277e634ea39bb6d2b90a43b875 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.625908 14811 raft_consensus.cc:357] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.625986 14809 raft_consensus.cc:738] T 51b412277e634ea39bb6d2b90a43b875 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.625998 14811 raft_consensus.cc:383] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.626029 14811 raft_consensus.cc:738] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.626083 14809 consensus_queue.cc:260] T 51b412277e634ea39bb6d2b90a43b875 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.626124 14811 consensus_queue.cc:260] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.626375 14811 ts_tablet_manager.cc:1428] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:18.626380 14809 ts_tablet_manager.cc:1428] T 51b412277e634ea39bb6d2b90a43b875 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.626523 14811 tablet_bootstrap.cc:492] T edf17b41198b4f8fbe5db22c58aa6bb2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:18.626528 14809 tablet_bootstrap.cc:492] T d12d1f964d2b486bb703ed98cb2e7949 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:18.627382 14811 tablet_bootstrap.cc:654] T edf17b41198b4f8fbe5db22c58aa6bb2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.627468 14809 tablet_bootstrap.cc:654] T d12d1f964d2b486bb703ed98cb2e7949 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.629335 14820 log.cc:1198] T 58161a322a3d469c8444d353689bed65 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/58161a322a3d469c8444d353689bed65
I20250115 03:06:18.629585 14820 ts_tablet_manager.cc:1939] T 58161a322a3d469c8444d353689bed65 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:18.630206 14820 tablet_replica.cc:331] T 0d25aa6a292b454f8f451d6493d940c3 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:18.630450 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 58161a322a3d469c8444d353689bed65 (table default.Table_6df09ae8a979496abf73d384649b81b5 [id=e30f9e0f1aef45b7a1668a6d0f5f42bb]) successfully deleted
I20250115 03:06:18.630430 14520 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a59a9be8fa94ec4bdb42c723ea0ca17 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.631486 14820 raft_consensus.cc:2238] T 0d25aa6a292b454f8f451d6493d940c3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
W20250115 03:06:18.630599 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 78cc96de8b414ac19285338ccb1024e0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78cc96de8b414ac19285338ccb1024e0 already in progress: creating tablet
I20250115 03:06:18.631668 14820 raft_consensus.cc:2267] T 0d25aa6a292b454f8f451d6493d940c3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.632054 14820 ts_tablet_manager.cc:1905] T 0d25aa6a292b454f8f451d6493d940c3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.634196 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a1c9e28bac9c491e948bc3fc9b173bad" } modify_external_catalogs: true
I20250115 03:06:18.634285 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a1c9e28bac9c491e948bc3fc9b173bad" } modify_external_catalogs: true
I20250115 03:06:18.637255 16088 consensus_queue.cc:1035] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:18.639338 14820 ts_tablet_manager.cc:1918] T 0d25aa6a292b454f8f451d6493d940c3 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.641712 14820 log.cc:1198] T 0d25aa6a292b454f8f451d6493d940c3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/0d25aa6a292b454f8f451d6493d940c3
I20250115 03:06:18.642012 14820 ts_tablet_manager.cc:1939] T 0d25aa6a292b454f8f451d6493d940c3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:18.644371 16074 catalog_manager.cc:1260] Loaded cluster ID: 6933a79146ea47b6911c92899078d551
I20250115 03:06:18.646822 14165 catalog_manager.cc:5526] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:18.647053 14820 tablet_replica.cc:331] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:18.647125 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 0d25aa6a292b454f8f451d6493d940c3 (table default.Table_e0faf1e67748430cb2b830441fd58344 [id=8698f51a556d4fcc8ed5ebe0bbc9feb0]) successfully deleted
I20250115 03:06:18.647166 14820 raft_consensus.cc:2238] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.647248 14820 raft_consensus.cc:2267] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.647526 14820 ts_tablet_manager.cc:1905] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.647807 14169 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_91fae283e01841f2a274e89f22ebc882 [id=4d6cb6f5c7354beebee1b3fbe4384d54] alter complete (version 1)
I20250115 03:06:18.651173 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 4102586f32d94a998cf2807434ee0ebf
I20250115 03:06:18.651804 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 4102586f32d94a998cf2807434ee0ebf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:18.652254 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 4102586f32d94a998cf2807434ee0ebf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4102586f32d94a998cf2807434ee0ebf already in progress: creating tablet
I20250115 03:06:18.652381 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 2ae7d8e213544722bb30bfc7f0f4a9c4 (DEFAULT_TABLE table=default.Table_a0656bab78454d5aa134b66287b01eb0 [id=f54f86be6a9f4e07a6fb392f9d5544d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.652990 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 4102586f32d94a998cf2807434ee0ebf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.653054 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ab37ce65c0ae4e06b3e72a0e843d8f21"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:18.653286 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ae7d8e213544722bb30bfc7f0f4a9c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.653246 14520 tablet_service.cc:1467] Processing CreateTablet for tablet 2ae7d8e213544722bb30bfc7f0f4a9c4 (DEFAULT_TABLE table=default.Table_a0656bab78454d5aa134b66287b01eb0 [id=f54f86be6a9f4e07a6fb392f9d5544d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.653254 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 2ae7d8e213544722bb30bfc7f0f4a9c4 (DEFAULT_TABLE table=default.Table_a0656bab78454d5aa134b66287b01eb0 [id=f54f86be6a9f4e07a6fb392f9d5544d4]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:18.653337 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab37ce65c0ae4e06b3e72a0e843d8f21 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.653497 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ae7d8e213544722bb30bfc7f0f4a9c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.653927 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet 4102586f32d94a998cf2807434ee0ebf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.654585 14165 catalog_manager.cc:5526] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:18.654857 14165 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 5cb34cb2006f4716b87bef67f3a6dd8e (table default.Table_77b3fb817e8b4502af7fae7bcb0124a5 [id=ed28aa2aaacd4ba58dcb3216a7156a91]). Expected version 1 got 0
I20250115 03:06:18.656760 14520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ae7d8e213544722bb30bfc7f0f4a9c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.661346 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 66b748fc853f499fafc1f476c8f79b8b
W20250115 03:06:18.661496 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 4102586f32d94a998cf2807434ee0ebf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4102586f32d94a998cf2807434ee0ebf already in progress: creating tablet
I20250115 03:06:18.661998 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 66b748fc853f499fafc1f476c8f79b8b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:18.662333 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 66b748fc853f499fafc1f476c8f79b8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66b748fc853f499fafc1f476c8f79b8b already in progress: creating tablet
I20250115 03:06:18.663707 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 192a2efd3e74463e9ea5844eaaf53c97 (DEFAULT_TABLE table=default.Table_f2fdb3ede19849979bf289913d7698fe [id=e675e39b77124b97a17e8ce6a8b5b667]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.663923 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 192a2efd3e74463e9ea5844eaaf53c97. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.663950 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 074fdc09e4e24078ad0bb588b32ff24f (table default.Table_87c206116f514479ab9de8cd4a05817f [id=eaddac40b10648b1933b121cc70d68e8]). Expected version 1 got 0
I20250115 03:06:18.664070 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5cb34cb2006f4716b87bef67f3a6dd8e (table default.Table_77b3fb817e8b4502af7fae7bcb0124a5 [id=ed28aa2aaacd4ba58dcb3216a7156a91]). Expected version 1 got 0
I20250115 03:06:18.664175 14400 tablet_service.cc:1467] Processing CreateTablet for tablet fe513ca696f14838b3886aac174f2cb5 (DEFAULT_TABLE table=default.Table_d0c0059d665043bda7f63aaae45c079e [id=e54709f6a5194c01a2a57ac4c21b86b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.664379 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe513ca696f14838b3886aac174f2cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.665719 14853 raft_consensus.cc:491] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.665798 14853 raft_consensus.cc:513] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.666119 14853 leader_election.cc:290] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:18.666702 14493 leader_election.cc:343] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 858bbbb62fc249d5a915697c12362d0f
I20250115 03:06:18.666715 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 192a2efd3e74463e9ea5844eaaf53c97 (DEFAULT_TABLE table=default.Table_f2fdb3ede19849979bf289913d7698fe [id=e675e39b77124b97a17e8ce6a8b5b667]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:18.666854 14493 leader_election.cc:343] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 858bbbb62fc249d5a915697c12362d0f
I20250115 03:06:18.666915 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 192a2efd3e74463e9ea5844eaaf53c97. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.666909 14493 leader_election.cc:304] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:18.667119 14853 raft_consensus.cc:2743] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:18.667244 14820 ts_tablet_manager.cc:1918] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.667297 14820 log.cc:1198] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/41b22aa85ce34b38b0d6a5e2d38f0a1c
I20250115 03:06:18.667498 14820 ts_tablet_manager.cc:1939] T 41b22aa85ce34b38b0d6a5e2d38f0a1c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:18.667632 14665 tablet_service.cc:1467] Processing CreateTablet for tablet fe513ca696f14838b3886aac174f2cb5 (DEFAULT_TABLE table=default.Table_d0c0059d665043bda7f63aaae45c079e [id=e54709f6a5194c01a2a57ac4c21b86b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.667847 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe513ca696f14838b3886aac174f2cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.668744 14165 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_d7d37f44d845457e8a3656b4f7deabf2 [id=fe357a2a64fc4531870b305eb2480df1] alter complete (version 2)
I20250115 03:06:18.669373 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3bf849316a4ebd841c3345ca17087e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.669968 14823 tablet_replica.cc:331] T 0f3bf849316a4ebd841c3345ca17087e P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.670078 14823 raft_consensus.cc:2238] T 0f3bf849316a4ebd841c3345ca17087e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.670154 14823 raft_consensus.cc:2267] T 0f3bf849316a4ebd841c3345ca17087e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.671782 14402 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3bf849316a4ebd841c3345ca17087e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.671914 14819 tablet_replica.cc:331] T 0f3bf849316a4ebd841c3345ca17087e P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.672017 14819 raft_consensus.cc:2238] T 0f3bf849316a4ebd841c3345ca17087e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.672091 14819 raft_consensus.cc:2267] T 0f3bf849316a4ebd841c3345ca17087e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.670691 14823 ts_tablet_manager.cc:1905] T 0f3bf849316a4ebd841c3345ca17087e P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.672767 14819 ts_tablet_manager.cc:1905] T 0f3bf849316a4ebd841c3345ca17087e P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.673091 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_a222fccf6d3744cab2d0491ce93620be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.673238 14626 consensus_peers.cc:487] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20250115 03:06:18.673382 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a222fccf6d3744cab2d0491ce93620be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.674422 16088 raft_consensus.cc:491] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.674496 16088 raft_consensus.cc:513] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.674544 14819 ts_tablet_manager.cc:1918] T 0f3bf849316a4ebd841c3345ca17087e P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.674597 14819 log.cc:1198] T 0f3bf849316a4ebd841c3345ca17087e P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/0f3bf849316a4ebd841c3345ca17087e
I20250115 03:06:18.674767 14819 ts_tablet_manager.cc:1939] T 0f3bf849316a4ebd841c3345ca17087e P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.674823 16088 leader_election.cc:290] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:18.675107 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31446d60b85d4a878e9afe142011aeb1" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:18.675206 14421 raft_consensus.cc:2463] T 31446d60b85d4a878e9afe142011aeb1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:18.675307 14665 tablet_service.cc:1467] Processing CreateTablet for tablet cc328a01837a4c21abc38c7d85557e15 (DEFAULT_TABLE table=default.Table_d4d372b6bd40428090e1280d674a8b20 [id=578f876844a6410d83e0c3a1ddb77c35]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.675521 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc328a01837a4c21abc38c7d85557e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.675603 14628 leader_election.cc:304] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:18.675737 16088 raft_consensus.cc:2798] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:18.675782 16088 raft_consensus.cc:491] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:18.675817 16088 raft_consensus.cc:3054] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.676862 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 66b748fc853f499fafc1f476c8f79b8b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.676967 16088 raft_consensus.cc:513] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:18.677026 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 69533c30b57e45dab0b00239dfd90f13 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.677274 16088 leader_election.cc:290] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:18.677245 14400 tablet_service.cc:1467] Processing CreateTablet for tablet cc328a01837a4c21abc38c7d85557e15 (DEFAULT_TABLE table=default.Table_d4d372b6bd40428090e1280d674a8b20 [id=578f876844a6410d83e0c3a1ddb77c35]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.677446 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc328a01837a4c21abc38c7d85557e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.677457 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e50402205a344952ac6dd5064f9db8b2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:18.677646 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_75a17b2887884d69a0a674f69f2023fa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.677742 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e50402205a344952ac6dd5064f9db8b2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:18.677918 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_75a17b2887884d69a0a674f69f2023fa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.679191 14401 tablet_service.cc:1467] Processing CreateTablet for tablet c8ac40736c74479dab284ffc7a394883 (DEFAULT_TABLE table=default.Table_e419aadbc76442dd857e70a764ec3c44 [id=705cff14f79f409f938698bbdc962492]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.679394 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8ac40736c74479dab284ffc7a394883. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.680500 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31446d60b85d4a878e9afe142011aeb1" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:18.680595 14421 raft_consensus.cc:3054] T 31446d60b85d4a878e9afe142011aeb1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.681438 14421 raft_consensus.cc:2463] T 31446d60b85d4a878e9afe142011aeb1 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:18.681573 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f4fc7a15692446fba7fcdb17b6b33d56" } new_table_name: "default.Table_ad04661b1b684f879f1c0cd4f94e5458" modify_external_catalogs: true
I20250115 03:06:18.682415 14628 leader_election.cc:304] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:18.682691 16088 raft_consensus.cc:2798] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:18.682677 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_8ec408f3ef9f4344824b35f643021751"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:18.682782 16088 raft_consensus.cc:695] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:18.682893 16088 consensus_queue.cc:237] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
W20250115 03:06:18.682969 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8ec408f3ef9f4344824b35f643021751 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.685909 14163 catalog_manager.cc:5526] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:18.687021 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_478e448d8f6d494d90d421e2f0cd21fc" } new_table_name: "default.Table_c95d9b4b8beb4ebea3e620d35b56bd40" modify_external_catalogs: true
I20250115 03:06:18.687755 15325 consensus_queue.cc:1035] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:18.692791 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 7c503b29cb2843338fb1870a89aee443 (table default.Table_dfcccd3390fa4814a0a04bb1ff5e43fa [id=f0a7068682b9481a82fa34cb54ed528e]). Expected version 1 got 0
I20250115 03:06:18.693480 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet 80aa570e2c154172ad6da29d268bede7 from {username='slave'} at 127.0.0.1:60432
W20250115 03:06:18.693696 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c2afa048b3fd4362b50c8a6da306c25a consensus state has no leader
I20250115 03:06:18.694314 14819 tablet_replica.cc:331] T 69533c30b57e45dab0b00239dfd90f13 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.694424 14819 raft_consensus.cc:2238] T 69533c30b57e45dab0b00239dfd90f13 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.694504 14819 raft_consensus.cc:2267] T 69533c30b57e45dab0b00239dfd90f13 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.694618 14663 tablet_service.cc:1467] Processing CreateTablet for tablet c8ac40736c74479dab284ffc7a394883 (DEFAULT_TABLE table=default.Table_e419aadbc76442dd857e70a764ec3c44 [id=705cff14f79f409f938698bbdc962492]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.694826 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8ac40736c74479dab284ffc7a394883. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.672025 14520 tablet_service.cc:1514] Processing DeleteTablet for tablet 66b748fc853f499fafc1f476c8f79b8b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.694960 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 0f3bf849316a4ebd841c3345ca17087e (table default.Table_96c8d5f733034ee3a6dd8a61dabfd288 [id=ff4681cab8ce44ceb45ed76c70c1132e]) successfully deleted
I20250115 03:06:18.672230 14522 tablet_service.cc:1467] Processing CreateTablet for tablet fe513ca696f14838b3886aac174f2cb5 (DEFAULT_TABLE table=default.Table_d0c0059d665043bda7f63aaae45c079e [id=e54709f6a5194c01a2a57ac4c21b86b0]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:18.695108 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 66b748fc853f499fafc1f476c8f79b8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66b748fc853f499fafc1f476c8f79b8b already in progress: creating tablet
I20250115 03:06:18.695191 14522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe513ca696f14838b3886aac174f2cb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.695477 14819 ts_tablet_manager.cc:1905] T 69533c30b57e45dab0b00239dfd90f13 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.672307 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 192a2efd3e74463e9ea5844eaaf53c97 (DEFAULT_TABLE table=default.Table_f2fdb3ede19849979bf289913d7698fe [id=e675e39b77124b97a17e8ce6a8b5b667]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.697428 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 192a2efd3e74463e9ea5844eaaf53c97. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.697688 14819 ts_tablet_manager.cc:1918] T 69533c30b57e45dab0b00239dfd90f13 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.697742 14819 log.cc:1198] T 69533c30b57e45dab0b00239dfd90f13 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/69533c30b57e45dab0b00239dfd90f13
I20250115 03:06:18.697902 14819 ts_tablet_manager.cc:1939] T 69533c30b57e45dab0b00239dfd90f13 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.698567 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 69533c30b57e45dab0b00239dfd90f13 (table default.Table_d76368b5971a4d08a167c7e7bac8144e [id=3968ef424a944c9c83aacaea3ccc7c0d]) successfully deleted
I20250115 03:06:18.674104 14518 tablet_service.cc:1467] Processing CreateTablet for tablet cc328a01837a4c21abc38c7d85557e15 (DEFAULT_TABLE table=default.Table_d4d372b6bd40428090e1280d674a8b20 [id=578f876844a6410d83e0c3a1ddb77c35]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.699280 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc328a01837a4c21abc38c7d85557e15. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.675909 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31446d60b85d4a878e9afe142011aeb1" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:18.680140 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31446d60b85d4a878e9afe142011aeb1" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:18.700111 14626 leader_election.cc:343] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:18.700255 14626 leader_election.cc:343] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:18.683589 14524 tablet_service.cc:1467] Processing CreateTablet for tablet c8ac40736c74479dab284ffc7a394883 (DEFAULT_TABLE table=default.Table_e419aadbc76442dd857e70a764ec3c44 [id=705cff14f79f409f938698bbdc962492]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.701435 14524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8ac40736c74479dab284ffc7a394883. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.706384 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet 103f78d0d9fc42308e1e23316895be33 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:18.706599 16088 raft_consensus.cc:491] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.706667 16088 raft_consensus.cc:513] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.706688 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_070c9465ea944d50ad72cffc3a10681a" } new_table_name: "default.Table_37c3f2738b8948a08433bf40b28a93a0" modify_external_catalogs: true
I20250115 03:06:18.706941 16088 leader_election.cc:290] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:18.707016 14168 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 074fdc09e4e24078ad0bb588b32ff24f (table default.Table_87c206116f514479ab9de8cd4a05817f [id=eaddac40b10648b1933b121cc70d68e8]). Expected version 1 got 0
I20250115 03:06:18.707361 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b900c4395de448a08b0285a5807f2fdf" } new_table_name: "default.Table_f6d9aee213514fd9a38318e1fc091215" modify_external_catalogs: true
I20250115 03:06:18.707885 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 074fdc09e4e24078ad0bb588b32ff24f (table default.Table_87c206116f514479ab9de8cd4a05817f [id=eaddac40b10648b1933b121cc70d68e8]). Expected version 1 got 0
I20250115 03:06:18.707979 14170 master_service.cc:946] ReplaceTablet: received request to replace tablet 41b22aa85ce34b38b0d6a5e2d38f0a1c from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:18.708643 16074 catalog_manager.cc:1553] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246: loading cluster ID for follower catalog manager: success
I20250115 03:06:18.708979 14812 tablet_bootstrap.cc:492] T 907301ca08e0467f9f29c6dcb36bdd5e P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:18.709080 14812 ts_tablet_manager.cc:1397] T 907301ca08e0467f9f29c6dcb36bdd5e P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.088s user 0.002s sys 0.000s
I20250115 03:06:18.709193 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cb34cb2006f4716b87bef67f3a6dd8e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:18.709517 14626 leader_election.cc:343] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:18.709503 14812 raft_consensus.cc:357] T 907301ca08e0467f9f29c6dcb36bdd5e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.709609 14812 raft_consensus.cc:383] T 907301ca08e0467f9f29c6dcb36bdd5e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.709638 14812 raft_consensus.cc:738] T 907301ca08e0467f9f29c6dcb36bdd5e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.709756 14812 consensus_queue.cc:260] T 907301ca08e0467f9f29c6dcb36bdd5e P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.709996 14812 ts_tablet_manager.cc:1428] T 907301ca08e0467f9f29c6dcb36bdd5e P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.710157 14812 tablet_bootstrap.cc:492] T d12d1f964d2b486bb703ed98cb2e7949 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:18.710199 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cb34cb2006f4716b87bef67f3a6dd8e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:18.710292 14420 raft_consensus.cc:2463] T 5cb34cb2006f4716b87bef67f3a6dd8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:18.710961 14628 leader_election.cc:304] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:18.711014 14812 tablet_bootstrap.cc:654] T d12d1f964d2b486bb703ed98cb2e7949 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.711689 15325 raft_consensus.cc:2798] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:18.711740 15325 raft_consensus.cc:491] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:18.711776 15325 raft_consensus.cc:3054] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.712119 14811 tablet_bootstrap.cc:492] T edf17b41198b4f8fbe5db22c58aa6bb2 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:18.712186 14811 ts_tablet_manager.cc:1397] T edf17b41198b4f8fbe5db22c58aa6bb2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.086s user 0.001s sys 0.000s
I20250115 03:06:18.712568 15325 raft_consensus.cc:513] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.712610 14811 raft_consensus.cc:357] T edf17b41198b4f8fbe5db22c58aa6bb2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.712690 14811 raft_consensus.cc:383] T edf17b41198b4f8fbe5db22c58aa6bb2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.712718 14811 raft_consensus.cc:738] T edf17b41198b4f8fbe5db22c58aa6bb2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.712829 14811 consensus_queue.cc:260] T edf17b41198b4f8fbe5db22c58aa6bb2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.712893 15325 leader_election.cc:290] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:18.713116 14811 ts_tablet_manager.cc:1428] T edf17b41198b4f8fbe5db22c58aa6bb2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:18.713284 14811 tablet_bootstrap.cc:492] T 93fa227c0de74a3fa86778641fdcda3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:18.713647 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cb34cb2006f4716b87bef67f3a6dd8e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:18.713743 14420 raft_consensus.cc:3054] T 5cb34cb2006f4716b87bef67f3a6dd8e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.714272 14811 tablet_bootstrap.cc:654] T 93fa227c0de74a3fa86778641fdcda3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.714581 14420 raft_consensus.cc:2463] T 5cb34cb2006f4716b87bef67f3a6dd8e P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
W20250115 03:06:18.714932 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3724ff14f535416e859339a137c05050 consensus state has no leader
I20250115 03:06:18.714987 14628 leader_election.cc:304] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:18.715282 15153 raft_consensus.cc:2798] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:18.715342 15153 raft_consensus.cc:695] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:18.715435 15153 consensus_queue.cc:237] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.716130 14420 raft_consensus.cc:1270] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 5c0dae7aff4b44c699275b95ba639770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:18.716575 15153 consensus_queue.cc:1035] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:18.718835 14401 tablet_service.cc:1467] Processing CreateTablet for tablet d3728e87eecc4bc48bc1944907ceb8bd (DEFAULT_TABLE table=default.Table_42bcad362d394386967a7e1d627bd77c [id=90ed241bf1ec4428a40d6770ae228552]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.719058 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3728e87eecc4bc48bc1944907ceb8bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.720865 14518 tablet_service.cc:1467] Processing CreateTablet for tablet d3728e87eecc4bc48bc1944907ceb8bd (DEFAULT_TABLE table=default.Table_42bcad362d394386967a7e1d627bd77c [id=90ed241bf1ec4428a40d6770ae228552]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.721212 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 41b22aa85ce34b38b0d6a5e2d38f0a1c (table default.Table_67d007791e734b749420eb72d54c0c69 [id=588a068527c446e9aa2d1910ac9dca64]) successfully deleted
I20250115 03:06:18.722270 14823 ts_tablet_manager.cc:1918] T 0f3bf849316a4ebd841c3345ca17087e P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.722332 14823 log.cc:1198] T 0f3bf849316a4ebd841c3345ca17087e P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/0f3bf849316a4ebd841c3345ca17087e
I20250115 03:06:18.722510 14823 ts_tablet_manager.cc:1939] T 0f3bf849316a4ebd841c3345ca17087e P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.723176 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cb34cb2006f4716b87bef67f3a6dd8e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:18.723721 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 3efc2fb8427f493ea933347bec34f88f (table default.Table_ad04661b1b684f879f1c0cd4f94e5458 [id=364a7905f3f044f684e9c75927f7efda]). Expected version 1 got 0
W20250115 03:06:18.724995 14626 leader_election.cc:343] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:18.725175 14626 consensus_peers.cc:487] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:18.726351 14665 tablet_service.cc:1467] Processing CreateTablet for tablet d3728e87eecc4bc48bc1944907ceb8bd (DEFAULT_TABLE table=default.Table_42bcad362d394386967a7e1d627bd77c [id=90ed241bf1ec4428a40d6770ae228552]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.726570 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3728e87eecc4bc48bc1944907ceb8bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.726882 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_7ae87c57c8ac4747bf90d2a1f4c9f9d1" } new_table_name: "default.Table_03db1a048e7a4aff8c957fbc41cb4675" modify_external_catalogs: true
I20250115 03:06:18.727550 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3728e87eecc4bc48bc1944907ceb8bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.727537 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ff0aab929942464285edb009a9e2e049"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.727823 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff0aab929942464285edb009a9e2e049 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:18.729198 14626 consensus_peers.cc:487] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:18.729887 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_cbbf34723b2d4e2db235756e12af64b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.730157 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cbbf34723b2d4e2db235756e12af64b5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.730183 14170 catalog_manager.cc:5526] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:18.730341 14170 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 5cb34cb2006f4716b87bef67f3a6dd8e (table default.Table_77b3fb817e8b4502af7fae7bcb0124a5 [id=ed28aa2aaacd4ba58dcb3216a7156a91]). Expected version 1 got 0
I20250115 03:06:18.731388 14809 tablet_bootstrap.cc:492] T d12d1f964d2b486bb703ed98cb2e7949 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:18.731462 14809 ts_tablet_manager.cc:1397] T d12d1f964d2b486bb703ed98cb2e7949 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.105s user 0.001s sys 0.000s
I20250115 03:06:18.731447 14518 tablet_service.cc:1467] Processing CreateTablet for tablet 2f0779261368409c9e7b12088f88cb16 (DEFAULT_TABLE table=default.Table_4e4608a627b54f5e9d0acf5b1105de52 [id=2422fe434b714e058e49b31f85407d94]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.731640 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f0779261368409c9e7b12088f88cb16. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.731879 14809 raft_consensus.cc:357] T d12d1f964d2b486bb703ed98cb2e7949 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.731967 14809 raft_consensus.cc:383] T d12d1f964d2b486bb703ed98cb2e7949 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.732002 14809 raft_consensus.cc:738] T d12d1f964d2b486bb703ed98cb2e7949 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.732060 14811 tablet_bootstrap.cc:492] T 93fa227c0de74a3fa86778641fdcda3d P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:18.732126 14811 ts_tablet_manager.cc:1397] T 93fa227c0de74a3fa86778641fdcda3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.002s
I20250115 03:06:18.732103 14809 consensus_queue.cc:260] T d12d1f964d2b486bb703ed98cb2e7949 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.732347 14809 ts_tablet_manager.cc:1428] T d12d1f964d2b486bb703ed98cb2e7949 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.732509 14809 tablet_bootstrap.cc:492] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:18.732515 14811 raft_consensus.cc:357] T 93fa227c0de74a3fa86778641fdcda3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.732599 14811 raft_consensus.cc:383] T 93fa227c0de74a3fa86778641fdcda3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.732626 14811 raft_consensus.cc:738] T 93fa227c0de74a3fa86778641fdcda3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.732734 14811 consensus_queue.cc:260] T 93fa227c0de74a3fa86778641fdcda3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.733006 14811 ts_tablet_manager.cc:1428] T 93fa227c0de74a3fa86778641fdcda3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20250115 03:06:18.733355 14809 tablet_bootstrap.cc:654] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.733628 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 2f0779261368409c9e7b12088f88cb16 (DEFAULT_TABLE table=default.Table_4e4608a627b54f5e9d0acf5b1105de52 [id=2422fe434b714e058e49b31f85407d94]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.733964 14811 tablet_bootstrap.cc:492] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:18.734802 14811 tablet_bootstrap.cc:654] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.734864 16088 raft_consensus.cc:491] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.734930 16088 raft_consensus.cc:513] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.735230 16088 leader_election.cc:290] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:18.735637 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82bf1d7ac9fe4a2eb7b57cf1ab627c3d" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:18.735745 14419 raft_consensus.cc:2463] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:18.736049 14628 leader_election.cc:304] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:18.737146 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet d12d1f964d2b486bb703ed98cb2e7949 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.738915 14524 tablet_service.cc:1514] Processing DeleteTablet for tablet 93fa227c0de74a3fa86778641fdcda3d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.739114 16088 raft_consensus.cc:2798] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:18.739166 16088 raft_consensus.cc:491] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:18.739200 16088 raft_consensus.cc:3054] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.740033 16088 raft_consensus.cc:513] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.740651 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82bf1d7ac9fe4a2eb7b57cf1ab627c3d" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:18.740759 14419 raft_consensus.cc:3054] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.741008 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82bf1d7ac9fe4a2eb7b57cf1ab627c3d" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:18.741025 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82bf1d7ac9fe4a2eb7b57cf1ab627c3d" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:18.741801 14419 raft_consensus.cc:2463] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:18.742043 14628 leader_election.cc:304] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
W20250115 03:06:18.742162 14626 leader_election.cc:343] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:18.742267 14626 leader_election.cc:343] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:18.742316 15153 raft_consensus.cc:491] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.742372 15153 raft_consensus.cc:513] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.742642 15153 leader_election.cc:290] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:18.742728 15153 raft_consensus.cc:2798] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:18.742795 15153 raft_consensus.cc:695] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:18.742897 15153 consensus_queue.cc:237] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:18.742971 14165 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 00aad81f4ca3427cb683de71148cff5c consensus state has no leader
I20250115 03:06:18.743978 16088 leader_election.cc:290] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:18.744300 14819 tablet_replica.cc:331] T d12d1f964d2b486bb703ed98cb2e7949 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.744416 14819 raft_consensus.cc:2238] T d12d1f964d2b486bb703ed98cb2e7949 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.744519 14819 raft_consensus.cc:2267] T d12d1f964d2b486bb703ed98cb2e7949 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.744553 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b412277e634ea39bb6d2b90a43b875" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:18.744635 14419 raft_consensus.cc:2463] T 51b412277e634ea39bb6d2b90a43b875 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:18.744913 14628 leader_election.cc:304] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:18.745119 15325 raft_consensus.cc:2798] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:18.745167 15325 raft_consensus.cc:491] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:18.745203 15325 raft_consensus.cc:3054] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.745997 15325 raft_consensus.cc:513] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.746277 15325 leader_election.cc:290] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:18.746587 14819 ts_tablet_manager.cc:1905] T d12d1f964d2b486bb703ed98cb2e7949 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.747678 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b412277e634ea39bb6d2b90a43b875" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:18.747769 14420 raft_consensus.cc:3054] T 51b412277e634ea39bb6d2b90a43b875 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.747963 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 0f3bf849316a4ebd841c3345ca17087e (table default.Table_96c8d5f733034ee3a6dd8a61dabfd288 [id=ff4681cab8ce44ceb45ed76c70c1132e]) successfully deleted
I20250115 03:06:18.748492 14819 ts_tablet_manager.cc:1918] T d12d1f964d2b486bb703ed98cb2e7949 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.748521 14420 raft_consensus.cc:2463] T 51b412277e634ea39bb6d2b90a43b875 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:18.748551 14819 log.cc:1198] T d12d1f964d2b486bb703ed98cb2e7949 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/d12d1f964d2b486bb703ed98cb2e7949
I20250115 03:06:18.748734 14819 ts_tablet_manager.cc:1939] T d12d1f964d2b486bb703ed98cb2e7949 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.748824 14628 leader_election.cc:304] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:18.749287 15153 raft_consensus.cc:2798] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:18.749351 15153 raft_consensus.cc:695] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:18.749462 14809 tablet_bootstrap.cc:492] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:18.749445 15153 consensus_queue.cc:237] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.749531 14809 ts_tablet_manager.cc:1397] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20250115 03:06:18.749545 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b412277e634ea39bb6d2b90a43b875" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:18.749742 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51b412277e634ea39bb6d2b90a43b875" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:18.749926 14809 raft_consensus.cc:357] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.750010 14809 raft_consensus.cc:383] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.750041 14809 raft_consensus.cc:738] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.750140 14809 consensus_queue.cc:260] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.750276 15970 tablet.cc:1721] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:18.750371 14809 ts_tablet_manager.cc:1428] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.750536 14809 tablet_bootstrap.cc:492] T 7d894471f7b94aec8a19df043906d7e7 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
W20250115 03:06:18.750787 14626 leader_election.cc:343] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:18.750924 14626 leader_election.cc:343] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:18.751368 14809 tablet_bootstrap.cc:654] T 7d894471f7b94aec8a19df043906d7e7 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.753782 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet d12d1f964d2b486bb703ed98cb2e7949 (table default.Table_0e5754dbfde84156b74d86bba62a0046 [id=739ec7394ea24ccb849d2486e451496b]) successfully deleted
I20250115 03:06:18.755805 15617 tablet.cc:1721] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:18.756500 14858 raft_consensus.cc:491] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.756577 14858 raft_consensus.cc:513] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.756897 14858 leader_election.cc:290] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:18.757565 14493 leader_election.cc:343] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 6a59a9be8fa94ec4bdb42c723ea0ca17
W20250115 03:06:18.759306 14493 leader_election.cc:343] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 6a59a9be8fa94ec4bdb42c723ea0ca17
I20250115 03:06:18.759371 14493 leader_election.cc:304] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:18.759558 14858 raft_consensus.cc:2743] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:18.761564 14169 catalog_manager.cc:5526] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:18.764194 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f0779261368409c9e7b12088f88cb16. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.766803 14820 tablet_replica.cc:331] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:18.766969 14820 raft_consensus.cc:2238] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.767107 14820 raft_consensus.cc:2267] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.767464 14820 ts_tablet_manager.cc:1905] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.767791 14809 tablet_bootstrap.cc:492] T 7d894471f7b94aec8a19df043906d7e7 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:18.767889 14809 ts_tablet_manager.cc:1397] T 7d894471f7b94aec8a19df043906d7e7 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20250115 03:06:18.768013 14170 catalog_manager.cc:5526] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:18.768278 14809 raft_consensus.cc:357] T 7d894471f7b94aec8a19df043906d7e7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.768360 14809 raft_consensus.cc:383] T 7d894471f7b94aec8a19df043906d7e7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.768390 14809 raft_consensus.cc:738] T 7d894471f7b94aec8a19df043906d7e7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.768503 14809 consensus_queue.cc:260] T 7d894471f7b94aec8a19df043906d7e7 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.768749 14809 ts_tablet_manager.cc:1428] T 7d894471f7b94aec8a19df043906d7e7 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.768906 14809 tablet_bootstrap.cc:492] T cff95c69fb9045cf9ec0ebb128c9ef6b P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:18.769546 14820 ts_tablet_manager.cc:1918] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:18.769603 14820 log.cc:1198] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/4d4b8d4ed9f44112b83c6ff94aee2206
I20250115 03:06:18.769753 14809 tablet_bootstrap.cc:654] T cff95c69fb9045cf9ec0ebb128c9ef6b P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.770040 14820 ts_tablet_manager.cc:1939] T 4d4b8d4ed9f44112b83c6ff94aee2206 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:18.770390 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 4d4b8d4ed9f44112b83c6ff94aee2206 (table default.Table_f6d9aee213514fd9a38318e1fc091215 [id=adfd7e86aa0f48e5b5e36b4113517dba]) successfully deleted
I20250115 03:06:18.770570 14820 tablet_replica.cc:331] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:18.770685 14820 raft_consensus.cc:2238] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.770761 14820 raft_consensus.cc:2267] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.771039 14820 ts_tablet_manager.cc:1905] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.772158 14169 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_ad04661b1b684f879f1c0cd4f94e5458 [id=364a7905f3f044f684e9c75927f7efda] alter complete (version 1)
I20250115 03:06:18.773674 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 2f0779261368409c9e7b12088f88cb16 (DEFAULT_TABLE table=default.Table_4e4608a627b54f5e9d0acf5b1105de52 [id=2422fe434b714e058e49b31f85407d94]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.773854 14518 tablet_service.cc:1467] Processing CreateTablet for tablet 695927c78bb74b669f2aeca1abf1b648 (DEFAULT_TABLE table=default.Table_8688ed8444804f65b144ec7ac48c5e73 [id=ccb0d57971b94cf3b5ccaed448fe3223]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.773900 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f0779261368409c9e7b12088f88cb16. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.774631 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 695927c78bb74b669f2aeca1abf1b648. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.776367 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b922b874a412477f8cf1a410e26beb1d" } modify_external_catalogs: true
I20250115 03:06:18.776473 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b922b874a412477f8cf1a410e26beb1d" } modify_external_catalogs: true
I20250115 03:06:18.777395 15802 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55756 (request call id 0) took 1760 ms. Trace:
I20250115 03:06:18.781416 14809 tablet_bootstrap.cc:492] T cff95c69fb9045cf9ec0ebb128c9ef6b P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:18.781513 14809 ts_tablet_manager.cc:1397] T cff95c69fb9045cf9ec0ebb128c9ef6b P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.002s
I20250115 03:06:18.781904 14809 raft_consensus.cc:357] T cff95c69fb9045cf9ec0ebb128c9ef6b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.782001 14809 raft_consensus.cc:383] T cff95c69fb9045cf9ec0ebb128c9ef6b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.782030 14809 raft_consensus.cc:738] T cff95c69fb9045cf9ec0ebb128c9ef6b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.782130 14809 consensus_queue.cc:260] T cff95c69fb9045cf9ec0ebb128c9ef6b P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.782388 14809 ts_tablet_manager.cc:1428] T cff95c69fb9045cf9ec0ebb128c9ef6b P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:18.782558 14809 tablet_bootstrap.cc:492] T 1f2dc6928e2941ef9b5b5a7891376500 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:18.783354 14809 tablet_bootstrap.cc:654] T 1f2dc6928e2941ef9b5b5a7891376500 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.784444 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d894471f7b94aec8a19df043906d7e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 194d943d173340e7a8fba69967d9b090) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.784574 14819 tablet_replica.cc:331] T 7d894471f7b94aec8a19df043906d7e7 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.784678 14819 raft_consensus.cc:2238] T 7d894471f7b94aec8a19df043906d7e7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.784747 14819 raft_consensus.cc:2267] T 7d894471f7b94aec8a19df043906d7e7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.785077 14819 ts_tablet_manager.cc:1905] T 7d894471f7b94aec8a19df043906d7e7 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.785392 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet daea48ecc69d441ab66fcb40ea2e7a66
I20250115 03:06:18.786827 14819 ts_tablet_manager.cc:1918] T 7d894471f7b94aec8a19df043906d7e7 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.786904 14819 log.cc:1198] T 7d894471f7b94aec8a19df043906d7e7 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/7d894471f7b94aec8a19df043906d7e7
I20250115 03:06:18.787073 14819 ts_tablet_manager.cc:1939] T 7d894471f7b94aec8a19df043906d7e7 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.787091 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet 0792e52d171446b48a7c9d1f03d37a79 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:18.787801 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_57e91bade4c54372abe1dbba44434098" } modify_external_catalogs: true
I20250115 03:06:18.787887 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_57e91bade4c54372abe1dbba44434098" } modify_external_catalogs: true
I20250115 03:06:18.788542 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_166b9ddd229c4e079ebf7b12ae5e54dc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.788838 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_166b9ddd229c4e079ebf7b12ae5e54dc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.782265 16074 catalog_manager.cc:1575] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246: acquiring CA information for follower catalog manager: success
I20250115 03:06:18.789311 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 7d894471f7b94aec8a19df043906d7e7 (table default.Table_1a2ec424c5cb40fbb3c93f327531e36f [id=50a51914c37b462aa45a71fb260c809b]) successfully deleted
I20250115 03:06:18.789749 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet daea48ecc69d441ab66fcb40ea2e7a66 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:18.790201 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet daea48ecc69d441ab66fcb40ea2e7a66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet daea48ecc69d441ab66fcb40ea2e7a66 already in progress: creating tablet
I20250115 03:06:18.791530 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet cff95c69fb9045cf9ec0ebb128c9ef6b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.791705 14819 tablet_replica.cc:331] T cff95c69fb9045cf9ec0ebb128c9ef6b P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.791843 14819 raft_consensus.cc:2238] T cff95c69fb9045cf9ec0ebb128c9ef6b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.791960 14819 raft_consensus.cc:2267] T cff95c69fb9045cf9ec0ebb128c9ef6b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.792382 14518 tablet_service.cc:1514] Processing DeleteTablet for tablet daea48ecc69d441ab66fcb40ea2e7a66 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.793006 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet daea48ecc69d441ab66fcb40ea2e7a66 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.794004 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 695927c78bb74b669f2aeca1abf1b648 (DEFAULT_TABLE table=default.Table_8688ed8444804f65b144ec7ac48c5e73 [id=ccb0d57971b94cf3b5ccaed448fe3223]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.794226 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 695927c78bb74b669f2aeca1abf1b648. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.794564 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 7ac903c81a1f481ab587e6772c306a96 (DEFAULT_TABLE table=default.Table_e69034f6c27e4ba8bccd3bfa8d47428e [id=bbe51385424040a1a7ef7720483939fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.794770 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ac903c81a1f481ab587e6772c306a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.795913 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 38c9e64e1f5e4f2394afa876553357bf
I20250115 03:06:18.796154 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 7ac903c81a1f481ab587e6772c306a96 (DEFAULT_TABLE table=default.Table_e69034f6c27e4ba8bccd3bfa8d47428e [id=bbe51385424040a1a7ef7720483939fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.796357 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ac903c81a1f481ab587e6772c306a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.797444 14524 tablet_service.cc:1514] Processing DeleteTablet for tablet 38c9e64e1f5e4f2394afa876553357bf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.798007 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 38c9e64e1f5e4f2394afa876553357bf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.798360 14819 ts_tablet_manager.cc:1905] T cff95c69fb9045cf9ec0ebb128c9ef6b P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.798815 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 38c9e64e1f5e4f2394afa876553357bf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.798950 14823 tablet_replica.cc:331] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.799063 14823 raft_consensus.cc:2238] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.799081 14524 tablet_service.cc:1467] Processing CreateTablet for tablet c127b5ac695a456aa38479a5f9a8fe82 (DEFAULT_TABLE table=default.Table_ccc15a33d119445e810777ee918a6541 [id=b07546f80dae4629a963326c1afd4b4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.799170 14823 raft_consensus.cc:2267] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.799279 14524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c127b5ac695a456aa38479a5f9a8fe82. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.799505 14823 ts_tablet_manager.cc:1905] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.800117 14819 ts_tablet_manager.cc:1918] T cff95c69fb9045cf9ec0ebb128c9ef6b P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.800175 14819 log.cc:1198] T cff95c69fb9045cf9ec0ebb128c9ef6b P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/cff95c69fb9045cf9ec0ebb128c9ef6b
I20250115 03:06:18.800350 14819 ts_tablet_manager.cc:1939] T cff95c69fb9045cf9ec0ebb128c9ef6b P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.801496 14823 ts_tablet_manager.cc:1918] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:18.801555 14823 log.cc:1198] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/38c9e64e1f5e4f2394afa876553357bf
I20250115 03:06:18.801744 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet cff95c69fb9045cf9ec0ebb128c9ef6b (table default.Table_1ed9a3074bb7436797358399890084b7 [id=8a02676ab3334b7c81ce22069b29551f]) successfully deleted
I20250115 03:06:18.801770 14823 ts_tablet_manager.cc:1939] T 38c9e64e1f5e4f2394afa876553357bf P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
W20250115 03:06:18.801868 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet daea48ecc69d441ab66fcb40ea2e7a66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet daea48ecc69d441ab66fcb40ea2e7a66 already in progress: creating tablet
I20250115 03:06:18.802088 14518 tablet_service.cc:1467] Processing CreateTablet for tablet 7ac903c81a1f481ab587e6772c306a96 (DEFAULT_TABLE table=default.Table_e69034f6c27e4ba8bccd3bfa8d47428e [id=bbe51385424040a1a7ef7720483939fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.802273 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ac903c81a1f481ab587e6772c306a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.802238 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e9d0346fd3064c8d92a35b495ae3a3c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.802532 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e9d0346fd3064c8d92a35b495ae3a3c8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.803608 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 38c9e64e1f5e4f2394afa876553357bf (table default.Table_57e91bade4c54372abe1dbba44434098 [id=91731a4895d4478fad68b1bcbdc45d5d]) successfully deleted
I20250115 03:06:18.804981 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_7c8f638a9380432c844dc355d43df492" } modify_external_catalogs: true
I20250115 03:06:18.805081 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_7c8f638a9380432c844dc355d43df492" } modify_external_catalogs: true
I20250115 03:06:18.806546 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 695927c78bb74b669f2aeca1abf1b648 (DEFAULT_TABLE table=default.Table_8688ed8444804f65b144ec7ac48c5e73 [id=ccb0d57971b94cf3b5ccaed448fe3223]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.806800 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 695927c78bb74b669f2aeca1abf1b648. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.809216 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet d2564a20857f45d9a0c7e3b74eb58a91
I20250115 03:06:18.809731 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet d2564a20857f45d9a0c7e3b74eb58a91 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:18.810075 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet d2564a20857f45d9a0c7e3b74eb58a91 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d2564a20857f45d9a0c7e3b74eb58a91 already in progress: creating tablet
I20250115 03:06:18.810163 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet d2564a20857f45d9a0c7e3b74eb58a91 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.810519 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_befdedf4ed1940979834e31e3dee138e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.810812 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_befdedf4ed1940979834e31e3dee138e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:18.811870 14358 consensus_peers.cc:487] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 -> Peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Couldn't send request to peer 5c0dae7aff4b44c699275b95ba639770. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38c9e64e1f5e4f2394afa876553357bf. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:18.812307 14518 tablet_service.cc:1514] Processing DeleteTablet for tablet d2564a20857f45d9a0c7e3b74eb58a91 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.812719 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 0792e52d171446b48a7c9d1f03d37a79
I20250115 03:06:18.813369 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 0792e52d171446b48a7c9d1f03d37a79 with delete_type TABLET_DATA_DELETED (replaced by tablet d65bf3745b4f4d92a346b4dd18652432) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.813496 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 0792e52d171446b48a7c9d1f03d37a79 with delete_type TABLET_DATA_DELETED (replaced by tablet d65bf3745b4f4d92a346b4dd18652432) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:18.813796 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 0792e52d171446b48a7c9d1f03d37a79 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0792e52d171446b48a7c9d1f03d37a79 already in progress: creating tablet
I20250115 03:06:18.814258 14665 tablet_service.cc:1467] Processing CreateTablet for tablet c127b5ac695a456aa38479a5f9a8fe82 (DEFAULT_TABLE table=default.Table_ccc15a33d119445e810777ee918a6541 [id=b07546f80dae4629a963326c1afd4b4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.814261 14524 tablet_service.cc:1467] Processing CreateTablet for tablet 31b5928658fb40dc9bb72558b11767dc (DEFAULT_TABLE table=default.Table_2e466ac2b74a4a1b8a5b12182123fd7c [id=414ee4da78a64bc8944ac0df9d841945]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.814460 14524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31b5928658fb40dc9bb72558b11767dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.814478 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c127b5ac695a456aa38479a5f9a8fe82. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.814667 14169 catalog_manager.cc:6596] ReplaceTablet: tablet 0792e52d171446b48a7c9d1f03d37a79 deleted and replaced by tablet d65bf3745b4f4d92a346b4dd18652432
I20250115 03:06:18.814889 14819 tablet_replica.cc:331] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.814993 14819 raft_consensus.cc:2238] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:18.815167 14819 raft_consensus.cc:2267] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.815464 14819 ts_tablet_manager.cc:1905] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.817137 14518 tablet_service.cc:1514] Processing DeleteTablet for tablet 0792e52d171446b48a7c9d1f03d37a79 with delete_type TABLET_DATA_DELETED (replaced by tablet d65bf3745b4f4d92a346b4dd18652432) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.817312 14819 ts_tablet_manager.cc:1918] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:18.817385 14819 log.cc:1198] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/38c9e64e1f5e4f2394afa876553357bf
I20250115 03:06:18.817516 14401 tablet_service.cc:1467] Processing CreateTablet for tablet c127b5ac695a456aa38479a5f9a8fe82 (DEFAULT_TABLE table=default.Table_ccc15a33d119445e810777ee918a6541 [id=b07546f80dae4629a963326c1afd4b4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.817782 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c127b5ac695a456aa38479a5f9a8fe82. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.817953 14819 ts_tablet_manager.cc:1939] T 38c9e64e1f5e4f2394afa876553357bf P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.818534 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 38c9e64e1f5e4f2394afa876553357bf (table default.Table_57e91bade4c54372abe1dbba44434098 [id=91731a4895d4478fad68b1bcbdc45d5d]) successfully deleted
W20250115 03:06:18.818847 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet d2564a20857f45d9a0c7e3b74eb58a91 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d2564a20857f45d9a0c7e3b74eb58a91 already in progress: creating tablet
W20250115 03:06:18.818985 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 0792e52d171446b48a7c9d1f03d37a79 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0792e52d171446b48a7c9d1f03d37a79 already in progress: creating tablet
I20250115 03:06:18.824333 14164 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 7c503b29cb2843338fb1870a89aee443 (table default.Table_dfcccd3390fa4814a0a04bb1ff5e43fa [id=f0a7068682b9481a82fa34cb54ed528e]). Expected version 1 got 0
I20250115 03:06:18.826642 14164 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 7c503b29cb2843338fb1870a89aee443 (table default.Table_dfcccd3390fa4814a0a04bb1ff5e43fa [id=f0a7068682b9481a82fa34cb54ed528e]). Expected version 1 got 0
I20250115 03:06:18.830011 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet da95daa40c5b4615a0219f6d8e3770f6 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:18.830440 14812 tablet_bootstrap.cc:492] T d12d1f964d2b486bb703ed98cb2e7949 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:18.830515 14812 ts_tablet_manager.cc:1397] T d12d1f964d2b486bb703ed98cb2e7949 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.120s user 0.001s sys 0.000s
I20250115 03:06:18.830950 14812 raft_consensus.cc:357] T d12d1f964d2b486bb703ed98cb2e7949 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.831048 14812 raft_consensus.cc:383] T d12d1f964d2b486bb703ed98cb2e7949 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.831075 14812 raft_consensus.cc:738] T d12d1f964d2b486bb703ed98cb2e7949 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.831171 14812 consensus_queue.cc:260] T d12d1f964d2b486bb703ed98cb2e7949 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.831424 14812 ts_tablet_manager.cc:1428] T d12d1f964d2b486bb703ed98cb2e7949 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.831583 14812 tablet_bootstrap.cc:492] T 7d894471f7b94aec8a19df043906d7e7 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:18.831593 16074 catalog_manager.cc:1603] T 00000000000000000000000000000000 P 2db8cff032af46db986bf52b21b52246: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250115 03:06:18.832490 14812 tablet_bootstrap.cc:654] T 7d894471f7b94aec8a19df043906d7e7 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.837321 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet d12d1f964d2b486bb703ed98cb2e7949 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.837541 14823 tablet_replica.cc:331] T d12d1f964d2b486bb703ed98cb2e7949 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.837668 14823 raft_consensus.cc:2238] T d12d1f964d2b486bb703ed98cb2e7949 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.837754 14823 raft_consensus.cc:2267] T d12d1f964d2b486bb703ed98cb2e7949 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.838593 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 31b5928658fb40dc9bb72558b11767dc (DEFAULT_TABLE table=default.Table_2e466ac2b74a4a1b8a5b12182123fd7c [id=414ee4da78a64bc8944ac0df9d841945]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.838680 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet da95daa40c5b4615a0219f6d8e3770f6
I20250115 03:06:18.838822 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31b5928658fb40dc9bb72558b11767dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.838927 14164 catalog_manager.cc:6596] ReplaceTablet: tablet da95daa40c5b4615a0219f6d8e3770f6 deleted and replaced by tablet 678b4453675b42ab98832a20f1b0271b
I20250115 03:06:18.839145 14400 tablet_service.cc:1467] Processing CreateTablet for tablet d8f54e9ccd04475f826ee851b0f079c0 (DEFAULT_TABLE table=default.Table_59c02322b76b4ad18a2c6b0ac7438110 [id=46614a09d8e74d6aad051d1dc4042e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.839358 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8f54e9ccd04475f826ee851b0f079c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.840267 14524 tablet_service.cc:1514] Processing DeleteTablet for tablet da95daa40c5b4615a0219f6d8e3770f6 with delete_type TABLET_DATA_DELETED (replaced by tablet 678b4453675b42ab98832a20f1b0271b) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.840693 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet da95daa40c5b4615a0219f6d8e3770f6 with delete_type TABLET_DATA_DELETED (replaced by tablet 678b4453675b42ab98832a20f1b0271b) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.841284 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet da95daa40c5b4615a0219f6d8e3770f6 with delete_type TABLET_DATA_DELETED (replaced by tablet 678b4453675b42ab98832a20f1b0271b) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.842242 14665 tablet_service.cc:1467] Processing CreateTablet for tablet d8f54e9ccd04475f826ee851b0f079c0 (DEFAULT_TABLE table=default.Table_59c02322b76b4ad18a2c6b0ac7438110 [id=46614a09d8e74d6aad051d1dc4042e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.842444 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8f54e9ccd04475f826ee851b0f079c0. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:18.842893 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet da95daa40c5b4615a0219f6d8e3770f6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da95daa40c5b4615a0219f6d8e3770f6 already in progress: creating tablet
I20250115 03:06:18.843086 14518 tablet_service.cc:1467] Processing CreateTablet for tablet d8f54e9ccd04475f826ee851b0f079c0 (DEFAULT_TABLE table=default.Table_59c02322b76b4ad18a2c6b0ac7438110 [id=46614a09d8e74d6aad051d1dc4042e88]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.843408 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8f54e9ccd04475f826ee851b0f079c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.845571 14823 ts_tablet_manager.cc:1905] T d12d1f964d2b486bb703ed98cb2e7949 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.847294 14823 ts_tablet_manager.cc:1918] T d12d1f964d2b486bb703ed98cb2e7949 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.847348 14823 log.cc:1198] T d12d1f964d2b486bb703ed98cb2e7949 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/d12d1f964d2b486bb703ed98cb2e7949
I20250115 03:06:18.847523 14823 ts_tablet_manager.cc:1939] T d12d1f964d2b486bb703ed98cb2e7949 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.848825 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet d12d1f964d2b486bb703ed98cb2e7949 (table default.Table_0e5754dbfde84156b74d86bba62a0046 [id=739ec7394ea24ccb849d2486e451496b]) successfully deleted
W20250115 03:06:18.848990 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet da95daa40c5b4615a0219f6d8e3770f6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da95daa40c5b4615a0219f6d8e3770f6 already in progress: creating tablet
I20250115 03:06:18.853518 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_717056a529de4818b9c40f37ccb2083b" } modify_external_catalogs: true
I20250115 03:06:18.853631 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_717056a529de4818b9c40f37ccb2083b" } modify_external_catalogs: true
I20250115 03:06:18.854192 15426 tablet.cc:1721] T 29ef42586d164315b5fd54ba91227bad P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:18.854924 15970 tablet.cc:1721] T 29ef42586d164315b5fd54ba91227bad P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:18.857818 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 31446d60b85d4a878e9afe142011aeb1
I20250115 03:06:18.858525 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 31446d60b85d4a878e9afe142011aeb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.858601 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 31446d60b85d4a878e9afe142011aeb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.858659 14823 tablet_replica.cc:331] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.858739 14819 tablet_replica.cc:331] T 31446d60b85d4a878e9afe142011aeb1 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.858767 14823 raft_consensus.cc:2238] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:18.858852 14819 raft_consensus.cc:2238] T 31446d60b85d4a878e9afe142011aeb1 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.858894 14823 pending_rounds.cc:62] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 1 pending ops.
I20250115 03:06:18.858923 14823 pending_rounds.cc:69] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384911086448640 op_type: NO_OP noop_request { }
I20250115 03:06:18.858953 14819 raft_consensus.cc:2267] T 31446d60b85d4a878e9afe142011aeb1 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.858979 14823 raft_consensus.cc:2883] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:18.859025 14823 raft_consensus.cc:2267] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.859488 14819 ts_tablet_manager.cc:1905] T 31446d60b85d4a878e9afe142011aeb1 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.860224 14811 tablet_bootstrap.cc:492] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:18.860301 14811 ts_tablet_manager.cc:1397] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.126s user 0.001s sys 0.000s
I20250115 03:06:18.860746 14811 raft_consensus.cc:357] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.860832 14811 raft_consensus.cc:383] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.860858 14811 raft_consensus.cc:738] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.860977 14811 consensus_queue.cc:260] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.861222 14811 ts_tablet_manager.cc:1428] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.861402 14811 tablet_bootstrap.cc:492] T 5780da67ce984297ba33d3c6901c9e3a P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:18.861469 14819 ts_tablet_manager.cc:1918] T 31446d60b85d4a878e9afe142011aeb1 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.861537 14819 log.cc:1198] T 31446d60b85d4a878e9afe142011aeb1 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/31446d60b85d4a878e9afe142011aeb1
I20250115 03:06:18.861717 14819 ts_tablet_manager.cc:1939] T 31446d60b85d4a878e9afe142011aeb1 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.862249 14811 tablet_bootstrap.cc:654] T 5780da67ce984297ba33d3c6901c9e3a P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.862404 14823 ts_tablet_manager.cc:1905] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.863363 15617 tablet.cc:1721] T 29ef42586d164315b5fd54ba91227bad P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:18.863977 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 31446d60b85d4a878e9afe142011aeb1 (table default.Table_717056a529de4818b9c40f37ccb2083b [id=fa31aef04ed04e58888ba86469303e0b]) successfully deleted
I20250115 03:06:18.864166 14823 ts_tablet_manager.cc:1918] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.864216 14823 log.cc:1198] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/31446d60b85d4a878e9afe142011aeb1
I20250115 03:06:18.864411 14823 ts_tablet_manager.cc:1939] T 31446d60b85d4a878e9afe142011aeb1 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.864984 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 31446d60b85d4a878e9afe142011aeb1 (table default.Table_717056a529de4818b9c40f37ccb2083b [id=fa31aef04ed04e58888ba86469303e0b]) successfully deleted
I20250115 03:06:18.867058 14164 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_37c3f2738b8948a08433bf40b28a93a0 [id=c7ebf406fe8c4b4f91ab34cae586b47a] alter complete (version 2)
I20250115 03:06:18.871182 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_1a09d6243c154136aaa074ea2bfd7f6d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.871521 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1a09d6243c154136aaa074ea2bfd7f6d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.871932 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_73f837db71d74dae9606eb6995ad7270" } modify_external_catalogs: true
I20250115 03:06:18.872031 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_73f837db71d74dae9606eb6995ad7270" } modify_external_catalogs: true
I20250115 03:06:18.877163 14524 tablet_service.cc:1514] Processing DeleteTablet for tablet 56c600fe18504e05bab2bcde3c89861d with delete_type TABLET_DATA_DELETED (replaced by tablet cef4442bfcfd4ad4b343a2785c3a9a74) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.777473 15802 rpcz_store.cc:276] 0115 03:06:17.017399 (+ 0us) service_pool.cc:168] Inserting onto call queue
0115 03:06:17.017438 (+ 39us) service_pool.cc:225] Handling call
0115 03:06:18.431775 (+1414337us) raft_consensus.cc:1466] Updating replica for 286 ops
0115 03:06:18.435897 (+ 4122us) raft_consensus.cc:1529] Early marking committed up to index 491
0115 03:06:18.436115 (+ 218us) raft_consensus.cc:1534] Triggering prepare for 286 ops
0115 03:06:18.436311 (+ 196us) write_op.cc:270] Start()
0115 03:06:18.436337 (+ 26us) write_op.cc:276] Timestamp: P: 1736910374809470 usec, L: 0
0115 03:06:18.436508 (+ 171us) write_op.cc:270] Start()
0115 03:06:18.436530 (+ 22us) write_op.cc:276] Timestamp: P: 1736910374824555 usec, L: 0
0115 03:06:18.436618 (+ 88us) write_op.cc:270] Start()
0115 03:06:18.436632 (+ 14us) write_op.cc:276] Timestamp: P: 1736910374828860 usec, L: 0
0115 03:06:18.436725 (+ 93us) write_op.cc:270] Start()
0115 03:06:18.436737 (+ 12us) write_op.cc:276] Timestamp: P: 1736910374830993 usec, L: 0
0115 03:06:18.436829 (+ 92us) write_op.cc:270] Start()
0115 03:06:18.436843 (+ 14us) write_op.cc:276] Timestamp: P: 1736910374873337 usec, L: 0
0115 03:06:18.436978 (+ 135us) write_op.cc:270] Start()
0115 03:06:18.436992 (+ 14us) write_op.cc:276] Timestamp: P: 1736910374873764 usec, L: 0
0115 03:06:18.437105 (+ 113us) write_op.cc:270] Start()
0115 03:06:18.437125 (+ 20us) write_op.cc:276] Timestamp: P: 1736910374881196 usec, L: 0
0115 03:06:18.437209 (+ 84us) write_op.cc:270] Start()
0115 03:06:18.437219 (+ 10us) write_op.cc:276] Timestamp: P: 1736910374897715 usec, L: 0
0115 03:06:18.437274 (+ 55us) write_op.cc:270] Start()
0115 03:06:18.437282 (+ 8us) write_op.cc:276] Timestamp: P: 1736910374898115 usec, L: 0
0115 03:06:18.437340 (+ 58us) write_op.cc:270] Start()
0115 03:06:18.437349 (+ 9us) write_op.cc:276] Timestamp: P: 1736910374898499 usec, L: 0
0115 03:06:18.437410 (+ 61us) write_op.cc:270] Start()
0115 03:06:18.437423 (+ 13us) write_op.cc:276] Timestamp: P: 1736910374899053 usec, L: 0
0115 03:06:18.437510 (+ 87us) write_op.cc:270] Start()
0115 03:06:18.437523 (+ 13us) write_op.cc:276] Timestamp: P: 1736910374899546 usec, L: 0
0115 03:06:18.437579 (+ 56us) write_op.cc:270] Start()
0115 03:06:18.437587 (+ 8us) write_op.cc:276] Timestamp: P: 1736910374915693 usec, L: 0
0115 03:06:18.437659 (+ 72us) write_op.cc:270] Start()
0115 03:06:18.437667 (+ 8us) write_op.cc:276] Timestamp: P: 1736910374954186 usec, L: 0
0115 03:06:18.437725 (+ 58us) write_op.cc:270] Start()
0115 03:06:18.437734 (+ 9us) write_op.cc:276] Timestamp: P: 1736910374964464 usec, L: 0
0115 03:06:18.437793 (+ 59us) write_op.cc:270] Start()
0115 03:06:18.437801 (+ 8us) write_op.cc:276] Timestamp: P: 1736910374967858 usec, L: 0
0115 03:06:18.437852 (+ 51us) write_op.cc:270] Start()
0115 03:06:18.437863 (+ 11us) write_op.cc:276] Timestamp: P: 1736910374968783 usec, L: 0
0115 03:06:18.437912 (+ 49us) write_op.cc:270] Start()
0115 03:06:18.437921 (+ 9us) write_op.cc:276] Timestamp: P: 1736910374975288 usec, L: 0
0115 03:06:18.437987 (+ 66us) write_op.cc:270] Start()
0115 03:06:18.437999 (+ 12us) write_op.cc:276] Timestamp: P: 1736910374975662 usec, L: 0
0115 03:06:18.438062 (+ 63us) write_op.cc:270] Start()
0115 03:06:18.438072 (+ 10us) write_op.cc:276] Timestamp: P: 1736910374975998 usec, L: 0
0115 03:06:18.438135 (+ 63us) write_op.cc:270] Start()
0115 03:06:18.438147 (+ 12us) write_op.cc:276] Timestamp: P: 1736910374976385 usec, L: 0
0115 03:06:18.438214 (+ 67us) write_op.cc:270] Start()
0115 03:06:18.438225 (+ 11us) write_op.cc:276] Timestamp: P: 1736910374978594 usec, L: 0
0115 03:06:18.438288 (+ 63us) write_op.cc:270] Start()
0115 03:06:18.438299 (+ 11us) write_op.cc:276] Timestamp: P: 1736910374978979 usec, L: 0
0115 03:06:18.438375 (+ 76us) write_op.cc:270] Start()
0115 03:06:18.438388 (+ 13us) write_op.cc:276] Timestamp: P: 1736910374993403 usec, L: 0
0115 03:06:18.438468 (+ 80us) write_op.cc:270] Start()
0115 03:06:18.438481 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375005450 usec, L: 0
0115 03:06:18.438548 (+ 67us) write_op.cc:270] Start()
0115 03:06:18.438559 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375010775 usec, L: 0
0115 03:06:18.438647 (+ 88us) write_op.cc:270] Start()
0115 03:06:18.438661 (+ 14us) write_op.cc:276] Timestamp: P: 1736910375011400 usec, L: 0
0115 03:06:18.438758 (+ 97us) write_op.cc:270] Start()
0115 03:06:18.438772 (+ 14us) write_op.cc:276] Timestamp: P: 1736910375016057 usec, L: 0
0115 03:06:18.438843 (+ 71us) write_op.cc:270] Start()
0115 03:06:18.438854 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375022506 usec, L: 0
0115 03:06:18.438927 (+ 73us) write_op.cc:270] Start()
0115 03:06:18.438941 (+ 14us) write_op.cc:276] Timestamp: P: 1736910375031830 usec, L: 0
0115 03:06:18.439023 (+ 82us) write_op.cc:270] Start()
0115 03:06:18.439036 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375032715 usec, L: 0
0115 03:06:18.439123 (+ 87us) write_op.cc:270] Start()
0115 03:06:18.439138 (+ 15us) write_op.cc:276] Timestamp: P: 1736910375033165 usec, L: 0
0115 03:06:18.439227 (+ 89us) write_op.cc:270] Start()
0115 03:06:18.439240 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375033532 usec, L: 0
0115 03:06:18.439319 (+ 79us) write_op.cc:270] Start()
0115 03:06:18.439343 (+ 24us) write_op.cc:276] Timestamp: P: 1736910375073275 usec, L: 0
0115 03:06:18.439419 (+ 76us) write_op.cc:270] Start()
0115 03:06:18.439429 (+ 10us) write_op.cc:276] Timestamp: P: 1736910375076215 usec, L: 0
0115 03:06:18.439495 (+ 66us) write_op.cc:270] Start()
0115 03:06:18.439504 (+ 9us) write_op.cc:276] Timestamp: P: 1736910375078724 usec, L: 0
0115 03:06:18.439553 (+ 49us) write_op.cc:270] Start()
0115 03:06:18.439566 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375091607 usec, L: 0
0115 03:06:18.439625 (+ 59us) write_op.cc:270] Start()
0115 03:06:18.439634 (+ 9us) write_op.cc:276] Timestamp: P: 1736910375096398 usec, L: 0
0115 03:06:18.439681 (+ 47us) write_op.cc:270] Start()
0115 03:06:18.439689 (+ 8us) write_op.cc:276] Timestamp: P: 1736910375098106 usec, L: 0
0115 03:06:18.439755 (+ 66us) write_op.cc:270] Start()
0115 03:06:18.439766 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375110777 usec, L: 0
0115 03:06:18.439833 (+ 67us) write_op.cc:270] Start()
0115 03:06:18.439845 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375114572 usec, L: 0
0115 03:06:18.439904 (+ 59us) write_op.cc:270] Start()
0115 03:06:18.439914 (+ 10us) write_op.cc:276] Timestamp: P: 1736910375115956 usec, L: 0
0115 03:06:18.439988 (+ 74us) write_op.cc:270] Start()
0115 03:06:18.439999 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375116315 usec, L: 0
0115 03:06:18.440076 (+ 77us) write_op.cc:270] Start()
0115 03:06:18.440089 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375117005 usec, L: 0
0115 03:06:18.440169 (+ 80us) write_op.cc:270] Start()
0115 03:06:18.440181 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375121452 usec, L: 0
0115 03:06:18.440250 (+ 69us) write_op.cc:270] Start()
0115 03:06:18.440263 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375130415 usec, L: 0
0115 03:06:18.440332 (+ 69us) write_op.cc:270] Start()
0115 03:06:18.440343 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375135708 usec, L: 0
0115 03:06:18.440421 (+ 78us) write_op.cc:270] Start()
0115 03:06:18.440434 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375150205 usec, L: 0
0115 03:06:18.440514 (+ 80us) write_op.cc:270] Start()
0115 03:06:18.440527 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375154577 usec, L: 0
0115 03:06:18.440622 (+ 95us) write_op.cc:270] Start()
0115 03:06:18.440635 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375184093 usec, L: 0
0115 03:06:18.440715 (+ 80us) write_op.cc:270] Start()
0115 03:06:18.440727 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375184516 usec, L: 0
0115 03:06:18.440844 (+ 117us) write_op.cc:270] Start()
0115 03:06:18.440858 (+ 14us) write_op.cc:276] Timestamp: P: 1736910375184918 usec, L: 0
0115 03:06:18.445323 (+ 4465us) write_op.cc:270] Start()
0115 03:06:18.445345 (+ 22us) write_op.cc:276] Timestamp: P: 1736910375185311 usec, L: 0
0115 03:06:18.445444 (+ 99us) write_op.cc:270] Start()
0115 03:06:18.445457 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375185666 usec, L: 0
0115 03:06:18.445527 (+ 70us) write_op.cc:270] Start()
0115 03:06:18.445540 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375195894 usec, L: 0
0115 03:06:18.445617 (+ 77us) write_op.cc:270] Start()
0115 03:06:18.445629 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375197357 usec, L: 0
0115 03:06:18.445690 (+ 61us) write_op.cc:270] Start()
0115 03:06:18.445709 (+ 19us) write_op.cc:276] Timestamp: P: 1736910375197754 usec, L: 0
0115 03:06:18.445774 (+ 65us) write_op.cc:270] Start()
0115 03:06:18.445786 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375212331 usec, L: 0
0115 03:06:18.445862 (+ 76us) write_op.cc:270] Start()
0115 03:06:18.445873 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375218311 usec, L: 0
0115 03:06:18.445954 (+ 81us) write_op.cc:270] Start()
0115 03:06:18.445971 (+ 17us) write_op.cc:276] Timestamp: P: 1736910375220156 usec, L: 0
0115 03:06:18.446045 (+ 74us) write_op.cc:270] Start()
0115 03:06:18.446057 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375239076 usec, L: 0
0115 03:06:18.446212 (+ 155us) write_op.cc:270] Start()
0115 03:06:18.446629 (+ 417us) write_op.cc:276] Timestamp: P: 1736910375240112 usec, L: 0
0115 03:06:18.446755 (+ 126us) write_op.cc:270] Start()
0115 03:06:18.446771 (+ 16us) write_op.cc:276] Timestamp: P: 1736910375240644 usec, L: 0
0115 03:06:18.446870 (+ 99us) write_op.cc:270] Start()
0115 03:06:18.446886 (+ 16us) write_op.cc:276] Timestamp: P: 1736910375248195 usec, L: 0
0115 03:06:18.446977 (+ 91us) write_op.cc:270] Start()
0115 03:06:18.446992 (+ 15us) write_op.cc:276] Timestamp: P: 1736910375251144 usec, L: 0
0115 03:06:18.447088 (+ 96us) write_op.cc:270] Start()
0115 03:06:18.447112 (+ 24us) write_op.cc:276] Timestamp: P: 1736910375265566 usec, L: 0
0115 03:06:18.447209 (+ 97us) write_op.cc:270] Start()
0115 03:06:18.447224 (+ 15us) write_op.cc:276] Timestamp: P: 1736910375276043 usec, L: 0
0115 03:06:18.447908 (+ 684us) write_op.cc:270] Start()
0115 03:06:18.447938 (+ 30us) write_op.cc:276] Timestamp: P: 1736910375287134 usec, L: 0
0115 03:06:18.448037 (+ 99us) write_op.cc:270] Start()
0115 03:06:18.448052 (+ 15us) write_op.cc:276] Timestamp: P: 1736910375302449 usec, L: 0
0115 03:06:18.448129 (+ 77us) write_op.cc:270] Start()
0115 03:06:18.448143 (+ 14us) write_op.cc:276] Timestamp: P: 1736910375305817 usec, L: 0
0115 03:06:18.448230 (+ 87us) write_op.cc:270] Start()
0115 03:06:18.448246 (+ 16us) write_op.cc:276] Timestamp: P: 1736910375309474 usec, L: 0
0115 03:06:18.448331 (+ 85us) write_op.cc:270] Start()
0115 03:06:18.448345 (+ 14us) write_op.cc:276] Timestamp: P: 1736910375313712 usec, L: 0
0115 03:06:18.448417 (+ 72us) write_op.cc:270] Start()
0115 03:06:18.448430 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375324173 usec, L: 0
0115 03:06:18.448501 (+ 71us) write_op.cc:270] Start()
0115 03:06:18.448515 (+ 14us) write_op.cc:276] Timestamp: P: 1736910375324535 usec, L: 0
0115 03:06:18.448600 (+ 85us) write_op.cc:270] Start()
0115 03:06:18.448613 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375326472 usec, L: 0
0115 03:06:18.448711 (+ 98us) write_op.cc:270] Start()
0115 03:06:18.450080 (+ 1369us) write_op.cc:276] Timestamp: P: 1736910375327971 usec, L: 0
0115 03:06:18.450184 (+ 104us) write_op.cc:270] Start()
0115 03:06:18.450200 (+ 16us) write_op.cc:276] Timestamp: P: 1736910375329051 usec, L: 0
0115 03:06:18.450280 (+ 80us) write_op.cc:270] Start()
0115 03:06:18.450295 (+ 15us) write_op.cc:276] Timestamp: P: 1736910375332486 usec, L: 0
0115 03:06:18.450369 (+ 74us) write_op.cc:270] Start()
0115 03:06:18.450415 (+ 46us) write_op.cc:276] Timestamp: P: 1736910375334121 usec, L: 0
0115 03:06:18.450507 (+ 92us) write_op.cc:270] Start()
0115 03:06:18.450540 (+ 33us) write_op.cc:276] Timestamp: P: 1736910375347116 usec, L: 0
0115 03:06:18.450613 (+ 73us) write_op.cc:270] Start()
0115 03:06:18.450625 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375356446 usec, L: 0
0115 03:06:18.450714 (+ 89us) write_op.cc:270] Start()
0115 03:06:18.450729 (+ 15us) write_op.cc:276] Timestamp: P: 1736910375363747 usec, L: 0
0115 03:06:18.450807 (+ 78us) write_op.cc:270] Start()
0115 03:06:18.451022 (+ 215us) write_op.cc:276] Timestamp: P: 1736910375394948 usec, L: 0
0115 03:06:18.451127 (+ 105us) write_op.cc:270] Start()
0115 03:06:18.451140 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375398673 usec, L: 0
0115 03:06:18.451209 (+ 69us) write_op.cc:270] Start()
0115 03:06:18.451223 (+ 14us) write_op.cc:276] Timestamp: P: 1736910375399679 usec, L: 0
0115 03:06:18.451318 (+ 95us) write_op.cc:270] Start()
0115 03:06:18.451334 (+ 16us) write_op.cc:276] Timestamp: P: 1736910375400021 usec, L: 0
0115 03:06:18.451409 (+ 75us) write_op.cc:270] Start()
0115 03:06:18.451422 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375400840 usec, L: 0
0115 03:06:18.451510 (+ 88us) write_op.cc:270] Start()
0115 03:06:18.451524 (+ 14us) write_op.cc:276] Timestamp: P: 1736910375406864 usec, L: 0
0115 03:06:18.451601 (+ 77us) write_op.cc:270] Start()
0115 03:06:18.451613 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375415743 usec, L: 0
0115 03:06:18.451670 (+ 57us) write_op.cc:270] Start()
0115 03:06:18.451679 (+ 9us) write_op.cc:276] Timestamp: P: 1736910375419745 usec, L: 0
0115 03:06:18.451759 (+ 80us) write_op.cc:270] Start()
0115 03:06:18.451773 (+ 14us) write_op.cc:276] Timestamp: P: 1736910375440294 usec, L: 0
0115 03:06:18.451870 (+ 97us) write_op.cc:270] Start()
0115 03:06:18.451884 (+ 14us) write_op.cc:276] Timestamp: P: 1736910375445148 usec, L: 0
0115 03:06:18.451979 (+ 95us) write_op.cc:270] Start()
0115 03:06:18.451992 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375455228 usec, L: 0
0115 03:06:18.452069 (+ 77us) write_op.cc:270] Start()
0115 03:06:18.452083 (+ 14us) write_op.cc:276] Timestamp: P: 1736910375460416 usec, L: 0
0115 03:06:18.452150 (+ 67us) write_op.cc:270] Start()
0115 03:06:18.452162 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375470259 usec, L: 0
0115 03:06:18.458738 (+ 6576us) write_op.cc:270] Start()
0115 03:06:18.458759 (+ 21us) write_op.cc:276] Timestamp: P: 1736910375488600 usec, L: 0
0115 03:06:18.458870 (+ 111us) write_op.cc:270] Start()
0115 03:06:18.458887 (+ 17us) write_op.cc:276] Timestamp: P: 1736910375504780 usec, L: 0
0115 03:06:18.458966 (+ 79us) write_op.cc:270] Start()
0115 03:06:18.458978 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375505198 usec, L: 0
0115 03:06:18.459041 (+ 63us) write_op.cc:270] Start()
0115 03:06:18.459051 (+ 10us) write_op.cc:276] Timestamp: P: 1736910375505553 usec, L: 0
0115 03:06:18.459127 (+ 76us) write_op.cc:270] Start()
0115 03:06:18.459139 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375505925 usec, L: 0
0115 03:06:18.459218 (+ 79us) write_op.cc:270] Start()
0115 03:06:18.459231 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375529821 usec, L: 0
0115 03:06:18.459308 (+ 77us) write_op.cc:270] Start()
0115 03:06:18.459321 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375530202 usec, L: 0
0115 03:06:18.459392 (+ 71us) write_op.cc:270] Start()
0115 03:06:18.459403 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375530583 usec, L: 0
0115 03:06:18.459494 (+ 91us) write_op.cc:270] Start()
0115 03:06:18.459505 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375534113 usec, L: 0
0115 03:06:18.459566 (+ 61us) write_op.cc:270] Start()
0115 03:06:18.459576 (+ 10us) write_op.cc:276] Timestamp: P: 1736910375534797 usec, L: 0
0115 03:06:18.459634 (+ 58us) write_op.cc:270] Start()
0115 03:06:18.459645 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375584978 usec, L: 0
0115 03:06:18.459708 (+ 63us) write_op.cc:270] Start()
0115 03:06:18.459719 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375585934 usec, L: 0
0115 03:06:18.459793 (+ 74us) write_op.cc:270] Start()
0115 03:06:18.459805 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375586347 usec, L: 0
0115 03:06:18.459866 (+ 61us) write_op.cc:270] Start()
0115 03:06:18.459876 (+ 10us) write_op.cc:276] Timestamp: P: 1736910375586666 usec, L: 0
0115 03:06:18.459932 (+ 56us) write_op.cc:270] Start()
0115 03:06:18.459941 (+ 9us) write_op.cc:276] Timestamp: P: 1736910375595971 usec, L: 0
0115 03:06:18.459999 (+ 58us) write_op.cc:270] Start()
0115 03:06:18.460010 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375601893 usec, L: 0
0115 03:06:18.460481 (+ 471us) write_op.cc:270] Start()
0115 03:06:18.460499 (+ 18us) write_op.cc:276] Timestamp: P: 1736910375613193 usec, L: 0
0115 03:06:18.460595 (+ 96us) write_op.cc:270] Start()
0115 03:06:18.460608 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375616552 usec, L: 0
0115 03:06:18.460681 (+ 73us) write_op.cc:270] Start()
0115 03:06:18.460692 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375620538 usec, L: 0
0115 03:06:18.460775 (+ 83us) write_op.cc:270] Start()
0115 03:06:18.460788 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375621124 usec, L: 0
0115 03:06:18.460878 (+ 90us) write_op.cc:270] Start()
0115 03:06:18.460891 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375621865 usec, L: 0
0115 03:06:18.461002 (+ 111us) write_op.cc:270] Start()
0115 03:06:18.461016 (+ 14us) write_op.cc:276] Timestamp: P: 1736910375631892 usec, L: 0
0115 03:06:18.461095 (+ 79us) write_op.cc:270] Start()
0115 03:06:18.461107 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375635380 usec, L: 0
0115 03:06:18.461190 (+ 83us) write_op.cc:270] Start()
0115 03:06:18.461202 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375649497 usec, L: 0
0115 03:06:18.461284 (+ 82us) write_op.cc:270] Start()
0115 03:06:18.461293 (+ 9us) write_op.cc:276] Timestamp: P: 1736910375651026 usec, L: 0
0115 03:06:18.461344 (+ 51us) write_op.cc:270] Start()
0115 03:06:18.461355 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375651925 usec, L: 0
0115 03:06:18.461417 (+ 62us) write_op.cc:270] Start()
0115 03:06:18.461426 (+ 9us) write_op.cc:276] Timestamp: P: 1736910375652491 usec, L: 0
0115 03:06:18.461498 (+ 72us) write_op.cc:270] Start()
0115 03:06:18.461510 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375653096 usec, L: 0
0115 03:06:18.461596 (+ 86us) write_op.cc:270] Start()
0115 03:06:18.461608 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375675001 usec, L: 0
0115 03:06:18.461679 (+ 71us) write_op.cc:270] Start()
0115 03:06:18.461692 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375690239 usec, L: 0
0115 03:06:18.461776 (+ 84us) write_op.cc:270] Start()
0115 03:06:18.461786 (+ 10us) write_op.cc:276] Timestamp: P: 1736910375694796 usec, L: 0
0115 03:06:18.461839 (+ 53us) write_op.cc:270] Start()
0115 03:06:18.461847 (+ 8us) write_op.cc:276] Timestamp: P: 1736910375704436 usec, L: 0
0115 03:06:18.461908 (+ 61us) write_op.cc:270] Start()
0115 03:06:18.461917 (+ 9us) write_op.cc:276] Timestamp: P: 1736910375711826 usec, L: 0
0115 03:06:18.461975 (+ 58us) write_op.cc:270] Start()
0115 03:06:18.461983 (+ 8us) write_op.cc:276] Timestamp: P: 1736910375717154 usec, L: 0
0115 03:06:18.462039 (+ 56us) write_op.cc:270] Start()
0115 03:06:18.462047 (+ 8us) write_op.cc:276] Timestamp: P: 1736910375730663 usec, L: 0
0115 03:06:18.462098 (+ 51us) write_op.cc:270] Start()
0115 03:06:18.462112 (+ 14us) write_op.cc:276] Timestamp: P: 1736910375735546 usec, L: 0
0115 03:06:18.462190 (+ 78us) write_op.cc:270] Start()
0115 03:06:18.462203 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375744707 usec, L: 0
0115 03:06:18.462271 (+ 68us) write_op.cc:270] Start()
0115 03:06:18.462281 (+ 10us) write_op.cc:276] Timestamp: P: 1736910375749087 usec, L: 0
0115 03:06:18.462354 (+ 73us) write_op.cc:270] Start()
0115 03:06:18.462365 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375762295 usec, L: 0
0115 03:06:18.462431 (+ 66us) write_op.cc:270] Start()
0115 03:06:18.462442 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375772258 usec, L: 0
0115 03:06:18.462539 (+ 97us) write_op.cc:270] Start()
0115 03:06:18.462551 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375777769 usec, L: 0
0115 03:06:18.462628 (+ 77us) write_op.cc:270] Start()
0115 03:06:18.462640 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375786880 usec, L: 0
0115 03:06:18.462707 (+ 67us) write_op.cc:270] Start()
0115 03:06:18.462718 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375791775 usec, L: 0
0115 03:06:18.462782 (+ 64us) write_op.cc:270] Start()
0115 03:06:18.462798 (+ 16us) write_op.cc:276] Timestamp: P: 1736910375799421 usec, L: 0
0115 03:06:18.462875 (+ 77us) write_op.cc:270] Start()
0115 03:06:18.462887 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375815964 usec, L: 0
0115 03:06:18.462953 (+ 66us) write_op.cc:270] Start()
0115 03:06:18.462964 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375817649 usec, L: 0
0115 03:06:18.463034 (+ 70us) write_op.cc:270] Start()
0115 03:06:18.463046 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375825754 usec, L: 0
0115 03:06:18.463116 (+ 70us) write_op.cc:270] Start()
0115 03:06:18.463127 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375840447 usec, L: 0
0115 03:06:18.463201 (+ 74us) write_op.cc:270] Start()
0115 03:06:18.463213 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375841729 usec, L: 0
0115 03:06:18.463279 (+ 66us) write_op.cc:270] Start()
0115 03:06:18.463290 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375842770 usec, L: 0
0115 03:06:18.463353 (+ 63us) write_op.cc:270] Start()
0115 03:06:18.463364 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375843111 usec, L: 0
0115 03:06:18.463430 (+ 66us) write_op.cc:270] Start()
0115 03:06:18.463441 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375843451 usec, L: 0
0115 03:06:18.463544 (+ 103us) write_op.cc:270] Start()
0115 03:06:18.463555 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375844086 usec, L: 0
0115 03:06:18.463621 (+ 66us) write_op.cc:270] Start()
0115 03:06:18.463633 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375875445 usec, L: 0
0115 03:06:18.463699 (+ 66us) write_op.cc:270] Start()
0115 03:06:18.463711 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375883533 usec, L: 0
0115 03:06:18.463934 (+ 223us) write_op.cc:270] Start()
0115 03:06:18.463952 (+ 18us) write_op.cc:276] Timestamp: P: 1736910375887358 usec, L: 0
0115 03:06:18.464062 (+ 110us) write_op.cc:270] Start()
0115 03:06:18.464078 (+ 16us) write_op.cc:276] Timestamp: P: 1736910375926130 usec, L: 0
0115 03:06:18.464158 (+ 80us) write_op.cc:270] Start()
0115 03:06:18.464171 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375931391 usec, L: 0
0115 03:06:18.464242 (+ 71us) write_op.cc:270] Start()
0115 03:06:18.464255 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375934896 usec, L: 0
0115 03:06:18.464328 (+ 73us) write_op.cc:270] Start()
0115 03:06:18.464341 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375940507 usec, L: 0
0115 03:06:18.464412 (+ 71us) write_op.cc:270] Start()
0115 03:06:18.464424 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375945780 usec, L: 0
0115 03:06:18.464512 (+ 88us) write_op.cc:270] Start()
0115 03:06:18.464525 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375948130 usec, L: 0
0115 03:06:18.464594 (+ 69us) write_op.cc:270] Start()
0115 03:06:18.464605 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375948696 usec, L: 0
0115 03:06:18.464671 (+ 66us) write_op.cc:270] Start()
0115 03:06:18.464682 (+ 11us) write_op.cc:276] Timestamp: P: 1736910375951573 usec, L: 0
0115 03:06:18.464751 (+ 69us) write_op.cc:270] Start()
0115 03:06:18.464764 (+ 13us) write_op.cc:276] Timestamp: P: 1736910375960005 usec, L: 0
0115 03:06:18.464835 (+ 71us) write_op.cc:270] Start()
0115 03:06:18.464847 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375971156 usec, L: 0
0115 03:06:18.464913 (+ 66us) write_op.cc:270] Start()
0115 03:06:18.465009 (+ 96us) write_op.cc:276] Timestamp: P: 1736910375978003 usec, L: 0
0115 03:06:18.465103 (+ 94us) write_op.cc:270] Start()
0115 03:06:18.465119 (+ 16us) write_op.cc:276] Timestamp: P: 1736910375984274 usec, L: 0
0115 03:06:18.465201 (+ 82us) write_op.cc:270] Start()
0115 03:06:18.465213 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375989060 usec, L: 0
0115 03:06:18.465283 (+ 70us) write_op.cc:270] Start()
0115 03:06:18.465295 (+ 12us) write_op.cc:276] Timestamp: P: 1736910375994586 usec, L: 0
0115 03:06:18.465372 (+ 77us) write_op.cc:270] Start()
0115 03:06:18.465384 (+ 12us) write_op.cc:276] Timestamp: P: 1736910376062435 usec, L: 0
0115 03:06:18.465463 (+ 79us) write_op.cc:270] Start()
0115 03:06:18.465501 (+ 38us) write_op.cc:276] Timestamp: P: 1736910376063263 usec, L: 0
0115 03:06:18.465615 (+ 114us) write_op.cc:270] Start()
0115 03:06:18.465629 (+ 14us) write_op.cc:276] Timestamp: P: 1736910376063946 usec, L: 0
0115 03:06:18.465715 (+ 86us) write_op.cc:270] Start()
0115 03:06:18.465727 (+ 12us) write_op.cc:276] Timestamp: P: 1736910376064874 usec, L: 0
0115 03:06:18.465805 (+ 78us) write_op.cc:270] Start()
0115 03:06:18.465818 (+ 13us) write_op.cc:276] Timestamp: P: 1736910376065311 usec, L: 0
0115 03:06:18.465890 (+ 72us) write_op.cc:270] Start()
0115 03:06:18.465902 (+ 12us) write_op.cc:276] Timestamp: P: 1736910376065742 usec, L: 0
0115 03:06:18.465979 (+ 77us) write_op.cc:270] Start()
0115 03:06:18.465991 (+ 12us) write_op.cc:276] Timestamp: P: 1736910376066116 usec, L: 0
0115 03:06:18.466057 (+ 66us) write_op.cc:270] Start()
0115 03:06:18.466068 (+ 11us) write_op.cc:276] Timestamp: P: 1736910376082074 usec, L: 0
0115 03:06:18.466132 (+ 64us) write_op.cc:270] Start()
0115 03:06:18.466143 (+ 11us) write_op.cc:276] Timestamp: P: 1736910376083522 usec, L: 0
0115 03:06:18.466223 (+ 80us) write_op.cc:270] Start()
0115 03:06:18.466236 (+ 13us) write_op.cc:276] Timestamp: P: 1736910376084254 usec, L: 0
0115 03:06:18.466338 (+ 102us) write_op.cc:270] Start()
0115 03:06:18.466353 (+ 15us) write_op.cc:276] Timestamp: P: 1736910376086632 usec, L: 0
0115 03:06:18.466434 (+ 81us) write_op.cc:270] Start()
0115 03:06:18.466446 (+ 12us) write_op.cc:276] Timestamp: P: 1736910376105604 usec, L: 0
0115 03:06:18.466532 (+ 86us) write_op.cc:270] Start()
0115 03:06:18.466546 (+ 14us) write_op.cc:276] Timestamp: P: 1736910376106927 usec, L: 0
0115 03:06:18.466618 (+ 72us) write_op.cc:270] Start()
0115 03:06:18.466631 (+ 13us) write_op.cc:276] Timestamp: P: 1736910376109794 usec, L: 0
0115 03:06:18.466722 (+ 91us) write_op.cc:270] Start()
0115 03:06:18.466735 (+ 13us) write_op.cc:276] Timestamp: P: 1736910376151374 usec, L: 0
0115 03:06:18.466826 (+ 91us) write_op.cc:270] Start()
0115 03:06:18.466840 (+ 14us) write_op.cc:276] Timestamp: P: 1736910376151797 usec, L: 0
0115 03:06:18.466915 (+ 75us) write_op.cc:270] Start()
0115 03:06:18.466927 (+ 12us) write_op.cc:276] Timestamp: P: 1736910376152199 usec, L: 0
0115 03:06:18.466997 (+ 70us) write_op.cc:270] Start()
0115 03:06:18.467010 (+ 13us) write_op.cc:276] Timestamp: P: 1736910376154515 usec, L: 0
0115 03:06:18.467107 (+ 97us) write_op.cc:270] Start()
0115 03:06:18.467122 (+ 15us) write_op.cc:276] Timestamp: P: 1736910376185870 usec, L: 0
0115 03:06:18.467208 (+ 86us) write_op.cc:270] Start()
0115 03:06:18.467219 (+ 11us) write_op.cc:276] Timestamp: P: 1736910376186316 usec, L: 0
0115 03:06:18.467285 (+ 66us) write_op.cc:270] Start()
0115 03:06:18.467295 (+ 10us) write_op.cc:276] Timestamp: P: 1736910376186743 usec, L: 0
0115 03:06:18.467356 (+ 61us) write_op.cc:270] Start()
0115 03:06:18.467367 (+ 11us) write_op.cc:276] Timestamp: P: 1736910376187142 usec, L: 0
0115 03:06:18.467444 (+ 77us) write_op.cc:270] Start()
0115 03:06:18.467467 (+ 23us) write_op.cc:276] Timestamp: P: 1736910376187557 usec, L: 0
0115 03:06:18.467544 (+ 77us) write_op.cc:270] Start()
0115 03:06:18.467557 (+ 13us) write_op.cc:276] Timestamp: P: 1736910376200261 usec, L: 0
0115 03:06:18.467734 (+ 177us) write_op.cc:270] Start()
0115 03:06:18.467750 (+ 16us) write_op.cc:276] Timestamp: P: 1736910376248859 usec, L: 0
0115 03:06:18.467836 (+ 86us) write_op.cc:270] Start()
0115 03:06:18.467851 (+ 15us) write_op.cc:276] Timestamp: P: 1736910376261246 usec, L: 0
0115 03:06:18.467952 (+ 101us) write_op.cc:270] Start()
0115 03:06:18.467968 (+ 16us) write_op.cc:276] Timestamp: P: 1736910376263018 usec, L: 0
0115 03:06:18.468051 (+ 83us) write_op.cc:270] Start()
0115 03:06:18.468066 (+ 15us) write_op.cc:276] Timestamp: P: 1736910376270990 usec, L: 0
0115 03:06:18.468145 (+ 79us) write_op.cc:270] Start()
0115 03:06:18.468159 (+ 14us) write_op.cc:276] Timestamp: P: 1736910376278068 usec, L: 0
0115 03:06:18.468233 (+ 74us) write_op.cc:270] Start()
0115 03:06:18.468245 (+ 12us) write_op.cc:276] Timestamp: P: 1736910376286825 usec, L: 0
0115 03:06:18.468324 (+ 79us) write_op.cc:270] Start()
0115 03:06:18.468337 (+ 13us) write_op.cc:276] Timestamp: P: 1736910376288438 usec, L: 0
0115 03:06:18.468402 (+ 65us) write_op.cc:270] Start()
0115 03:06:18.468413 (+ 11us) write_op.cc:276] Timestamp: P: 17369
I20250115 03:06:18.877940 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet d891020ed0bb4fc49155bd9f2f6ed35c
I20250115 03:06:18.878538 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet d891020ed0bb4fc49155bd9f2f6ed35c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:18.878902 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet d891020ed0bb4fc49155bd9f2f6ed35c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d891020ed0bb4fc49155bd9f2f6ed35c already in progress: creating tablet
I20250115 03:06:18.879189 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet d891020ed0bb4fc49155bd9f2f6ed35c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.879765 14811 tablet_bootstrap.cc:492] T 5780da67ce984297ba33d3c6901c9e3a P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:18.879730 14663 tablet_service.cc:1467] Processing CreateTablet for tablet d1c4e127c99241da950af20754e6667c (DEFAULT_TABLE table=default.Table_174d0415fa854efd9c7dd3405d04a414 [id=fc65db087e2a4b158e2db97175d1863f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.879838 14811 ts_tablet_manager.cc:1397] T 5780da67ce984297ba33d3c6901c9e3a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20250115 03:06:18.879954 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1c4e127c99241da950af20754e6667c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.880229 14811 raft_consensus.cc:357] T 5780da67ce984297ba33d3c6901c9e3a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.880317 14811 raft_consensus.cc:383] T 5780da67ce984297ba33d3c6901c9e3a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.880303 14521 tablet_service.cc:1467] Processing CreateTablet for tablet f25edfc3d7c043bfa337b31fa9131f1d (DEFAULT_TABLE table=default.Table_0c00b03c9b824693aa27df8649e12c7b [id=6215957ce1914a0996b374e39b4e9f76]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.880349 14811 raft_consensus.cc:738] T 5780da67ce984297ba33d3c6901c9e3a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.880504 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f25edfc3d7c043bfa337b31fa9131f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.880460 14811 consensus_queue.cc:260] T 5780da67ce984297ba33d3c6901c9e3a P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.880738 14811 ts_tablet_manager.cc:1428] T 5780da67ce984297ba33d3c6901c9e3a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.881117 14665 tablet_service.cc:1467] Processing CreateTablet for tablet f25edfc3d7c043bfa337b31fa9131f1d (DEFAULT_TABLE table=default.Table_0c00b03c9b824693aa27df8649e12c7b [id=6215957ce1914a0996b374e39b4e9f76]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.881311 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f25edfc3d7c043bfa337b31fa9131f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.880911 14811 tablet_bootstrap.cc:492] T a1b8b5b7fe03496ea2639dbc9c920974 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:18.883646 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 520574b17c68472e8b624c2ec02759db (DEFAULT_TABLE table=default.Table_d31b716d5bb04894865c704529827af0 [id=4a8f79d47b614b8d8cc9109d3a2ff589]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.883832 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 520574b17c68472e8b624c2ec02759db. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.881120 14522 tablet_service.cc:1467] Processing CreateTablet for tablet d1c4e127c99241da950af20754e6667c (DEFAULT_TABLE table=default.Table_174d0415fa854efd9c7dd3405d04a414 [id=fc65db087e2a4b158e2db97175d1863f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.884043 14811 tablet_bootstrap.cc:654] T a1b8b5b7fe03496ea2639dbc9c920974 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.884110 14522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1c4e127c99241da950af20754e6667c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.885850 14397 tablet_service.cc:1467] Processing CreateTablet for tablet f25edfc3d7c043bfa337b31fa9131f1d (DEFAULT_TABLE table=default.Table_0c00b03c9b824693aa27df8649e12c7b [id=6215957ce1914a0996b374e39b4e9f76]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.881238 14520 tablet_service.cc:1467] Processing CreateTablet for tablet 4febfae2f2794669b5997a1fa7ef7412 (DEFAULT_TABLE table=default.Table_1df66a4877bb4ba993ae442eb173bf82 [id=7bec6149db914c939bc6fd50e47097b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.885972 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 4febfae2f2794669b5997a1fa7ef7412 (DEFAULT_TABLE table=default.Table_1df66a4877bb4ba993ae442eb173bf82 [id=7bec6149db914c939bc6fd50e47097b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.886060 14520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4febfae2f2794669b5997a1fa7ef7412. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.886060 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f25edfc3d7c043bfa337b31fa9131f1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.886161 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4febfae2f2794669b5997a1fa7ef7412. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.886363 14400 tablet_service.cc:1467] Processing CreateTablet for tablet d1c4e127c99241da950af20754e6667c (DEFAULT_TABLE table=default.Table_174d0415fa854efd9c7dd3405d04a414 [id=fc65db087e2a4b158e2db97175d1863f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.886571 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1c4e127c99241da950af20754e6667c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.887913 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 520574b17c68472e8b624c2ec02759db (DEFAULT_TABLE table=default.Table_d31b716d5bb04894865c704529827af0 [id=4a8f79d47b614b8d8cc9109d3a2ff589]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.888090 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 520574b17c68472e8b624c2ec02759db. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.881357 14519 tablet_service.cc:1467] Processing CreateTablet for tablet 520574b17c68472e8b624c2ec02759db (DEFAULT_TABLE table=default.Table_d31b716d5bb04894865c704529827af0 [id=4a8f79d47b614b8d8cc9109d3a2ff589]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.888295 14519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 520574b17c68472e8b624c2ec02759db. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.881467 14523 tablet_service.cc:1467] Processing CreateTablet for tablet 1fd348b3ae2a45dfa05c52f02536a201 (DEFAULT_TABLE table=default.Table_cf7f3fbd11804e94848a7c5357169a86 [id=d64515775f264a419768662b8ce0a854]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.889256 14523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fd348b3ae2a45dfa05c52f02536a201. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:18.890178 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet d891020ed0bb4fc49155bd9f2f6ed35c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d891020ed0bb4fc49155bd9f2f6ed35c already in progress: creating tablet
I20250115 03:06:18.892004 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 4febfae2f2794669b5997a1fa7ef7412 (DEFAULT_TABLE table=default.Table_1df66a4877bb4ba993ae442eb173bf82 [id=7bec6149db914c939bc6fd50e47097b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.892187 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4febfae2f2794669b5997a1fa7ef7412. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.893774 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_91812981cd64473f84541f170fb473bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:18.893913 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f2ff78f7081d4b56a30785f4d6ae8e71"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.894083 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91812981cd64473f84541f170fb473bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:18.894214 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f2ff78f7081d4b56a30785f4d6ae8e71 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.894726 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_4946942766de43b7b86e7c77ab3f850c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:18.894860 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0e3a624b9d9a460184f8686f35ec2a05"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.894973 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4946942766de43b7b86e7c77ab3f850c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:18.895099 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0e3a624b9d9a460184f8686f35ec2a05 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.895532 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_551ca9fefcb74ed5aa29803f19de6ab6" } new_table_name: "default.Table_d40762fac357436da2798c2bb019dff6" modify_external_catalogs: true
I20250115 03:06:18.897150 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ff0aab929942464285edb009a9e2e049" } new_table_name: "default.Table_e539b5abff2f433da0b195f3a6fb9765" modify_external_catalogs: true
I20250115 03:06:18.881712 14524 tablet_service.cc:1514] Processing DeleteTablet for tablet d891020ed0bb4fc49155bd9f2f6ed35c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.897974 14809 tablet_bootstrap.cc:492] T 1f2dc6928e2941ef9b5b5a7891376500 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:18.898042 14809 ts_tablet_manager.cc:1397] T 1f2dc6928e2941ef9b5b5a7891376500 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.116s user 0.000s sys 0.001s
I20250115 03:06:18.898278 16029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "147c86b98f604d518e71f2ba3121fc21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:18.898365 16029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [sys.catalog]: This master's current role is: LEADER
I20250115 03:06:18.898386 14809 raft_consensus.cc:357] T 1f2dc6928e2941ef9b5b5a7891376500 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.898465 14809 raft_consensus.cc:383] T 1f2dc6928e2941ef9b5b5a7891376500 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.898492 14809 raft_consensus.cc:738] T 1f2dc6928e2941ef9b5b5a7891376500 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.898587 14809 consensus_queue.cc:260] T 1f2dc6928e2941ef9b5b5a7891376500 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.898818 14809 ts_tablet_manager.cc:1428] T 1f2dc6928e2941ef9b5b5a7891376500 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.898960 14809 tablet_bootstrap.cc:492] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:18.899152 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 5780da67ce984297ba33d3c6901c9e3a (table default.Table_551ca9fefcb74ed5aa29803f19de6ab6 [id=79eb8c9694b74a31a4253b33d064e230]). Expected version 1 got 0
I20250115 03:06:18.899631 14809 tablet_bootstrap.cc:654] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.905114 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f2dc6928e2941ef9b5b5a7891376500 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f9818d0493547358fd5654a01d4f456) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:18.905251 14819 tablet_replica.cc:331] T 1f2dc6928e2941ef9b5b5a7891376500 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:18.905346 14819 raft_consensus.cc:2238] T 1f2dc6928e2941ef9b5b5a7891376500 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.905433 14819 raft_consensus.cc:2267] T 1f2dc6928e2941ef9b5b5a7891376500 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.911026 14518 tablet_service.cc:1514] Processing DeleteTablet for tablet 31446d60b85d4a878e9afe142011aeb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:18.911279 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 31b5928658fb40dc9bb72558b11767dc (DEFAULT_TABLE table=default.Table_2e466ac2b74a4a1b8a5b12182123fd7c [id=414ee4da78a64bc8944ac0df9d841945]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.911501 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31b5928658fb40dc9bb72558b11767dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.911854 14819 ts_tablet_manager.cc:1905] T 1f2dc6928e2941ef9b5b5a7891376500 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:18.911971 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 96656c3217354b468df86d96d4802883 has no consensus state
I20250115 03:06:18.913758 14819 ts_tablet_manager.cc:1918] T 1f2dc6928e2941ef9b5b5a7891376500 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.913817 14819 log.cc:1198] T 1f2dc6928e2941ef9b5b5a7891376500 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/1f2dc6928e2941ef9b5b5a7891376500
I20250115 03:06:18.913998 14819 ts_tablet_manager.cc:1939] T 1f2dc6928e2941ef9b5b5a7891376500 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:18.914295 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 1fd348b3ae2a45dfa05c52f02536a201 (DEFAULT_TABLE table=default.Table_cf7f3fbd11804e94848a7c5357169a86 [id=d64515775f264a419768662b8ce0a854]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.914506 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fd348b3ae2a45dfa05c52f02536a201. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.914949 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 8d5ea34c28e940c98b4ac8fee80a600b (DEFAULT_TABLE table=default.Table_30324f936db349899fc29c4213990c43 [id=8c508f46cd2446bcaee8691af2897199]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.915141 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d5ea34c28e940c98b4ac8fee80a600b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.916505 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 00aad81f4ca3427cb683de71148cff5c (DEFAULT_TABLE table=default.Table_03db1a048e7a4aff8c957fbc41cb4675 [id=77a560c93816435894621976eaaf0d05]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.916702 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00aad81f4ca3427cb683de71148cff5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.918586 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 00aad81f4ca3427cb683de71148cff5c (DEFAULT_TABLE table=default.Table_03db1a048e7a4aff8c957fbc41cb4675 [id=77a560c93816435894621976eaaf0d05]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.918788 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00aad81f4ca3427cb683de71148cff5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.919884 14666 tablet_service.cc:1467] Processing CreateTablet for tablet 3724ff14f535416e859339a137c05050 (DEFAULT_TABLE table=default.Table_f6d9aee213514fd9a38318e1fc091215 [id=adfd7e86aa0f48e5b5e36b4113517dba]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.920091 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3724ff14f535416e859339a137c05050. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.920244 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 29c72f33a2034c4eae81c7dfcca27ca4 (DEFAULT_TABLE table=default.Table_2cb3931be82e48859ff5afc497e38338 [id=eda284c3f34d4ee4aa9aa36c29f0e6cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.920433 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29c72f33a2034c4eae81c7dfcca27ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.921032 15617 tablet.cc:1721] T 5780da67ce984297ba33d3c6901c9e3a P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:18.921108 14662 tablet_service.cc:1467] Processing CreateTablet for tablet e522484f4cbc40c289a3661a42bdaef3 (DEFAULT_TABLE table=default.Table_f889454d62ab418fabd1240a45fa959f [id=f5eceab19dec473dacf35e0742cd2c28]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.921239 15970 tablet.cc:1721] T 5780da67ce984297ba33d3c6901c9e3a P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:18.921478 14398 tablet_service.cc:1467] Processing CreateTablet for tablet ef7cd35eb1614a29a9197e9c03185f7d (DEFAULT_TABLE table=default.Table_397a360e045b4b5fabed16c0feff8112 [id=e6e7970abc534933b349d57c0345f205]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.921677 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef7cd35eb1614a29a9197e9c03185f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.923506 14672 tablet_service.cc:1467] Processing CreateTablet for tablet ef7cd35eb1614a29a9197e9c03185f7d (DEFAULT_TABLE table=default.Table_397a360e045b4b5fabed16c0feff8112 [id=e6e7970abc534933b349d57c0345f205]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.921288 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e522484f4cbc40c289a3661a42bdaef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.925918 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 3724ff14f535416e859339a137c05050 (DEFAULT_TABLE table=default.Table_f6d9aee213514fd9a38318e1fc091215 [id=adfd7e86aa0f48e5b5e36b4113517dba]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.926110 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3724ff14f535416e859339a137c05050. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.926491 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef7cd35eb1614a29a9197e9c03185f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.928033 14397 tablet_service.cc:1467] Processing CreateTablet for tablet c07e0a0d439041cdbe089b3de4597098 (DEFAULT_TABLE table=default.Table_2773b577ce554a45834f200dca2c2987 [id=8005446109b64adabae59e8612c35f58]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.928218 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c07e0a0d439041cdbe089b3de4597098. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.928993 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a57c628374dc4ba2b91cb5c9a3ce8836" } new_table_name: "default.Table_3ca292c29f184579b5fd9fe897150035" modify_external_catalogs: true
I20250115 03:06:18.930079 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 29c72f33a2034c4eae81c7dfcca27ca4 (DEFAULT_TABLE table=default.Table_2cb3931be82e48859ff5afc497e38338 [id=eda284c3f34d4ee4aa9aa36c29f0e6cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.930472 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_c7c26755389347ff821c3e4cd1448247" } new_table_name: "default.Table_d64c7e1e71a3478687247bda551f3fe2" modify_external_catalogs: true
I20250115 03:06:18.931141 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 1f2dc6928e2941ef9b5b5a7891376500 (table default.Table_652f8e06e005491381404a11759aefb1 [id=6fb6e311a0af4f51b190a880f84373d1]) successfully deleted
I20250115 03:06:18.933020 14812 tablet_bootstrap.cc:492] T 7d894471f7b94aec8a19df043906d7e7 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:18.933100 14812 ts_tablet_manager.cc:1397] T 7d894471f7b94aec8a19df043906d7e7 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.102s user 0.002s sys 0.000s
I20250115 03:06:18.933521 14812 raft_consensus.cc:357] T 7d894471f7b94aec8a19df043906d7e7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.933611 14812 raft_consensus.cc:383] T 7d894471f7b94aec8a19df043906d7e7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.933640 14812 raft_consensus.cc:738] T 7d894471f7b94aec8a19df043906d7e7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.933746 14812 consensus_queue.cc:260] T 7d894471f7b94aec8a19df043906d7e7 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.933831 14402 tablet_service.cc:1467] Processing CreateTablet for tablet e522484f4cbc40c289a3661a42bdaef3 (DEFAULT_TABLE table=default.Table_f889454d62ab418fabd1240a45fa959f [id=f5eceab19dec473dacf35e0742cd2c28]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.933933 14168 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_d40762fac357436da2798c2bb019dff6 [id=79eb8c9694b74a31a4253b33d064e230] alter complete (version 2)
I20250115 03:06:18.933997 14812 ts_tablet_manager.cc:1428] T 7d894471f7b94aec8a19df043906d7e7 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.934024 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e522484f4cbc40c289a3661a42bdaef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.935876 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 8d5ea34c28e940c98b4ac8fee80a600b (DEFAULT_TABLE table=default.Table_30324f936db349899fc29c4213990c43 [id=8c508f46cd2446bcaee8691af2897199]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.936084 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d5ea34c28e940c98b4ac8fee80a600b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.937565 14809 tablet_bootstrap.cc:492] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:18.937644 14809 ts_tablet_manager.cc:1397] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.039s user 0.001s sys 0.000s
I20250115 03:06:18.938033 14809 raft_consensus.cc:357] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:18.938093 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 781f6a9148cd4bd684f24c294c0b019b consensus state has no leader
I20250115 03:06:18.938123 14809 raft_consensus.cc:383] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.938153 14809 raft_consensus.cc:738] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.938256 14809 consensus_queue.cc:260] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.938514 14809 ts_tablet_manager.cc:1428] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:18.938683 14809 tablet_bootstrap.cc:492] T 3b9c09b6315345039cdea9eeee3f181a P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:18.939538 14809 tablet_bootstrap.cc:654] T 3b9c09b6315345039cdea9eeee3f181a P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.943912 14811 tablet_bootstrap.cc:492] T a1b8b5b7fe03496ea2639dbc9c920974 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:18.943996 14811 ts_tablet_manager.cc:1397] T a1b8b5b7fe03496ea2639dbc9c920974 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.063s user 0.000s sys 0.002s
I20250115 03:06:18.944434 14811 raft_consensus.cc:357] T a1b8b5b7fe03496ea2639dbc9c920974 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.944533 14811 raft_consensus.cc:383] T a1b8b5b7fe03496ea2639dbc9c920974 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.944562 14811 raft_consensus.cc:738] T a1b8b5b7fe03496ea2639dbc9c920974 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.944535 14666 tablet_service.cc:1467] Processing CreateTablet for tablet c164db2cf23149258bdc14e68def00b3 (DEFAULT_TABLE table=default.Table_b94f9ef5ede64627b994e94e674b1cfa [id=3f4310bb3227447ba6fcd011f05d6924]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.944666 14811 consensus_queue.cc:260] T a1b8b5b7fe03496ea2639dbc9c920974 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.944917 14811 ts_tablet_manager.cc:1428] T a1b8b5b7fe03496ea2639dbc9c920974 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:18.945113 14811 tablet_bootstrap.cc:492] T 8efd181c42eb42628a9f36fad4fa8d64 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:18.947036 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b3d1f8a0bba14291b9577e6201a2bb06"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.947327 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b3d1f8a0bba14291b9577e6201a2bb06 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.947489 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b20b6ea81915403f986c91997f637c86" } new_table_name: "default.Table_6e2e63a19a64494884fc71e4b3364978" modify_external_catalogs: true
I20250115 03:06:18.947760 14662 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d894471f7b94aec8a19df043906d7e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 194d943d173340e7a8fba69967d9b090) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:18.948967 14422 raft_consensus.cc:1270] T 01433c76a398462bb6708a02997047fa P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 5c0dae7aff4b44c699275b95ba639770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:18.949386 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c164db2cf23149258bdc14e68def00b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.950090 15325 consensus_queue.cc:1035] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:18.950804 14401 tablet_service.cc:1467] Processing CreateTablet for tablet c164db2cf23149258bdc14e68def00b3 (DEFAULT_TABLE table=default.Table_b94f9ef5ede64627b994e94e674b1cfa [id=3f4310bb3227447ba6fcd011f05d6924]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.951206 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29c72f33a2034c4eae81c7dfcca27ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.951397 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c164db2cf23149258bdc14e68def00b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.951639 14812 tablet_bootstrap.cc:492] T cff95c69fb9045cf9ec0ebb128c9ef6b P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:18.952906 14812 tablet_bootstrap.cc:654] T cff95c69fb9045cf9ec0ebb128c9ef6b P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.953761 14823 tablet_replica.cc:331] T 7d894471f7b94aec8a19df043906d7e7 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:18.953809 14669 tablet_service.cc:1467] Processing CreateTablet for tablet c07e0a0d439041cdbe089b3de4597098 (DEFAULT_TABLE table=default.Table_2773b577ce554a45834f200dca2c2987 [id=8005446109b64adabae59e8612c35f58]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.953872 14823 raft_consensus.cc:2238] T 7d894471f7b94aec8a19df043906d7e7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.953955 14823 raft_consensus.cc:2267] T 7d894471f7b94aec8a19df043906d7e7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:18.954129 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 1fd348b3ae2a45dfa05c52f02536a201 (DEFAULT_TABLE table=default.Table_cf7f3fbd11804e94848a7c5357169a86 [id=d64515775f264a419768662b8ce0a854]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.954316 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c07e0a0d439041cdbe089b3de4597098. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.957612 14524 tablet_service.cc:1467] Processing CreateTablet for tablet 00aad81f4ca3427cb683de71148cff5c (DEFAULT_TABLE table=default.Table_03db1a048e7a4aff8c957fbc41cb4675 [id=77a560c93816435894621976eaaf0d05]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.957975 14519 tablet_service.cc:1467] Processing CreateTablet for tablet e522484f4cbc40c289a3661a42bdaef3 (DEFAULT_TABLE table=default.Table_f889454d62ab418fabd1240a45fa959f [id=f5eceab19dec473dacf35e0742cd2c28]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.958181 14554 raft_consensus.cc:3054] T e63e854c49b34c1ab286f1f97fe443ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:18.958318 14523 tablet_service.cc:1467] Processing CreateTablet for tablet c07e0a0d439041cdbe089b3de4597098 (DEFAULT_TABLE table=default.Table_2773b577ce554a45834f200dca2c2987 [id=8005446109b64adabae59e8612c35f58]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.958631 14520 tablet_service.cc:1467] Processing CreateTablet for tablet ef7cd35eb1614a29a9197e9c03185f7d (DEFAULT_TABLE table=default.Table_397a360e045b4b5fabed16c0feff8112 [id=e6e7970abc534933b349d57c0345f205]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.958967 14522 tablet_service.cc:1467] Processing CreateTablet for tablet 3724ff14f535416e859339a137c05050 (DEFAULT_TABLE table=default.Table_f6d9aee213514fd9a38318e1fc091215 [id=adfd7e86aa0f48e5b5e36b4113517dba]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.959271 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 29c72f33a2034c4eae81c7dfcca27ca4 (DEFAULT_TABLE table=default.Table_2cb3931be82e48859ff5afc497e38338 [id=eda284c3f34d4ee4aa9aa36c29f0e6cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:18.959578 14525 tablet_service.cc:1467] Processing CreateTablet for tablet c164db2cf23149258bdc14e68def00b3 (DEFAULT_TABLE table=default.Table_b94f9ef5ede64627b994e94e674b1cfa [id=3f4310bb3227447ba6fcd011f05d6924]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:18.959864 14626 consensus_peers.cc:487] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:18.959949 14525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c164db2cf23149258bdc14e68def00b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.961910 14523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c07e0a0d439041cdbe089b3de4597098. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.961930 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fd348b3ae2a45dfa05c52f02536a201. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.962126 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29c72f33a2034c4eae81c7dfcca27ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.963764 14520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef7cd35eb1614a29a9197e9c03185f7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.965188 14811 tablet_bootstrap.cc:654] T 8efd181c42eb42628a9f36fad4fa8d64 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:18.965646 14522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3724ff14f535416e859339a137c05050. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.967669 14823 ts_tablet_manager.cc:1905] T 7d894471f7b94aec8a19df043906d7e7 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.969614 14823 ts_tablet_manager.cc:1918] T 7d894471f7b94aec8a19df043906d7e7 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.969683 14823 log.cc:1198] T 7d894471f7b94aec8a19df043906d7e7 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/7d894471f7b94aec8a19df043906d7e7
I20250115 03:06:18.969900 14823 ts_tablet_manager.cc:1939] T 7d894471f7b94aec8a19df043906d7e7 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:18.971125 14519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e522484f4cbc40c289a3661a42bdaef3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.972908 14524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00aad81f4ca3427cb683de71148cff5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:18.973816 14820 ts_tablet_manager.cc:1918] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:18.973874 14820 log.cc:1198] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/d0c3536533e94174bb1e3888a3458395
I20250115 03:06:18.974040 14820 ts_tablet_manager.cc:1939] T d0c3536533e94174bb1e3888a3458395 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:18.976677 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet d28ed8538c4a44d18279371b826c22a5 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:18.976830 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0816f2fcac824630b5042becb8b0ed18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:18.977166 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0816f2fcac824630b5042becb8b0ed18 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:18.978505 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 7d894471f7b94aec8a19df043906d7e7 (table default.Table_1a2ec424c5cb40fbb3c93f327531e36f [id=50a51914c37b462aa45a71fb260c809b]) successfully deleted
I20250115 03:06:18.979816 15970 tablet.cc:1721] T 7c503b29cb2843338fb1870a89aee443 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:18.980257 15617 tablet.cc:1721] T 7c503b29cb2843338fb1870a89aee443 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:18.982426 14853 raft_consensus.cc:491] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.982504 14853 raft_consensus.cc:513] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.982822 14853 leader_election.cc:290] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:18.983587 14493 leader_election.cc:343] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 56c600fe18504e05bab2bcde3c89861d
W20250115 03:06:18.983980 14493 leader_election.cc:343] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 56c600fe18504e05bab2bcde3c89861d
I20250115 03:06:18.984037 14493 leader_election.cc:304] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:18.984225 14853 raft_consensus.cc:2743] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:18.985190 14167 master_service.cc:946] ReplaceTablet: received request to replace tablet 0e1ce43146ed4e4d82a7c134216e2872 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:18.986127 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet d0c3536533e94174bb1e3888a3458395 (table default.Table_1c52ca26880a48289f5916fabdb05ac4 [id=44c03f21942545319762494029cebc02]) successfully deleted
I20250115 03:06:18.986199 14820 tablet_replica.cc:331] T ae0efe52ac6c40fbae45c725b07b03a3 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:18.986407 14820 raft_consensus.cc:2238] T ae0efe52ac6c40fbae45c725b07b03a3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:18.986590 14820 raft_consensus.cc:2267] T ae0efe52ac6c40fbae45c725b07b03a3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:18.986727 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 3007a0506ce14c6c89c600b98a81b168 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3007a0506ce14c6c89c600b98a81b168 already in progress: creating tablet
I20250115 03:06:18.987136 14820 ts_tablet_manager.cc:1905] T ae0efe52ac6c40fbae45c725b07b03a3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:18.990288 14820 ts_tablet_manager.cc:1918] T ae0efe52ac6c40fbae45c725b07b03a3 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:18.990446 14820 log.cc:1198] T ae0efe52ac6c40fbae45c725b07b03a3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/ae0efe52ac6c40fbae45c725b07b03a3
I20250115 03:06:18.991027 14820 ts_tablet_manager.cc:1939] T ae0efe52ac6c40fbae45c725b07b03a3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:18.991732 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0ab5bfc343e6431890e7f64f444e1c05" } modify_external_catalogs: true
I20250115 03:06:18.996788 14853 raft_consensus.cc:491] T edf17b41198b4f8fbe5db22c58aa6bb2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:18.996865 14853 raft_consensus.cc:513] T edf17b41198b4f8fbe5db22c58aa6bb2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.997172 14853 leader_election.cc:290] T edf17b41198b4f8fbe5db22c58aa6bb2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:18.997473 14809 tablet_bootstrap.cc:492] T 3b9c09b6315345039cdea9eeee3f181a P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:18.997558 14809 ts_tablet_manager.cc:1397] T 3b9c09b6315345039cdea9eeee3f181a P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.059s user 0.001s sys 0.000s
I20250115 03:06:18.997756 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edf17b41198b4f8fbe5db22c58aa6bb2" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:18.997951 14809 raft_consensus.cc:357] T 3b9c09b6315345039cdea9eeee3f181a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.998042 14809 raft_consensus.cc:383] T 3b9c09b6315345039cdea9eeee3f181a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:18.998072 14809 raft_consensus.cc:738] T 3b9c09b6315345039cdea9eeee3f181a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:18.998180 14809 consensus_queue.cc:260] T 3b9c09b6315345039cdea9eeee3f181a P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:18.998420 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edf17b41198b4f8fbe5db22c58aa6bb2" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:18.998812 14493 leader_election.cc:304] T edf17b41198b4f8fbe5db22c58aa6bb2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:18.999102 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0ab5bfc343e6431890e7f64f444e1c05" } modify_external_catalogs: true
I20250115 03:06:19.005594 14853 raft_consensus.cc:2743] T edf17b41198b4f8fbe5db22c58aa6bb2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:19.006079 14820 tablet_replica.cc:331] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.006243 14820 raft_consensus.cc:2238] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.006377 14820 raft_consensus.cc:2267] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.006742 14820 ts_tablet_manager.cc:1905] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.007709 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 82bf1d7ac9fe4a2eb7b57cf1ab627c3d
I20250115 03:06:18.998454 14809 ts_tablet_manager.cc:1428] T 3b9c09b6315345039cdea9eeee3f181a P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.009034 14809 tablet_bootstrap.cc:492] T cc67fb818b8146629d92f9ac1944dba6 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.009207 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet d28ed8538c4a44d18279371b826c22a5
W20250115 03:06:19.009266 14163 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9c436975bd034068a6d58d5831db1896 consensus state has no leader
I20250115 03:06:19.009883 14809 tablet_bootstrap.cc:654] T cc67fb818b8146629d92f9ac1944dba6 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.010236 15970 tablet.cc:1721] T 01433c76a398462bb6708a02997047fa P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:19.012799 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e9d0346fd3064c8d92a35b495ae3a3c8" } modify_external_catalogs: true
I20250115 03:06:19.012794 15617 tablet.cc:1721] T 01433c76a398462bb6708a02997047fa P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:19.012902 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e9d0346fd3064c8d92a35b495ae3a3c8" } modify_external_catalogs: true
I20250115 03:06:19.013620 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ea36537d443240e99ab0ff1719ab9939"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:19.014081 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet adbb0ae719f24dcc96709a83981e213d from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:19.014433 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e1ceae92543048f39acd7f81454b14f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:19.017721 14166 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5ddb4f54525d4439b54c2c898471fa7e" } modify_external_catalogs: true
I20250115 03:06:19.017813 14166 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5ddb4f54525d4439b54c2c898471fa7e" } modify_external_catalogs: true
I20250115 03:06:19.017818 14820 ts_tablet_manager.cc:1918] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.017887 14820 log.cc:1198] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/341180618b4748eabc568888e2e6a7df
I20250115 03:06:19.018074 14820 ts_tablet_manager.cc:1939] T 341180618b4748eabc568888e2e6a7df P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.018595 14820 tablet_replica.cc:331] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.018908 14164 catalog_manager.cc:6596] ReplaceTablet: tablet d28ed8538c4a44d18279371b826c22a5 deleted and replaced by tablet 55538831d8ea47eda40c36d42c19435f
I20250115 03:06:19.019104 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_debb6a0c409a4d8383c41a628fcb8450"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.019510 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_debb6a0c409a4d8383c41a628fcb8450 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:19.019627 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e1ceae92543048f39acd7f81454b14f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:19.020190 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ea36537d443240e99ab0ff1719ab9939 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.020555 14820 raft_consensus.cc:2238] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.020676 14820 raft_consensus.cc:2267] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.018930 14554 raft_consensus.cc:3054] T edf17b41198b4f8fbe5db22c58aa6bb2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:19.020860 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d894471f7b94aec8a19df043906d7e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 194d943d173340e7a8fba69967d9b090) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.021028 14820 ts_tablet_manager.cc:1905] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.022850 14820 ts_tablet_manager.cc:1918] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:19.026806 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet d28ed8538c4a44d18279371b826c22a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 55538831d8ea47eda40c36d42c19435f) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.026914 14402 tablet_service.cc:1514] Processing DeleteTablet for tablet 82bf1d7ac9fe4a2eb7b57cf1ab627c3d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.026988 14819 tablet_replica.cc:331] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.027108 14819 raft_consensus.cc:2238] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:19.027285 14819 raft_consensus.cc:2267] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:19.027413 14158 catalog_manager.cc:4839] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 7d894471f7b94aec8a19df043906d7e7 because the tablet was not found. No further retry: Not found: Tablet not found: 7d894471f7b94aec8a19df043906d7e7
I20250115 03:06:19.027582 14170 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_dfcccd3390fa4814a0a04bb1ff5e43fa [id=f0a7068682b9481a82fa34cb54ed528e] alter complete (version 1)
I20250115 03:06:19.027632 14819 ts_tablet_manager.cc:1905] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.027884 14669 tablet_service.cc:1514] Processing DeleteTablet for tablet 82bf1d7ac9fe4a2eb7b57cf1ab627c3d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.028023 14823 tablet_replica.cc:331] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:19.028129 14823 raft_consensus.cc:2238] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:19.028263 14823 pending_rounds.cc:62] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 1 pending ops.
I20250115 03:06:19.028292 14823 pending_rounds.cc:69] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384911332016128 op_type: NO_OP noop_request { }
I20250115 03:06:19.028342 14823 raft_consensus.cc:2883] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:19.028390 14823 raft_consensus.cc:2267] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.029830 14819 ts_tablet_manager.cc:1918] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:19.029892 14819 log.cc:1198] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/d28ed8538c4a44d18279371b826c22a5
I20250115 03:06:19.030115 14819 ts_tablet_manager.cc:1939] T d28ed8538c4a44d18279371b826c22a5 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.030630 14819 tablet_replica.cc:331] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.030752 14819 raft_consensus.cc:2238] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.030836 14819 raft_consensus.cc:2267] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.031795 14819 ts_tablet_manager.cc:1905] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:19.031912 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 4c4ce5c2921341ce837ee4f5f7d131b7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4c4ce5c2921341ce837ee4f5f7d131b7 already in progress: creating tablet
I20250115 03:06:19.032096 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 341180618b4748eabc568888e2e6a7df (table default.Table_65f0fd5511f64bfeb6539f9019a4873c [id=7191c3890ae642558fe99561455d01e7]) successfully deleted
I20250115 03:06:19.032507 14665 tablet_service.cc:1514] Processing DeleteTablet for tablet d28ed8538c4a44d18279371b826c22a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 55538831d8ea47eda40c36d42c19435f) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.033819 14819 ts_tablet_manager.cc:1918] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.033895 14819 log.cc:1198] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/82bf1d7ac9fe4a2eb7b57cf1ab627c3d
I20250115 03:06:19.034098 14819 ts_tablet_manager.cc:1939] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.034456 15390 raft_consensus.cc:491] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.034531 15390 raft_consensus.cc:513] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.034898 15390 leader_election.cc:290] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:19.035319 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "907301ca08e0467f9f29c6dcb36bdd5e" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:19.035436 14689 raft_consensus.cc:2463] T 907301ca08e0467f9f29c6dcb36bdd5e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:19.035703 14358 leader_election.cc:304] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:19.035889 15390 raft_consensus.cc:2798] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:19.035944 15390 raft_consensus.cc:491] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:19.035979 15390 raft_consensus.cc:3054] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:19.036792 15390 raft_consensus.cc:513] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.037114 15390 leader_election.cc:290] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:19.037634 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "907301ca08e0467f9f29c6dcb36bdd5e" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:19.037736 14689 raft_consensus.cc:3054] T 907301ca08e0467f9f29c6dcb36bdd5e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:19.038628 14689 raft_consensus.cc:2463] T 907301ca08e0467f9f29c6dcb36bdd5e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 1.
I20250115 03:06:19.032372 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet d28ed8538c4a44d18279371b826c22a5 (table default.Table_fe0a1f3ae7a04225a5a482c33a15c579 [id=c99c3573d2394eada9631427ac38322d]) successfully deleted
I20250115 03:06:19.038913 14358 leader_election.cc:304] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:19.039057 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 82bf1d7ac9fe4a2eb7b57cf1ab627c3d (table default.Table_0ab5bfc343e6431890e7f64f444e1c05 [id=2f9b0aa8377744488a89ba41fb20d5c3]) successfully deleted
I20250115 03:06:19.039131 15390 raft_consensus.cc:2798] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:19.039209 15390 raft_consensus.cc:695] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Becoming Leader. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Running, Role: LEADER
I20250115 03:06:19.039319 15390 consensus_queue.cc:237] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.039762 14823 ts_tablet_manager.cc:1905] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.042011 14823 ts_tablet_manager.cc:1918] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:19.042272 14823 log.cc:1198] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/82bf1d7ac9fe4a2eb7b57cf1ab627c3d
I20250115 03:06:19.042366 14820 log.cc:1198] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/df44c5f5a9ae4d1aa7647dcbe2303a39
I20250115 03:06:19.042682 14823 ts_tablet_manager.cc:1939] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:19.043437 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 82bf1d7ac9fe4a2eb7b57cf1ab627c3d (table default.Table_0ab5bfc343e6431890e7f64f444e1c05 [id=2f9b0aa8377744488a89ba41fb20d5c3]) successfully deleted
I20250115 03:06:19.025717 14523 tablet_service.cc:1514] Processing DeleteTablet for tablet d28ed8538c4a44d18279371b826c22a5 with delete_type TABLET_DATA_DELETED (replaced by tablet 55538831d8ea47eda40c36d42c19435f) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.035899 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "907301ca08e0467f9f29c6dcb36bdd5e" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:19.043787 14820 ts_tablet_manager.cc:1939] T df44c5f5a9ae4d1aa7647dcbe2303a39 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.037351 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "907301ca08e0467f9f29c6dcb36bdd5e" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:19.044164 14356 leader_election.cc:343] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:19.044337 14356 leader_election.cc:343] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.025643 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet 82bf1d7ac9fe4a2eb7b57cf1ab627c3d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.044677 14823 tablet_replica.cc:331] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:19.044836 14823 raft_consensus.cc:2238] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.045071 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet df44c5f5a9ae4d1aa7647dcbe2303a39 (table default.Table_098c9884c0494d4592f142b4a197f38c [id=8bf1da8d633940258affd0e11c334786]) successfully deleted
I20250115 03:06:19.045168 14885 raft_consensus.cc:491] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.045238 14885 raft_consensus.cc:513] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.045372 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet d6cf1a0b9c81413b8f72de0cafc76963 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:19.045534 14885 leader_election.cc:290] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:19.047895 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5655935aabd48a19d0ef03739e089a4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:19.048220 14820 tablet_replica.cc:331] T a1b8b5b7fe03496ea2639dbc9c920974 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
W20250115 03:06:19.048231 14356 leader_election.cc:343] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.048398 14820 raft_consensus.cc:2238] T a1b8b5b7fe03496ea2639dbc9c920974 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.048534 14820 raft_consensus.cc:2267] T a1b8b5b7fe03496ea2639dbc9c920974 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.049641 14170 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_3ca292c29f184579b5fd9fe897150035 [id=960713d5552d4e339aa8c94edbe9c7b7] alter complete (version 1)
I20250115 03:06:19.050581 14809 tablet_bootstrap.cc:492] T cc67fb818b8146629d92f9ac1944dba6 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
W20250115 03:06:19.050621 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 9bb0b7a8acfd40738abcc951936b1116 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9bb0b7a8acfd40738abcc951936b1116 already in progress: creating tablet
W20250115 03:06:19.050751 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet adbb0ae719f24dcc96709a83981e213d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet adbb0ae719f24dcc96709a83981e213d already in progress: creating tablet
I20250115 03:06:19.050760 14809 ts_tablet_manager.cc:1397] T cc67fb818b8146629d92f9ac1944dba6 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.042s user 0.002s sys 0.000s
W20250115 03:06:19.050849 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet cff95c69fb9045cf9ec0ebb128c9ef6b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cff95c69fb9045cf9ec0ebb128c9ef6b already in progress: creating tablet
W20250115 03:06:19.050940 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet d12d1f964d2b486bb703ed98cb2e7949 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d12d1f964d2b486bb703ed98cb2e7949 already in progress: creating tablet
W20250115 03:06:19.051056 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet cef4442bfcfd4ad4b343a2785c3a9a74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cef4442bfcfd4ad4b343a2785c3a9a74 already in progress: creating tablet
I20250115 03:06:19.051412 14809 raft_consensus.cc:357] T cc67fb818b8146629d92f9ac1944dba6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.051577 14809 raft_consensus.cc:383] T cc67fb818b8146629d92f9ac1944dba6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.051645 14809 raft_consensus.cc:738] T cc67fb818b8146629d92f9ac1944dba6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.051833 14809 consensus_queue.cc:260] T cc67fb818b8146629d92f9ac1944dba6 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.052227 14809 ts_tablet_manager.cc:1428] T cc67fb818b8146629d92f9ac1944dba6 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.052420 14809 tablet_bootstrap.cc:492] T 80aa570e2c154172ad6da29d268bede7 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.053368 14518 tablet_service.cc:1467] Processing CreateTablet for tablet 8d5ea34c28e940c98b4ac8fee80a600b (DEFAULT_TABLE table=default.Table_30324f936db349899fc29c4213990c43 [id=8c508f46cd2446bcaee8691af2897199]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.053622 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d5ea34c28e940c98b4ac8fee80a600b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.054050 14809 tablet_bootstrap.cc:654] T 80aa570e2c154172ad6da29d268bede7 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.055467 14809 tablet_bootstrap.cc:492] T 80aa570e2c154172ad6da29d268bede7 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.055600 14809 ts_tablet_manager.cc:1397] T 80aa570e2c154172ad6da29d268bede7 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:19.056557 14809 raft_consensus.cc:357] T 80aa570e2c154172ad6da29d268bede7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.056705 14809 raft_consensus.cc:383] T 80aa570e2c154172ad6da29d268bede7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.056782 14809 raft_consensus.cc:738] T 80aa570e2c154172ad6da29d268bede7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.057003 14809 consensus_queue.cc:260] T 80aa570e2c154172ad6da29d268bede7 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.057379 14809 ts_tablet_manager.cc:1428] T 80aa570e2c154172ad6da29d268bede7 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:19.057605 14809 tablet_bootstrap.cc:492] T 5737f10aa50e4059a814585019346872 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.058737 14809 tablet_bootstrap.cc:654] T 5737f10aa50e4059a814585019346872 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.061674 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6bb3ca6745554454a044c329e6b7e8d3" } modify_external_catalogs: true
I20250115 03:06:19.061838 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6bb3ca6745554454a044c329e6b7e8d3" } modify_external_catalogs: true
I20250115 03:06:19.065935 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet c84127dd93bc47ada551f93295ead47b from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:19.066668 14820 ts_tablet_manager.cc:1905] T a1b8b5b7fe03496ea2639dbc9c920974 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.068715 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 75b7e033e3ad4e2381ed0c2919e55dd9 (DEFAULT_TABLE table=default.Table_8ec408f3ef9f4344824b35f643021751 [id=315a2b6aa9c743e0bc40e5e70b7a9cb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.068972 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75b7e033e3ad4e2381ed0c2919e55dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.069197 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 2c5be0e7f0214c408ecf49b471fac883 (DEFAULT_TABLE table=default.Table_1440f76edbb04d508d0304afb1dd436a [id=b0fef8fcc75346bf847837e8e5963519]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.069414 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c5be0e7f0214c408ecf49b471fac883. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.070439 14519 tablet_service.cc:1467] Processing CreateTablet for tablet f73685801c0f4a2ca0324b50fc9f842d (DEFAULT_TABLE table=default.Table_0138c3ede76d425b9b0950b0e328c7a6 [id=35a0f55e72bd43428e170a5578f873f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.070652 14519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f73685801c0f4a2ca0324b50fc9f842d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.071043 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 713225c578dd4691baf65ae29100078b (DEFAULT_TABLE table=default.Table_75a17b2887884d69a0a674f69f2023fa [id=cb13748db1a94607b24692d5bc160f0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.071237 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 713225c578dd4691baf65ae29100078b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.071841 14820 ts_tablet_manager.cc:1918] T a1b8b5b7fe03496ea2639dbc9c920974 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.071950 14820 log.cc:1198] T a1b8b5b7fe03496ea2639dbc9c920974 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/a1b8b5b7fe03496ea2639dbc9c920974
I20250115 03:06:19.071979 14399 tablet_service.cc:1467] Processing CreateTablet for tablet a095c071aa6a400fb5b678627ddb118c (DEFAULT_TABLE table=default.Table_506e223627794e8bafd5f796b12b6049 [id=bd3be536db514d5fbba665e7b1990498]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.072189 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a095c071aa6a400fb5b678627ddb118c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.072278 14820 ts_tablet_manager.cc:1939] T a1b8b5b7fe03496ea2639dbc9c920974 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.073001 14524 tablet_service.cc:1467] Processing CreateTablet for tablet a095c071aa6a400fb5b678627ddb118c (DEFAULT_TABLE table=default.Table_506e223627794e8bafd5f796b12b6049 [id=bd3be536db514d5fbba665e7b1990498]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.073222 14524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a095c071aa6a400fb5b678627ddb118c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.073665 14820 tablet_replica.cc:331] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.073823 14820 raft_consensus.cc:2238] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.073948 14820 raft_consensus.cc:2267] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.074532 14820 ts_tablet_manager.cc:1905] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.075263 14523 tablet_service.cc:1467] Processing CreateTablet for tablet 2c5be0e7f0214c408ecf49b471fac883 (DEFAULT_TABLE table=default.Table_1440f76edbb04d508d0304afb1dd436a [id=b0fef8fcc75346bf847837e8e5963519]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.075484 14523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c5be0e7f0214c408ecf49b471fac883. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.077024 14809 tablet_bootstrap.cc:492] T 5737f10aa50e4059a814585019346872 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.077093 14809 ts_tablet_manager.cc:1397] T 5737f10aa50e4059a814585019346872 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20250115 03:06:19.077375 14820 ts_tablet_manager.cc:1918] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.077487 14820 log.cc:1198] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/858bbbb62fc249d5a915697c12362d0f
I20250115 03:06:19.077504 14809 raft_consensus.cc:357] T 5737f10aa50e4059a814585019346872 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.077601 14809 raft_consensus.cc:383] T 5737f10aa50e4059a814585019346872 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.077636 14809 raft_consensus.cc:738] T 5737f10aa50e4059a814585019346872 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.077780 14820 ts_tablet_manager.cc:1939] T 858bbbb62fc249d5a915697c12362d0f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.077742 14809 consensus_queue.cc:260] T 5737f10aa50e4059a814585019346872 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.077895 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 713225c578dd4691baf65ae29100078b (DEFAULT_TABLE table=default.Table_75a17b2887884d69a0a674f69f2023fa [id=cb13748db1a94607b24692d5bc160f0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.078013 14809 ts_tablet_manager.cc:1428] T 5737f10aa50e4059a814585019346872 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.078076 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 713225c578dd4691baf65ae29100078b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.078186 14809 tablet_bootstrap.cc:492] T ca2093980c8e480e99ae0cd79606f318 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.079021 14809 tablet_bootstrap.cc:654] T ca2093980c8e480e99ae0cd79606f318 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.080652 14820 tablet_replica.cc:331] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.080821 14820 raft_consensus.cc:2238] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.081209 14820 raft_consensus.cc:2267] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.081566 14820 ts_tablet_manager.cc:1905] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.083318 14820 ts_tablet_manager.cc:1918] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:19.084661 14163 catalog_manager.cc:5526] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193). New cstate: current_term: 1 leader_uuid: "e8df7247ee9c43e0aa18216da2661f75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:19.085215 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet d895f75911b7456a88e7a5159db7d60b
I20250115 03:06:19.085898 14402 tablet_service.cc:1467] Processing CreateTablet for tablet f73685801c0f4a2ca0324b50fc9f842d (DEFAULT_TABLE table=default.Table_0138c3ede76d425b9b0950b0e328c7a6 [id=35a0f55e72bd43428e170a5578f873f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.086107 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f73685801c0f4a2ca0324b50fc9f842d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.086318 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 9a6b950eafe74180b39a4c0c586f9f86 (DEFAULT_TABLE table=default.Table_befdedf4ed1940979834e31e3dee138e [id=3171ed8a044e4456924289d3c7e167ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.086531 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a6b950eafe74180b39a4c0c586f9f86. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.087846 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet d895f75911b7456a88e7a5159db7d60b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.088541 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 96656c3217354b468df86d96d4802883 (DEFAULT_TABLE table=default.Table_e539b5abff2f433da0b195f3a6fb9765 [id=72c977709fee4dc8ae1e82cc3a3b608d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.088658 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet a1b8b5b7fe03496ea2639dbc9c920974 (table default.Table_2773b577ce554a45834f200dca2c2987 [id=8005446109b64adabae59e8612c35f58]) successfully deleted
I20250115 03:06:19.088747 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96656c3217354b468df86d96d4802883. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:19.088779 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet b8ec24d2937d44e79b839276bf615fa0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b8ec24d2937d44e79b839276bf615fa0 already in progress: creating tablet
W20250115 03:06:19.088873 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet fdaf7c090ef642b8b339f61a5d3fb06e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fdaf7c090ef642b8b339f61a5d3fb06e already in progress: creating tablet
I20250115 03:06:19.089097 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 858bbbb62fc249d5a915697c12362d0f (table default.Table_79d7aa1c665d4695a2a17501157b3807 [id=5cd92dd0835f4c9d88d917d8ff277bbb]) successfully deleted
I20250115 03:06:19.089424 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_87c206116f514479ab9de8cd4a05817f" } modify_external_catalogs: true
I20250115 03:06:19.089463 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_a92db72c6c4c4f8280f5329dc717930a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:19.089524 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_87c206116f514479ab9de8cd4a05817f" } modify_external_catalogs: true
I20250115 03:06:19.089643 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 88e39c2bbe7d4c098e09836447d49fe9 (DEFAULT_TABLE table=default.Table_4946942766de43b7b86e7c77ab3f850c [id=f5729d169da54db5957e2a44ad4e8a4a]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:19.089772 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a92db72c6c4c4f8280f5329dc717930a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.089833 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88e39c2bbe7d4c098e09836447d49fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.090904 14819 tablet_replica.cc:331] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.091014 14819 raft_consensus.cc:2238] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:19.091172 14819 raft_consensus.cc:2267] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.091720 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 8b53474ea40642cc83b3b6d217b4b2d7 (DEFAULT_TABLE table=default.Table_cbbf34723b2d4e2db235756e12af64b5 [id=2978742c22da4b2fa12cb7c8b754b43e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.091920 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b53474ea40642cc83b3b6d217b4b2d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.091900 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 4c459696ca2a448dad39c3e00773bc0b (DEFAULT_TABLE table=default.Table_0e3a624b9d9a460184f8686f35ec2a05 [id=8dfa0161acb54a91a0ce330fc1a8dc61]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.092096 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c459696ca2a448dad39c3e00773bc0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.093367 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet d6cf1a0b9c81413b8f72de0cafc76963
I20250115 03:06:19.093426 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 0e1ce43146ed4e4d82a7c134216e2872
I20250115 03:06:19.093632 14164 catalog_manager.cc:6596] ReplaceTablet: tablet d6cf1a0b9c81413b8f72de0cafc76963 deleted and replaced by tablet a9807cb8c17d4b98be44e6fd5dccc150
I20250115 03:06:19.093669 14167 catalog_manager.cc:6596] ReplaceTablet: tablet 0e1ce43146ed4e4d82a7c134216e2872 deleted and replaced by tablet 578506b0528442cf834a8e65a400c635
I20250115 03:06:19.093854 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet f72c9bf04a3942ceacc37fd407bff033
I20250115 03:06:19.094033 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet dcb99303cec64a889c3057554f90561e
I20250115 03:06:19.094231 14819 ts_tablet_manager.cc:1905] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.096231 14819 ts_tablet_manager.cc:1918] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:19.096302 14819 log.cc:1198] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/d895f75911b7456a88e7a5159db7d60b
I20250115 03:06:19.096522 14819 ts_tablet_manager.cc:1939] T d895f75911b7456a88e7a5159db7d60b P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.097615 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 4dedf130dea441a1a6969e2daaf3567f (DEFAULT_TABLE table=default.Table_e50402205a344952ac6dd5064f9db8b2 [id=548d9993600743d49dff53577e77f4d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.097823 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dedf130dea441a1a6969e2daaf3567f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.097952 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet f72c9bf04a3942ceacc37fd407bff033 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.098116 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet dcb99303cec64a889c3057554f90561e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.098265 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet d6cf1a0b9c81413b8f72de0cafc76963 with delete_type TABLET_DATA_DELETED (replaced by tablet a9807cb8c17d4b98be44e6fd5dccc150) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.099038 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b9c09b6315345039cdea9eeee3f181a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.099040 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet d895f75911b7456a88e7a5159db7d60b (table default.Table_6bb3ca6745554454a044c329e6b7e8d3 [id=4b6ce1381c024498a78d91e346f488f2]) successfully deleted
W20250115 03:06:19.099157 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet f72c9bf04a3942ceacc37fd407bff033 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f72c9bf04a3942ceacc37fd407bff033 already in progress: creating tablet
I20250115 03:06:19.099220 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet cc67fb818b8146629d92f9ac1944dba6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.099314 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 80aa570e2c154172ad6da29d268bede7 with delete_type TABLET_DATA_DELETED (replaced by tablet 815ef85ce5cf4ae0a56c5734837e03ba) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.099399 14819 tablet_replica.cc:331] T 3b9c09b6315345039cdea9eeee3f181a P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.099457 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 074fdc09e4e24078ad0bb588b32ff24f
I20250115 03:06:19.099536 14819 raft_consensus.cc:2238] T 3b9c09b6315345039cdea9eeee3f181a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.099808 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 678b4453675b42ab98832a20f1b0271b (DEFAULT_TABLE table=default.Table_4bde77af131940a099eabda16fa2ccb2 [id=4f952e57c86746f6bb1df72e13589832]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.100083 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 678b4453675b42ab98832a20f1b0271b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.100162 14402 tablet_service.cc:1467] Processing CreateTablet for tablet d65bf3745b4f4d92a346b4dd18652432 (DEFAULT_TABLE table=default.Table_72e82b3e776e4d398fa5d8240afde936 [id=17ae925a7c0942d59058b2fd619ccb45]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.100279 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1ce43146ed4e4d82a7c134216e2872 with delete_type TABLET_DATA_DELETED (replaced by tablet 578506b0528442cf834a8e65a400c635) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.100368 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 2a9144a303eb4db08840b2f2749c52d6 (DEFAULT_TABLE table=default.Table_1a09d6243c154136aaa074ea2bfd7f6d [id=33b9ac162d2448909f6bd9af742d58f3]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:19.099520 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet dcb99303cec64a889c3057554f90561e because the tablet was not found. No further retry: Not found: Tablet not found: dcb99303cec64a889c3057554f90561e
W20250115 03:06:19.101752 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet d6cf1a0b9c81413b8f72de0cafc76963 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d6cf1a0b9c81413b8f72de0cafc76963 already in progress: creating tablet
I20250115 03:06:19.102339 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 074fdc09e4e24078ad0bb588b32ff24f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.102442 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d65bf3745b4f4d92a346b4dd18652432. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.046535 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5655935aabd48a19d0ef03739e089a4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:19.066233 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 75b7e033e3ad4e2381ed0c2919e55dd9 (DEFAULT_TABLE table=default.Table_8ec408f3ef9f4344824b35f643021751 [id=315a2b6aa9c743e0bc40e5e70b7a9cb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.104693 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a9144a303eb4db08840b2f2749c52d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.104872 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75b7e033e3ad4e2381ed0c2919e55dd9. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:19.105269 14358 leader_election.cc:343] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.105329 14358 leader_election.cc:304] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:19.105496 15390 raft_consensus.cc:2743] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:19.099630 14819 raft_consensus.cc:2267] T 3b9c09b6315345039cdea9eeee3f181a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.105984 14819 ts_tablet_manager.cc:1905] T 3b9c09b6315345039cdea9eeee3f181a P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.074244 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 713225c578dd4691baf65ae29100078b (DEFAULT_TABLE table=default.Table_75a17b2887884d69a0a674f69f2023fa [id=cb13748db1a94607b24692d5bc160f0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.107064 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 713225c578dd4691baf65ae29100078b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.107667 14819 ts_tablet_manager.cc:1918] T 3b9c09b6315345039cdea9eeee3f181a P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.107730 14819 log.cc:1198] T 3b9c09b6315345039cdea9eeee3f181a P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/3b9c09b6315345039cdea9eeee3f181a
I20250115 03:06:19.107904 14819 ts_tablet_manager.cc:1939] T 3b9c09b6315345039cdea9eeee3f181a P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.108512 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 3b9c09b6315345039cdea9eeee3f181a (table default.Table_561cca4d810b4e73934fcee99a8994c7 [id=270d1081a9a04f09b76df0f45cfbbff6]) successfully deleted
I20250115 03:06:19.108647 14819 tablet_replica.cc:331] T cc67fb818b8146629d92f9ac1944dba6 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.108707 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 31a04af0754044659a81561d9eddcfa7 (DEFAULT_TABLE table=default.Table_a222fccf6d3744cab2d0491ce93620be [id=9aa37ebd3bc54640b822106c489038fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.108760 14819 raft_consensus.cc:2238] T cc67fb818b8146629d92f9ac1944dba6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.108834 14819 raft_consensus.cc:2267] T cc67fb818b8146629d92f9ac1944dba6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.074469 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 2c5be0e7f0214c408ecf49b471fac883 (DEFAULT_TABLE table=default.Table_1440f76edbb04d508d0304afb1dd436a [id=b0fef8fcc75346bf847837e8e5963519]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.108888 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31a04af0754044659a81561d9eddcfa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.109125 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c5be0e7f0214c408ecf49b471fac883. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.110757 14820 log.cc:1198] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/9adb11e56c394461a91c924dc1eb404b
I20250115 03:06:19.110780 14819 ts_tablet_manager.cc:1905] T cc67fb818b8146629d92f9ac1944dba6 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.074700 14666 tablet_service.cc:1467] Processing CreateTablet for tablet a095c071aa6a400fb5b678627ddb118c (DEFAULT_TABLE table=default.Table_506e223627794e8bafd5f796b12b6049 [id=bd3be536db514d5fbba665e7b1990498]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.110991 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a095c071aa6a400fb5b678627ddb118c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.110994 14820 ts_tablet_manager.cc:1939] T 9adb11e56c394461a91c924dc1eb404b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.111501 14820 tablet_replica.cc:331] T a1c93bfd3da44f9f8a8d585f2d21f1be P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.111614 14820 raft_consensus.cc:2238] T a1c93bfd3da44f9f8a8d585f2d21f1be P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.111711 14820 raft_consensus.cc:2267] T a1c93bfd3da44f9f8a8d585f2d21f1be P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.112022 14820 ts_tablet_manager.cc:1905] T a1c93bfd3da44f9f8a8d585f2d21f1be P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.112378 14819 ts_tablet_manager.cc:1918] T cc67fb818b8146629d92f9ac1944dba6 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.112437 14819 log.cc:1198] T cc67fb818b8146629d92f9ac1944dba6 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/cc67fb818b8146629d92f9ac1944dba6
I20250115 03:06:19.112607 14819 ts_tablet_manager.cc:1939] T cc67fb818b8146629d92f9ac1944dba6 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.074903 14663 tablet_service.cc:1467] Processing CreateTablet for tablet f73685801c0f4a2ca0324b50fc9f842d (DEFAULT_TABLE table=default.Table_0138c3ede76d425b9b0950b0e328c7a6 [id=35a0f55e72bd43428e170a5578f873f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.112948 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f73685801c0f4a2ca0324b50fc9f842d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.086815 14667 tablet_service.cc:1467] Processing CreateTablet for tablet 8b53474ea40642cc83b3b6d217b4b2d7 (DEFAULT_TABLE table=default.Table_cbbf34723b2d4e2db235756e12af64b5 [id=2978742c22da4b2fa12cb7c8b754b43e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.114841 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b53474ea40642cc83b3b6d217b4b2d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.114856 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 99c6a9baf0f84489b6f38b4616ee1c15 (DEFAULT_TABLE table=default.Table_05584d1988f6439180926449a68a7198 [id=b67cf3fc6c134817a44199430100250d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.115087 14396 tablet_service.cc:1467] Processing CreateTablet for tablet e356bbb02b9649b9b30a3cbadcf59a28 (DEFAULT_TABLE table=default.Table_91812981cd64473f84541f170fb473bc [id=0329646c42e44a68a8ec46ed9338cf26]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.115236 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e356bbb02b9649b9b30a3cbadcf59a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.087035 14661 tablet_service.cc:1467] Processing CreateTablet for tablet 2a9144a303eb4db08840b2f2749c52d6 (DEFAULT_TABLE table=default.Table_1a09d6243c154136aaa074ea2bfd7f6d [id=33b9ac162d2448909f6bd9af742d58f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.116699 14661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a9144a303eb4db08840b2f2749c52d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.117030 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99c6a9baf0f84489b6f38b4616ee1c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.118139 14819 tablet_replica.cc:331] T 80aa570e2c154172ad6da29d268bede7 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.118240 14819 raft_consensus.cc:2238] T 80aa570e2c154172ad6da29d268bede7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.118288 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet cc67fb818b8146629d92f9ac1944dba6 (table default.Table_3fab7f59cc324dcf811b852f62ba1f9e [id=0a2e80b35ce148ff85d040843c7a1a2d]) successfully deleted
I20250115 03:06:19.118316 14819 raft_consensus.cc:2267] T 80aa570e2c154172ad6da29d268bede7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.088078 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 88e39c2bbe7d4c098e09836447d49fe9 (DEFAULT_TABLE table=default.Table_4946942766de43b7b86e7c77ab3f850c [id=f5729d169da54db5957e2a44ad4e8a4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.118595 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88e39c2bbe7d4c098e09836447d49fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.118649 14819 ts_tablet_manager.cc:1905] T 80aa570e2c154172ad6da29d268bede7 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.119863 14820 ts_tablet_manager.cc:1918] T a1c93bfd3da44f9f8a8d585f2d21f1be P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:19.119917 14820 log.cc:1198] T a1c93bfd3da44f9f8a8d585f2d21f1be P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/a1c93bfd3da44f9f8a8d585f2d21f1be
I20250115 03:06:19.120131 14820 ts_tablet_manager.cc:1939] T a1c93bfd3da44f9f8a8d585f2d21f1be P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.088284 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 4c459696ca2a448dad39c3e00773bc0b (DEFAULT_TABLE table=default.Table_0e3a624b9d9a460184f8686f35ec2a05 [id=8dfa0161acb54a91a0ce330fc1a8dc61]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.120393 14819 ts_tablet_manager.cc:1918] T 80aa570e2c154172ad6da29d268bede7 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.120451 14819 log.cc:1198] T 80aa570e2c154172ad6da29d268bede7 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/80aa570e2c154172ad6da29d268bede7
I20250115 03:06:19.120514 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c459696ca2a448dad39c3e00773bc0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.120644 14819 ts_tablet_manager.cc:1939] T 80aa570e2c154172ad6da29d268bede7 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.121253 14819 tablet_replica.cc:331] T 074fdc09e4e24078ad0bb588b32ff24f P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.121294 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 80aa570e2c154172ad6da29d268bede7 (table default.Table_24cbe6b962b248a9b340506b63c41b20 [id=9866275e65ab4dfcbfee6bf209c3654e]) successfully deleted
I20250115 03:06:19.121359 14819 raft_consensus.cc:2238] T 074fdc09e4e24078ad0bb588b32ff24f P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.121444 14819 raft_consensus.cc:2267] T 074fdc09e4e24078ad0bb588b32ff24f P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:19.121444 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 0e1ce43146ed4e4d82a7c134216e2872 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e1ce43146ed4e4d82a7c134216e2872 already in progress: creating tablet
I20250115 03:06:19.088490 14671 tablet_service.cc:1467] Processing CreateTablet for tablet 96656c3217354b468df86d96d4802883 (DEFAULT_TABLE table=default.Table_e539b5abff2f433da0b195f3a6fb9765 [id=72c977709fee4dc8ae1e82cc3a3b608d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.122366 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96656c3217354b468df86d96d4802883. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.088598 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet d895f75911b7456a88e7a5159db7d60b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.090777 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 9a6b950eafe74180b39a4c0c586f9f86 (DEFAULT_TABLE table=default.Table_befdedf4ed1940979834e31e3dee138e [id=3171ed8a044e4456924289d3c7e167ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.124176 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a6b950eafe74180b39a4c0c586f9f86. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.099915 14659 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1ce43146ed4e4d82a7c134216e2872 with delete_type TABLET_DATA_DELETED (replaced by tablet 578506b0528442cf834a8e65a400c635) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.099982 14670 tablet_service.cc:1514] Processing DeleteTablet for tablet d6cf1a0b9c81413b8f72de0cafc76963 with delete_type TABLET_DATA_DELETED (replaced by tablet a9807cb8c17d4b98be44e6fd5dccc150) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.100167 14658 tablet_service.cc:1514] Processing DeleteTablet for tablet f72c9bf04a3942ceacc37fd407bff033 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.100518 14657 tablet_service.cc:1467] Processing CreateTablet for tablet 4dedf130dea441a1a6969e2daaf3567f (DEFAULT_TABLE table=default.Table_e50402205a344952ac6dd5064f9db8b2 [id=548d9993600743d49dff53577e77f4d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.125165 14819 ts_tablet_manager.cc:1905] T 074fdc09e4e24078ad0bb588b32ff24f P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.125233 14657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dedf130dea441a1a6969e2daaf3567f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.090648 14522 tablet_service.cc:1467] Processing CreateTablet for tablet 96656c3217354b468df86d96d4802883 (DEFAULT_TABLE table=default.Table_e539b5abff2f433da0b195f3a6fb9765 [id=72c977709fee4dc8ae1e82cc3a3b608d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.126335 14522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96656c3217354b468df86d96d4802883. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.100780 14656 tablet_service.cc:1514] Processing DeleteTablet for tablet dcb99303cec64a889c3057554f90561e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.101123 14655 tablet_service.cc:1467] Processing CreateTablet for tablet d65bf3745b4f4d92a346b4dd18652432 (DEFAULT_TABLE table=default.Table_72e82b3e776e4d398fa5d8240afde936 [id=17ae925a7c0942d59058b2fd619ccb45]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.127832 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d65bf3745b4f4d92a346b4dd18652432. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.128115 14819 ts_tablet_manager.cc:1918] T 074fdc09e4e24078ad0bb588b32ff24f P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:19.128229 14819 log.cc:1198] T 074fdc09e4e24078ad0bb588b32ff24f P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/074fdc09e4e24078ad0bb588b32ff24f
I20250115 03:06:19.090946 14520 tablet_service.cc:1467] Processing CreateTablet for tablet 2a9144a303eb4db08840b2f2749c52d6 (DEFAULT_TABLE table=default.Table_1a09d6243c154136aaa074ea2bfd7f6d [id=33b9ac162d2448909f6bd9af742d58f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.128574 14819 ts_tablet_manager.cc:1939] T 074fdc09e4e24078ad0bb588b32ff24f P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.128638 14520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a9144a303eb4db08840b2f2749c52d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.101481 14654 tablet_service.cc:1467] Processing CreateTablet for tablet 678b4453675b42ab98832a20f1b0271b (DEFAULT_TABLE table=default.Table_4bde77af131940a099eabda16fa2ccb2 [id=4f952e57c86746f6bb1df72e13589832]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.129742 14654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 678b4453675b42ab98832a20f1b0271b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.091137 14525 tablet_service.cc:1514] Processing DeleteTablet for tablet d895f75911b7456a88e7a5159db7d60b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.131453 15390 raft_consensus.cc:491] T 5737f10aa50e4059a814585019346872 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.091358 14523 tablet_service.cc:1467] Processing CreateTablet for tablet 4c459696ca2a448dad39c3e00773bc0b (DEFAULT_TABLE table=default.Table_0e3a624b9d9a460184f8686f35ec2a05 [id=8dfa0161acb54a91a0ce330fc1a8dc61]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.131532 15390 raft_consensus.cc:513] T 5737f10aa50e4059a814585019346872 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.131698 14523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c459696ca2a448dad39c3e00773bc0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.131875 15390 leader_election.cc:290] T 5737f10aa50e4059a814585019346872 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:19.102133 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 074fdc09e4e24078ad0bb588b32ff24f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.133437 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5737f10aa50e4059a814585019346872" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:19.133776 14358 leader_election.cc:343] T 5737f10aa50e4059a814585019346872 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.133932 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5737f10aa50e4059a814585019346872" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:19.134143 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 074fdc09e4e24078ad0bb588b32ff24f (table default.Table_87c206116f514479ab9de8cd4a05817f [id=eaddac40b10648b1933b121cc70d68e8]) successfully deleted
W20250115 03:06:19.134248 14356 leader_election.cc:343] T 5737f10aa50e4059a814585019346872 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.134356 14356 leader_election.cc:304] T 5737f10aa50e4059a814585019346872 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:19.134527 15390 raft_consensus.cc:2743] T 5737f10aa50e4059a814585019346872 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:19.091576 14519 tablet_service.cc:1467] Processing CreateTablet for tablet 9a6b950eafe74180b39a4c0c586f9f86 (DEFAULT_TABLE table=default.Table_befdedf4ed1940979834e31e3dee138e [id=3171ed8a044e4456924289d3c7e167ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.134649 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 0675b412073741128f11b84c0e0582f1 (DEFAULT_TABLE table=default.Table_166b9ddd229c4e079ebf7b12ae5e54dc [id=2984836024444aa9b308d510b31328ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.134718 14519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a6b950eafe74180b39a4c0c586f9f86. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.134986 14820 tablet_replica.cc:331] T a2fc04950cfe4c608a4712afd257ca76 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.135953 14820 raft_consensus.cc:2238] T a2fc04950cfe4c608a4712afd257ca76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.136471 14820 raft_consensus.cc:2267] T a2fc04950cfe4c608a4712afd257ca76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.091754 14524 tablet_service.cc:1467] Processing CreateTablet for tablet 8b53474ea40642cc83b3b6d217b4b2d7 (DEFAULT_TABLE table=default.Table_cbbf34723b2d4e2db235756e12af64b5 [id=2978742c22da4b2fa12cb7c8b754b43e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.136777 14524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b53474ea40642cc83b3b6d217b4b2d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.136858 14820 ts_tablet_manager.cc:1905] T a2fc04950cfe4c608a4712afd257ca76 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.103951 14526 tablet_service.cc:1514] Processing DeleteTablet for tablet d6cf1a0b9c81413b8f72de0cafc76963 with delete_type TABLET_DATA_DELETED (replaced by tablet a9807cb8c17d4b98be44e6fd5dccc150) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.104027 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1ce43146ed4e4d82a7c134216e2872 with delete_type TABLET_DATA_DELETED (replaced by tablet 578506b0528442cf834a8e65a400c635) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.104070 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet f72c9bf04a3942ceacc37fd407bff033 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.104338 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 4dedf130dea441a1a6969e2daaf3567f (DEFAULT_TABLE table=default.Table_e50402205a344952ac6dd5064f9db8b2 [id=548d9993600743d49dff53577e77f4d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.139146 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dedf130dea441a1a6969e2daaf3567f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.139250 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 31a04af0754044659a81561d9eddcfa7 (DEFAULT_TABLE table=default.Table_a222fccf6d3744cab2d0491ce93620be [id=9aa37ebd3bc54640b822106c489038fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.104485 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 074fdc09e4e24078ad0bb588b32ff24f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.139453 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31a04af0754044659a81561d9eddcfa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.104523 14533 tablet_service.cc:1467] Processing CreateTablet for tablet d65bf3745b4f4d92a346b4dd18652432 (DEFAULT_TABLE table=default.Table_72e82b3e776e4d398fa5d8240afde936 [id=17ae925a7c0942d59058b2fd619ccb45]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.104626 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet dcb99303cec64a889c3057554f90561e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.104821 14532 tablet_service.cc:1467] Processing CreateTablet for tablet 678b4453675b42ab98832a20f1b0271b (DEFAULT_TABLE table=default.Table_4bde77af131940a099eabda16fa2ccb2 [id=4f952e57c86746f6bb1df72e13589832]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.105043 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 31a04af0754044659a81561d9eddcfa7 (DEFAULT_TABLE table=default.Table_a222fccf6d3744cab2d0491ce93620be [id=9aa37ebd3bc54640b822106c489038fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.091922 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 88e39c2bbe7d4c098e09836447d49fe9 (DEFAULT_TABLE table=default.Table_4946942766de43b7b86e7c77ab3f850c [id=f5729d169da54db5957e2a44ad4e8a4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.141300 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 99c6a9baf0f84489b6f38b4616ee1c15 (DEFAULT_TABLE table=default.Table_05584d1988f6439180926449a68a7198 [id=b67cf3fc6c134817a44199430100250d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.141498 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99c6a9baf0f84489b6f38b4616ee1c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.143388 14654 tablet_service.cc:1467] Processing CreateTablet for tablet e356bbb02b9649b9b30a3cbadcf59a28 (DEFAULT_TABLE table=default.Table_91812981cd64473f84541f170fb473bc [id=0329646c42e44a68a8ec46ed9338cf26]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.143493 14518 tablet_service.cc:1467] Processing CreateTablet for tablet 75b7e033e3ad4e2381ed0c2919e55dd9 (DEFAULT_TABLE table=default.Table_8ec408f3ef9f4344824b35f643021751 [id=315a2b6aa9c743e0bc40e5e70b7a9cb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.143596 14654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e356bbb02b9649b9b30a3cbadcf59a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.143424 14823 raft_consensus.cc:2267] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.143956 14823 ts_tablet_manager.cc:1905] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.144624 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0675b412073741128f11b84c0e0582f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.145202 14655 tablet_service.cc:1467] Processing CreateTablet for tablet 0675b412073741128f11b84c0e0582f1 (DEFAULT_TABLE table=default.Table_166b9ddd229c4e079ebf7b12ae5e54dc [id=2984836024444aa9b308d510b31328ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.145401 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0675b412073741128f11b84c0e0582f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.147764 14655 tablet_service.cc:1467] Processing CreateTablet for tablet af712cc03750455087ecd381cc97d464 (DEFAULT_TABLE table=default.Table_ab37ce65c0ae4e06b3e72a0e843d8f21 [id=bdf35c4fd7554c2a85d02e37b7c5c2cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.147763 14396 tablet_service.cc:1467] Processing CreateTablet for tablet cd6f136b92ba4fc7b956d52aa0c20f53 (DEFAULT_TABLE table=default.Table_f2ff78f7081d4b56a30785f4d6ae8e71 [id=7a4016b430b049578159e70d6e2e237c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.147969 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd6f136b92ba4fc7b956d52aa0c20f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.147985 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af712cc03750455087ecd381cc97d464. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.149863 14396 tablet_service.cc:1467] Processing CreateTablet for tablet dcb99303cec64a889c3057554f90561e (DEFAULT_TABLE table=default.Table_e9d0346fd3064c8d92a35b495ae3a3c8 [id=aa9b18bc2f4642b3b7e57ad0c36b53f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.149868 14653 tablet_service.cc:1467] Processing CreateTablet for tablet dcb99303cec64a889c3057554f90561e (DEFAULT_TABLE table=default.Table_e9d0346fd3064c8d92a35b495ae3a3c8 [id=aa9b18bc2f4642b3b7e57ad0c36b53f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.150074 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcb99303cec64a889c3057554f90561e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.150076 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 9adb11e56c394461a91c924dc1eb404b (table default.Table_605aff02064949c1b48d374a50c910b2 [id=d0fc70d8af9944099f8e7ee9a416399d]) successfully deleted
I20250115 03:06:19.150205 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcb99303cec64a889c3057554f90561e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.150326 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet a1c93bfd3da44f9f8a8d585f2d21f1be (table default.Table_40b1953a4a364eb1ae58e3387fc65a91 [id=64d20b1792044c5e8add735aaf75a6d9]) successfully deleted
I20250115 03:06:19.151633 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_fa2d8bed137445d7a9773e35a10912b2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:19.151803 14654 tablet_service.cc:1467] Processing CreateTablet for tablet cd6f136b92ba4fc7b956d52aa0c20f53 (DEFAULT_TABLE table=default.Table_f2ff78f7081d4b56a30785f4d6ae8e71 [id=7a4016b430b049578159e70d6e2e237c]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:19.151968 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fa2d8bed137445d7a9773e35a10912b2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.151990 14654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd6f136b92ba4fc7b956d52aa0c20f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.152266 14402 tablet_service.cc:1467] Processing CreateTablet for tablet af712cc03750455087ecd381cc97d464 (DEFAULT_TABLE table=default.Table_ab37ce65c0ae4e06b3e72a0e843d8f21 [id=bdf35c4fd7554c2a85d02e37b7c5c2cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.152483 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af712cc03750455087ecd381cc97d464. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.154227 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_7c5632f41fd641dabc346cc2c77f2988"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.154523 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7c5632f41fd641dabc346cc2c77f2988 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.155865 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8f7e876bb60444429c30944284f9b10d" } new_table_name: "default.Table_b6002fb5cc1248bba2eb7abbd5a5d407" modify_external_catalogs: true
I20250115 03:06:19.155972 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_aed56e66118644fa9210464c3953ebdf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:19.156132 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_cf9c2dc4f74c4c7497a9b49023b79010"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.156221 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aed56e66118644fa9210464c3953ebdf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:19.156405 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cf9c2dc4f74c4c7497a9b49023b79010 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.157929 14823 ts_tablet_manager.cc:1918] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:19.157997 14823 log.cc:1198] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/d28ed8538c4a44d18279371b826c22a5
I20250115 03:06:19.158224 14823 ts_tablet_manager.cc:1939] T d28ed8538c4a44d18279371b826c22a5 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:19.159497 14823 tablet_replica.cc:331] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:19.159615 14823 raft_consensus.cc:2238] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.159667 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31a04af0754044659a81561d9eddcfa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.159719 14823 raft_consensus.cc:2267] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.160007 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d65bf3745b4f4d92a346b4dd18652432. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.160329 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 678b4453675b42ab98832a20f1b0271b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.161435 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88e39c2bbe7d4c098e09836447d49fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.161792 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75b7e033e3ad4e2381ed0c2919e55dd9. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:19.162236 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8efa998709034fe9bb37c7c608c99e76 consensus state has no leader
I20250115 03:06:19.165110 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet d28ed8538c4a44d18279371b826c22a5 (table default.Table_fe0a1f3ae7a04225a5a482c33a15c579 [id=c99c3573d2394eada9631427ac38322d]) successfully deleted
I20250115 03:06:19.165206 14654 tablet_service.cc:1467] Processing CreateTablet for tablet 0fa5ce72e20b4e0092b39c64160f0995 (DEFAULT_TABLE table=default.Table_6b1f81ee12e04acda4bcf7d7f289a67a [id=8f811bda53fc4d76a88a3dcf69aa3dc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.165400 14654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fa5ce72e20b4e0092b39c64160f0995. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.166051 14527 tablet_service.cc:1467] Processing CreateTablet for tablet e356bbb02b9649b9b30a3cbadcf59a28 (DEFAULT_TABLE table=default.Table_91812981cd64473f84541f170fb473bc [id=0329646c42e44a68a8ec46ed9338cf26]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.166090 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0138c3ede76d425b9b0950b0e328c7a6" } new_table_name: "default.Table_2af7611e98f34aa8ab6760cc834a2fdd" modify_external_catalogs: true
I20250115 03:06:19.166455 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 0675b412073741128f11b84c0e0582f1 (DEFAULT_TABLE table=default.Table_166b9ddd229c4e079ebf7b12ae5e54dc [id=2984836024444aa9b308d510b31328ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.166793 14534 tablet_service.cc:1467] Processing CreateTablet for tablet af712cc03750455087ecd381cc97d464 (DEFAULT_TABLE table=default.Table_ab37ce65c0ae4e06b3e72a0e843d8f21 [id=bdf35c4fd7554c2a85d02e37b7c5c2cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.166886 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_befdedf4ed1940979834e31e3dee138e" } new_table_name: "default.Table_36f5da5c6352445b9fb9e53d74ef4019" modify_external_catalogs: true
I20250115 03:06:19.167121 14536 tablet_service.cc:1467] Processing CreateTablet for tablet cd6f136b92ba4fc7b956d52aa0c20f53 (DEFAULT_TABLE table=default.Table_f2ff78f7081d4b56a30785f4d6ae8e71 [id=7a4016b430b049578159e70d6e2e237c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.167338 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd6f136b92ba4fc7b956d52aa0c20f53. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.167874 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_bb5db03314f149ad8d2a9efb8743db6f" } new_table_name: "default.Table_3acc4838fabe430994487140caafa10d" modify_external_catalogs: true
I20250115 03:06:19.167984 14820 ts_tablet_manager.cc:1918] T a2fc04950cfe4c608a4712afd257ca76 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:19.168133 14820 log.cc:1198] T a2fc04950cfe4c608a4712afd257ca76 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/a2fc04950cfe4c608a4712afd257ca76
I20250115 03:06:19.168349 14820 ts_tablet_manager.cc:1939] T a2fc04950cfe4c608a4712afd257ca76 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.168794 14820 tablet_replica.cc:331] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.168892 14820 raft_consensus.cc:2238] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.168979 14820 raft_consensus.cc:2267] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.169131 14526 tablet_service.cc:1467] Processing CreateTablet for tablet dcb99303cec64a889c3057554f90561e (DEFAULT_TABLE table=default.Table_e9d0346fd3064c8d92a35b495ae3a3c8 [id=aa9b18bc2f4642b3b7e57ad0c36b53f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.169250 14820 ts_tablet_manager.cc:1905] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.170214 14526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcb99303cec64a889c3057554f90561e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.171331 14820 ts_tablet_manager.cc:1918] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.171396 14820 log.cc:1198] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/6a59a9be8fa94ec4bdb42c723ea0ca17
I20250115 03:06:19.171574 14820 ts_tablet_manager.cc:1939] T 6a59a9be8fa94ec4bdb42c723ea0ca17 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.171967 14823 ts_tablet_manager.cc:1905] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.173914 14823 ts_tablet_manager.cc:1918] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:19.173982 14823 log.cc:1198] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/d895f75911b7456a88e7a5159db7d60b
I20250115 03:06:19.174196 14823 ts_tablet_manager.cc:1939] T d895f75911b7456a88e7a5159db7d60b P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:19.174958 14823 tablet_replica.cc:331] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:19.175065 14823 raft_consensus.cc:2238] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:19.175207 14823 raft_consensus.cc:2267] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.175745 14812 tablet_bootstrap.cc:492] T cff95c69fb9045cf9ec0ebb128c9ef6b P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:19.175810 14812 ts_tablet_manager.cc:1397] T cff95c69fb9045cf9ec0ebb128c9ef6b P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.224s user 0.000s sys 0.002s
I20250115 03:06:19.176399 14812 raft_consensus.cc:357] T cff95c69fb9045cf9ec0ebb128c9ef6b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.176491 14812 raft_consensus.cc:383] T cff95c69fb9045cf9ec0ebb128c9ef6b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.176524 14812 raft_consensus.cc:738] T cff95c69fb9045cf9ec0ebb128c9ef6b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.176617 14812 consensus_queue.cc:260] T cff95c69fb9045cf9ec0ebb128c9ef6b P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.176885 14812 ts_tablet_manager.cc:1428] T cff95c69fb9045cf9ec0ebb128c9ef6b P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:19.176967 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet d895f75911b7456a88e7a5159db7d60b (table default.Table_6bb3ca6745554454a044c329e6b7e8d3 [id=4b6ce1381c024498a78d91e346f488f2]) successfully deleted
I20250115 03:06:19.177094 14812 tablet_bootstrap.cc:492] T 1f2dc6928e2941ef9b5b5a7891376500 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:19.177096 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 0e1ce43146ed4e4d82a7c134216e2872 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e1ce43146ed4e4d82a7c134216e2872 already in progress: creating tablet
W20250115 03:06:19.177196 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet d6cf1a0b9c81413b8f72de0cafc76963 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d6cf1a0b9c81413b8f72de0cafc76963 already in progress: creating tablet
W20250115 03:06:19.177291 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet f72c9bf04a3942ceacc37fd407bff033 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f72c9bf04a3942ceacc37fd407bff033 already in progress: creating tablet
W20250115 03:06:19.177393 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet dcb99303cec64a889c3057554f90561e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dcb99303cec64a889c3057554f90561e already in progress: creating tablet
I20250115 03:06:19.177961 14812 tablet_bootstrap.cc:654] T 1f2dc6928e2941ef9b5b5a7891376500 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.178256 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet a2fc04950cfe4c608a4712afd257ca76 (table default.Table_6b1f81ee12e04acda4bcf7d7f289a67a [id=8f811bda53fc4d76a88a3dcf69aa3dc6]) successfully deleted
I20250115 03:06:19.182543 14654 tablet_service.cc:1514] Processing DeleteTablet for tablet cff95c69fb9045cf9ec0ebb128c9ef6b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.183339 14811 tablet_bootstrap.cc:492] T 8efd181c42eb42628a9f36fad4fa8d64 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.183414 14811 ts_tablet_manager.cc:1397] T 8efd181c42eb42628a9f36fad4fa8d64 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.238s user 0.001s sys 0.002s
I20250115 03:06:19.183832 14811 raft_consensus.cc:357] T 8efd181c42eb42628a9f36fad4fa8d64 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.183924 14811 raft_consensus.cc:383] T 8efd181c42eb42628a9f36fad4fa8d64 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.183955 14811 raft_consensus.cc:738] T 8efd181c42eb42628a9f36fad4fa8d64 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.184058 14811 consensus_queue.cc:260] T 8efd181c42eb42628a9f36fad4fa8d64 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.184325 14811 ts_tablet_manager.cc:1428] T 8efd181c42eb42628a9f36fad4fa8d64 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:19.184492 14811 tablet_bootstrap.cc:492] T 58130f2b27b745d08c0002ac96af6d72 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.185233 14811 tablet_bootstrap.cc:654] T 58130f2b27b745d08c0002ac96af6d72 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.185899 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 0fa5ce72e20b4e0092b39c64160f0995 (DEFAULT_TABLE table=default.Table_6b1f81ee12e04acda4bcf7d7f289a67a [id=8f811bda53fc4d76a88a3dcf69aa3dc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.186151 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fa5ce72e20b4e0092b39c64160f0995. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.188504 14809 tablet_bootstrap.cc:492] T ca2093980c8e480e99ae0cd79606f318 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.188582 14809 ts_tablet_manager.cc:1397] T ca2093980c8e480e99ae0cd79606f318 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.110s user 0.002s sys 0.000s
W20250115 03:06:19.189026 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9a6b950eafe74180b39a4c0c586f9f86 consensus state has no leader
I20250115 03:06:19.189172 14809 raft_consensus.cc:357] T ca2093980c8e480e99ae0cd79606f318 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.189270 14809 raft_consensus.cc:383] T ca2093980c8e480e99ae0cd79606f318 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.189303 14809 raft_consensus.cc:738] T ca2093980c8e480e99ae0cd79606f318 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.189404 14809 consensus_queue.cc:260] T ca2093980c8e480e99ae0cd79606f318 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:19.189513 14163 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f73685801c0f4a2ca0324b50fc9f842d consensus state has no leader
I20250115 03:06:19.189657 14809 ts_tablet_manager.cc:1428] T ca2093980c8e480e99ae0cd79606f318 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:19.189816 14809 tablet_bootstrap.cc:492] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.190832 14809 tablet_bootstrap.cc:654] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.192845 14853 raft_consensus.cc:491] T 5780da67ce984297ba33d3c6901c9e3a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.192963 14853 raft_consensus.cc:513] T 5780da67ce984297ba33d3c6901c9e3a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.193245 14853 leader_election.cc:290] T 5780da67ce984297ba33d3c6901c9e3a P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:19.193619 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5780da67ce984297ba33d3c6901c9e3a" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:19.194931 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e356bbb02b9649b9b30a3cbadcf59a28. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.195220 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af712cc03750455087ecd381cc97d464. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.195408 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5780da67ce984297ba33d3c6901c9e3a" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:19.195487 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0675b412073741128f11b84c0e0582f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.195947 14493 leader_election.cc:304] T 5780da67ce984297ba33d3c6901c9e3a P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:19.196121 14853 raft_consensus.cc:2743] T 5780da67ce984297ba33d3c6901c9e3a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:19.200625 14402 tablet_service.cc:1514] Processing DeleteTablet for tablet ca2093980c8e480e99ae0cd79606f318 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.200779 14819 tablet_replica.cc:331] T ca2093980c8e480e99ae0cd79606f318 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.200882 14819 raft_consensus.cc:2238] T ca2093980c8e480e99ae0cd79606f318 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.200975 14819 raft_consensus.cc:2267] T ca2093980c8e480e99ae0cd79606f318 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.201438 14812 tablet_bootstrap.cc:492] T 1f2dc6928e2941ef9b5b5a7891376500 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:19.201510 14812 ts_tablet_manager.cc:1397] T 1f2dc6928e2941ef9b5b5a7891376500 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20250115 03:06:19.201687 14820 tablet_replica.cc:331] T 93fa227c0de74a3fa86778641fdcda3d P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.201786 14820 raft_consensus.cc:2238] T 93fa227c0de74a3fa86778641fdcda3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.201862 14820 raft_consensus.cc:2267] T 93fa227c0de74a3fa86778641fdcda3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.201916 14812 raft_consensus.cc:357] T 1f2dc6928e2941ef9b5b5a7891376500 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.201995 14812 raft_consensus.cc:383] T 1f2dc6928e2941ef9b5b5a7891376500 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.202021 14812 raft_consensus.cc:738] T 1f2dc6928e2941ef9b5b5a7891376500 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.202122 14812 consensus_queue.cc:260] T 1f2dc6928e2941ef9b5b5a7891376500 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.202360 14812 ts_tablet_manager.cc:1428] T 1f2dc6928e2941ef9b5b5a7891376500 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.202508 14812 tablet_bootstrap.cc:492] T 3b9c09b6315345039cdea9eeee3f181a P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:19.203330 14812 tablet_bootstrap.cc:654] T 3b9c09b6315345039cdea9eeee3f181a P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.205338 14811 tablet_bootstrap.cc:492] T 58130f2b27b745d08c0002ac96af6d72 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.205415 14811 ts_tablet_manager.cc:1397] T 58130f2b27b745d08c0002ac96af6d72 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.000s
I20250115 03:06:19.205814 14811 raft_consensus.cc:357] T 58130f2b27b745d08c0002ac96af6d72 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.205899 14811 raft_consensus.cc:383] T 58130f2b27b745d08c0002ac96af6d72 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.205928 14811 raft_consensus.cc:738] T 58130f2b27b745d08c0002ac96af6d72 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.206029 14811 consensus_queue.cc:260] T 58130f2b27b745d08c0002ac96af6d72 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.206205 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 0fa5ce72e20b4e0092b39c64160f0995 (DEFAULT_TABLE table=default.Table_6b1f81ee12e04acda4bcf7d7f289a67a [id=8f811bda53fc4d76a88a3dcf69aa3dc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.206262 14811 ts_tablet_manager.cc:1428] T 58130f2b27b745d08c0002ac96af6d72 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.206413 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fa5ce72e20b4e0092b39c64160f0995. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.206418 14811 tablet_bootstrap.cc:492] T 6784e353df554bb2aa10b667c8b488da P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.207245 14811 tablet_bootstrap.cc:654] T 6784e353df554bb2aa10b667c8b488da P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.207701 14809 tablet_bootstrap.cc:492] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.207773 14809 ts_tablet_manager.cc:1397] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20250115 03:06:19.208175 14809 raft_consensus.cc:357] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.208289 14809 raft_consensus.cc:383] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.208320 14809 raft_consensus.cc:738] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.208416 14809 consensus_queue.cc:260] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.208662 14809 ts_tablet_manager.cc:1428] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.208827 14809 tablet_bootstrap.cc:492] T c251960c7c9a44429102a294f35f925c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.209326 14654 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f2dc6928e2941ef9b5b5a7891376500 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f9818d0493547358fd5654a01d4f456) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.209687 14809 tablet_bootstrap.cc:654] T c251960c7c9a44429102a294f35f925c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.210701 14552 raft_consensus.cc:3054] T 5780da67ce984297ba33d3c6901c9e3a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:19.212622 15426 tablet.cc:1721] T 5780da67ce984297ba33d3c6901c9e3a P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:19.217285 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 6a59a9be8fa94ec4bdb42c723ea0ca17 (table default.Table_68e894a7b2d7416a90c519daa7bf4fb9 [id=5b343e28259444528cbad21c3b048db5]) successfully deleted
W20250115 03:06:19.218354 14626 consensus_peers.cc:487] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20250115 03:06:19.218405 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 4102586f32d94a998cf2807434ee0ebf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4102586f32d94a998cf2807434ee0ebf already in progress: creating tablet
W20250115 03:06:19.218518 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 66b748fc853f499fafc1f476c8f79b8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66b748fc853f499fafc1f476c8f79b8b already in progress: creating tablet
I20250115 03:06:19.218664 16088 consensus_queue.cc:1035] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:19.218935 14626 consensus_peers.cc:487] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:19.219405 14819 ts_tablet_manager.cc:1905] T ca2093980c8e480e99ae0cd79606f318 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.221276 14402 tablet_service.cc:1467] Processing CreateTablet for tablet f2af9569a0254f32b4b6ceab5bfbfa74 (DEFAULT_TABLE table=default.Table_cf9c2dc4f74c4c7497a9b49023b79010 [id=6842ac1dc70e4a30a9e875cb0be7507b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.221462 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 55538831d8ea47eda40c36d42c19435f (DEFAULT_TABLE table=default.Table_fe0a1f3ae7a04225a5a482c33a15c579 [id=c99c3573d2394eada9631427ac38322d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.221505 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 578506b0528442cf834a8e65a400c635 (DEFAULT_TABLE table=default.Table_0c35f2414d284756b9c93ed61e75d451 [id=5775da4995774af39bfc049e48eaf2f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.221724 14393 tablet_service.cc:1467] Processing CreateTablet for tablet bf80a961cac644e7aa338c41bb430a1e (DEFAULT_TABLE table=default.Table_ea36537d443240e99ab0ff1719ab9939 [id=029616d69fd44b74aa35929fde1c3082]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.221985 14399 tablet_service.cc:1467] Processing CreateTablet for tablet f972872b022344418e9517131c27a22d (DEFAULT_TABLE table=default.Table_a92db72c6c4c4f8280f5329dc717930a [id=2d6fb92197894fa1b4d070defc329233]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.219789 14521 tablet_service.cc:1467] Processing CreateTablet for tablet f2af9569a0254f32b4b6ceab5bfbfa74 (DEFAULT_TABLE table=default.Table_cf9c2dc4f74c4c7497a9b49023b79010 [id=6842ac1dc70e4a30a9e875cb0be7507b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.222074 14653 tablet_service.cc:1467] Processing CreateTablet for tablet f972872b022344418e9517131c27a22d (DEFAULT_TABLE table=default.Table_a92db72c6c4c4f8280f5329dc717930a [id=2d6fb92197894fa1b4d070defc329233]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.222316 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f972872b022344418e9517131c27a22d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.223377 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2af9569a0254f32b4b6ceab5bfbfa74. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.224303 14655 tablet_service.cc:1467] Processing CreateTablet for tablet bf80a961cac644e7aa338c41bb430a1e (DEFAULT_TABLE table=default.Table_ea36537d443240e99ab0ff1719ab9939 [id=029616d69fd44b74aa35929fde1c3082]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.224505 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf80a961cac644e7aa338c41bb430a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.225667 14394 tablet_service.cc:1467] Processing CreateTablet for tablet e95269dd9a1c4d9a897f3c098628f2c0 (DEFAULT_TABLE table=default.Table_aed56e66118644fa9210464c3953ebdf [id=b57368b0727b4358b6510fae2d06d08e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.226018 14521 tablet_service.cc:1467] Processing CreateTablet for tablet f972872b022344418e9517131c27a22d (DEFAULT_TABLE table=default.Table_a92db72c6c4c4f8280f5329dc717930a [id=2d6fb92197894fa1b4d070defc329233]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.226377 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 55538831d8ea47eda40c36d42c19435f (DEFAULT_TABLE table=default.Table_fe0a1f3ae7a04225a5a482c33a15c579 [id=c99c3573d2394eada9631427ac38322d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.226573 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55538831d8ea47eda40c36d42c19435f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.226627 14534 tablet_service.cc:1467] Processing CreateTablet for tablet bf80a961cac644e7aa338c41bb430a1e (DEFAULT_TABLE table=default.Table_ea36537d443240e99ab0ff1719ab9939 [id=029616d69fd44b74aa35929fde1c3082]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.226830 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf80a961cac644e7aa338c41bb430a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.227064 14395 tablet_service.cc:1467] Processing CreateTablet for tablet a9807cb8c17d4b98be44e6fd5dccc150 (DEFAULT_TABLE table=default.Table_e51b0b04bb044b36be347afe90845b14 [id=17ea0c263011486f82895eb86b21fe78]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.227417 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 55538831d8ea47eda40c36d42c19435f (DEFAULT_TABLE table=default.Table_fe0a1f3ae7a04225a5a482c33a15c579 [id=c99c3573d2394eada9631427ac38322d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.227891 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 578506b0528442cf834a8e65a400c635 (DEFAULT_TABLE table=default.Table_0c35f2414d284756b9c93ed61e75d451 [id=5775da4995774af39bfc049e48eaf2f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.228585 14526 tablet_service.cc:1467] Processing CreateTablet for tablet 1f16c6d7c2ad4f4abe3ad599305ffa2e (DEFAULT_TABLE table=default.Table_debb6a0c409a4d8383c41a628fcb8450 [id=d4af093be8d14c39bc8565233c3cce56]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.228595 14656 tablet_service.cc:1467] Processing CreateTablet for tablet 578506b0528442cf834a8e65a400c635 (DEFAULT_TABLE table=default.Table_0c35f2414d284756b9c93ed61e75d451 [id=5775da4995774af39bfc049e48eaf2f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.228821 14656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 578506b0528442cf834a8e65a400c635. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.229775 14391 tablet_service.cc:1467] Processing CreateTablet for tablet c6d6cbc01b944e408dfe79a84674b78d (DEFAULT_TABLE table=default.Table_b3d1f8a0bba14291b9577e6201a2bb06 [id=bfaeaaf9694d4c1eb6a7247c4202a1c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.230692 14657 tablet_service.cc:1467] Processing CreateTablet for tablet 1f16c6d7c2ad4f4abe3ad599305ffa2e (DEFAULT_TABLE table=default.Table_debb6a0c409a4d8383c41a628fcb8450 [id=d4af093be8d14c39bc8565233c3cce56]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.230762 14392 tablet_service.cc:1467] Processing CreateTablet for tablet e5b483637c08401d92c1ab63cfe6f3c6 (DEFAULT_TABLE table=default.Table_0816f2fcac824630b5042becb8b0ed18 [id=df3d22f95df04db09d58e7c5f856d3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.230899 14657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f16c6d7c2ad4f4abe3ad599305ffa2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.232569 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_fe0a1f3ae7a04225a5a482c33a15c579" } new_table_name: "default.Table_b8b40ffaa9e04cf481788f0d2e8519e9" modify_external_catalogs: true
I20250115 03:06:19.232664 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet da95daa40c5b4615a0219f6d8e3770f6 with delete_type TABLET_DATA_DELETED (replaced by tablet 678b4453675b42ab98832a20f1b0271b) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.232893 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 1f16c6d7c2ad4f4abe3ad599305ffa2e (DEFAULT_TABLE table=default.Table_debb6a0c409a4d8383c41a628fcb8450 [id=d4af093be8d14c39bc8565233c3cce56]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.233196 14658 tablet_service.cc:1467] Processing CreateTablet for tablet e5b483637c08401d92c1ab63cfe6f3c6 (DEFAULT_TABLE table=default.Table_0816f2fcac824630b5042becb8b0ed18 [id=df3d22f95df04db09d58e7c5f856d3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.233409 14658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5b483637c08401d92c1ab63cfe6f3c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.233621 15390 raft_consensus.cc:491] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.233686 15390 raft_consensus.cc:513] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.234033 15390 leader_election.cc:290] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:19.235178 14670 tablet_service.cc:1467] Processing CreateTablet for tablet e63809cf809c453cb49411358050402b (DEFAULT_TABLE table=default.Table_e1ceae92543048f39acd7f81454b14f8 [id=a442ed735eab4be09b704ac69c7f4998]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.235390 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e63809cf809c453cb49411358050402b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.236049 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da95daa40c5b4615a0219f6d8e3770f6" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:19.236379 14358 leader_election.cc:343] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.237720 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 578506b0528442cf834a8e65a400c635. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.237886 14397 tablet_service.cc:1467] Processing CreateTablet for tablet e63809cf809c453cb49411358050402b (DEFAULT_TABLE table=default.Table_e1ceae92543048f39acd7f81454b14f8 [id=a442ed735eab4be09b704ac69c7f4998]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.238018 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55538831d8ea47eda40c36d42c19435f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.238178 14654 tablet_service.cc:1467] Processing CreateTablet for tablet f2af9569a0254f32b4b6ceab5bfbfa74 (DEFAULT_TABLE table=default.Table_cf9c2dc4f74c4c7497a9b49023b79010 [id=6842ac1dc70e4a30a9e875cb0be7507b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.238274 14526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f16c6d7c2ad4f4abe3ad599305ffa2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.238350 14654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2af9569a0254f32b4b6ceab5bfbfa74. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:19.241557 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 55538831d8ea47eda40c36d42c19435f consensus state has no leader
I20250115 03:06:19.245663 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da95daa40c5b4615a0219f6d8e3770f6" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:19.247386 14166 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5cb34cb2006f4716b87bef67f3a6dd8e (table default.Table_77b3fb817e8b4502af7fae7bcb0124a5 [id=ed28aa2aaacd4ba58dcb3216a7156a91]). Expected version 1 got 0
W20250115 03:06:19.247722 14356 leader_election.cc:343] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.247793 14356 leader_election.cc:304] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:19.248571 15401 raft_consensus.cc:2743] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:19.248899 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f972872b022344418e9517131c27a22d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.249781 14518 tablet_service.cc:1467] Processing CreateTablet for tablet 99c6a9baf0f84489b6f38b4616ee1c15 (DEFAULT_TABLE table=default.Table_05584d1988f6439180926449a68a7198 [id=b67cf3fc6c134817a44199430100250d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.254293 14655 tablet_service.cc:1467] Processing CreateTablet for tablet a9807cb8c17d4b98be44e6fd5dccc150 (DEFAULT_TABLE table=default.Table_e51b0b04bb044b36be347afe90845b14 [id=17ea0c263011486f82895eb86b21fe78]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.254505 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9807cb8c17d4b98be44e6fd5dccc150. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.254608 14166 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 5cb34cb2006f4716b87bef67f3a6dd8e (table default.Table_77b3fb817e8b4502af7fae7bcb0124a5 [id=ed28aa2aaacd4ba58dcb3216a7156a91]). Expected version 1 got 0
I20250115 03:06:19.255143 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99c6a9baf0f84489b6f38b4616ee1c15. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.257807 14823 ts_tablet_manager.cc:1905] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.258571 14660 tablet_service.cc:1467] Processing CreateTablet for tablet c6d6cbc01b944e408dfe79a84674b78d (DEFAULT_TABLE table=default.Table_b3d1f8a0bba14291b9577e6201a2bb06 [id=bfaeaaf9694d4c1eb6a7247c4202a1c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.258781 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6d6cbc01b944e408dfe79a84674b78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.259302 14820 ts_tablet_manager.cc:1905] T 93fa227c0de74a3fa86778641fdcda3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.260463 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 58130f2b27b745d08c0002ac96af6d72 (table default.Table_fe2cfc8f0c564170b3c291cab294193f [id=6aa86d8861174b9091f29b0cfd39fa55]). Expected version 2 got 0
I20250115 03:06:19.260524 15453 consensus_queue.cc:1035] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:19.260615 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 5780da67ce984297ba33d3c6901c9e3a (table default.Table_d40762fac357436da2798c2bb019dff6 [id=79eb8c9694b74a31a4253b33d064e230]). Expected version 2 got 1
I20250115 03:06:19.261291 14554 raft_consensus.cc:3054] T 8efd181c42eb42628a9f36fad4fa8d64 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:19.263546 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5cb34cb2006f4716b87bef67f3a6dd8e (table default.Table_77b3fb817e8b4502af7fae7bcb0124a5 [id=ed28aa2aaacd4ba58dcb3216a7156a91]). Expected version 1 got 0
I20250115 03:06:19.266721 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_71fadacb26bd4e4a8ddcd48e403350bb" } modify_external_catalogs: true
I20250115 03:06:19.266844 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_71fadacb26bd4e4a8ddcd48e403350bb" } modify_external_catalogs: true
I20250115 03:06:19.266870 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_48f283b8ffcc4e5c8416b91ca6038d23"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.267212 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_48f283b8ffcc4e5c8416b91ca6038d23 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.268460 14653 tablet_service.cc:1467] Processing CreateTablet for tablet e95269dd9a1c4d9a897f3c098628f2c0 (DEFAULT_TABLE table=default.Table_aed56e66118644fa9210464c3953ebdf [id=b57368b0727b4358b6510fae2d06d08e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.268692 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95269dd9a1c4d9a897f3c098628f2c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.270685 14534 tablet_service.cc:1467] Processing CreateTablet for tablet e5b483637c08401d92c1ab63cfe6f3c6 (DEFAULT_TABLE table=default.Table_0816f2fcac824630b5042becb8b0ed18 [id=df3d22f95df04db09d58e7c5f856d3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.270931 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5b483637c08401d92c1ab63cfe6f3c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.272537 14527 tablet_service.cc:1467] Processing CreateTablet for tablet c6d6cbc01b944e408dfe79a84674b78d (DEFAULT_TABLE table=default.Table_b3d1f8a0bba14291b9577e6201a2bb06 [id=bfaeaaf9694d4c1eb6a7247c4202a1c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.272758 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6d6cbc01b944e408dfe79a84674b78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.273818 14390 tablet_service.cc:1467] Processing CreateTablet for tablet dc8bbf060a644c27bde4b04a72e87c16 (DEFAULT_TABLE table=default.Table_fa2d8bed137445d7a9773e35a10912b2 [id=ebe573c2f94a4a5ea993017fec7b3267]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.273888 14823 ts_tablet_manager.cc:1918] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:19.274364 14823 log.cc:1198] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/074fdc09e4e24078ad0bb588b32ff24f
I20250115 03:06:19.274665 14823 ts_tablet_manager.cc:1939] T 074fdc09e4e24078ad0bb588b32ff24f P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:19.275300 14820 ts_tablet_manager.cc:1918] T 93fa227c0de74a3fa86778641fdcda3d P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.277463 14820 log.cc:1198] T 93fa227c0de74a3fa86778641fdcda3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/93fa227c0de74a3fa86778641fdcda3d
I20250115 03:06:19.278004 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 98913be4c8e74d7185957b0c4ad6abce
I20250115 03:06:19.278898 14389 tablet_service.cc:1514] Processing DeleteTablet for tablet 98913be4c8e74d7185957b0c4ad6abce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.275413 14537 tablet_service.cc:1467] Processing CreateTablet for tablet e63809cf809c453cb49411358050402b (DEFAULT_TABLE table=default.Table_e1ceae92543048f39acd7f81454b14f8 [id=a442ed735eab4be09b704ac69c7f4998]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.279131 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e63809cf809c453cb49411358050402b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.279201 14820 ts_tablet_manager.cc:1939] T 93fa227c0de74a3fa86778641fdcda3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.279605 14820 tablet_replica.cc:331] T 38c9e64e1f5e4f2394afa876553357bf P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.279853 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 074fdc09e4e24078ad0bb588b32ff24f (table default.Table_87c206116f514479ab9de8cd4a05817f [id=eaddac40b10648b1933b121cc70d68e8]) successfully deleted
I20250115 03:06:19.280484 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2af9569a0254f32b4b6ceab5bfbfa74. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.280622 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc8bbf060a644c27bde4b04a72e87c16. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.280722 14819 ts_tablet_manager.cc:1918] T ca2093980c8e480e99ae0cd79606f318 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.280740 14395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9807cb8c17d4b98be44e6fd5dccc150. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.281219 14392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5b483637c08401d92c1ab63cfe6f3c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.281354 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f16c6d7c2ad4f4abe3ad599305ffa2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.281476 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf80a961cac644e7aa338c41bb430a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.281625 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 578506b0528442cf834a8e65a400c635. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.281893 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55538831d8ea47eda40c36d42c19435f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.282909 14391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6d6cbc01b944e408dfe79a84674b78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.283604 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e63809cf809c453cb49411358050402b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.283926 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95269dd9a1c4d9a897f3c098628f2c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.284312 14823 tablet_replica.cc:331] T cff95c69fb9045cf9ec0ebb128c9ef6b P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:19.284431 14823 raft_consensus.cc:2238] T cff95c69fb9045cf9ec0ebb128c9ef6b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.284526 14823 raft_consensus.cc:2267] T cff95c69fb9045cf9ec0ebb128c9ef6b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.287204 14819 log.cc:1198] T ca2093980c8e480e99ae0cd79606f318 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/ca2093980c8e480e99ae0cd79606f318
I20250115 03:06:19.287487 14819 ts_tablet_manager.cc:1939] T ca2093980c8e480e99ae0cd79606f318 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.288134 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet ca2093980c8e480e99ae0cd79606f318 (table default.Table_0aecd3e601c04f0497cefafda025e6e1 [id=bde9696f4c534fe58bb1de4fd7e00201]) successfully deleted
I20250115 03:06:19.288570 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f972872b022344418e9517131c27a22d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.288780 14819 tablet_replica.cc:331] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.289147 14811 tablet_bootstrap.cc:492] T 6784e353df554bb2aa10b667c8b488da P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.289108 14819 raft_consensus.cc:2238] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.289227 14811 ts_tablet_manager.cc:1397] T 6784e353df554bb2aa10b667c8b488da P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.083s user 0.001s sys 0.001s
I20250115 03:06:19.289366 14819 raft_consensus.cc:2267] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.289994 14811 raft_consensus.cc:357] T 6784e353df554bb2aa10b667c8b488da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.290091 14811 raft_consensus.cc:383] T 6784e353df554bb2aa10b667c8b488da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.290124 14811 raft_consensus.cc:738] T 6784e353df554bb2aa10b667c8b488da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.290235 14811 consensus_queue.cc:260] T 6784e353df554bb2aa10b667c8b488da P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.290498 14811 ts_tablet_manager.cc:1428] T 6784e353df554bb2aa10b667c8b488da P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:19.290661 14811 tablet_bootstrap.cc:492] T 3007a0506ce14c6c89c600b98a81b168 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.291550 14811 tablet_bootstrap.cc:654] T 3007a0506ce14c6c89c600b98a81b168 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.266875 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_bfc2bf75532d47a091aeb698f89c36dd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.291826 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bfc2bf75532d47a091aeb698f89c36dd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.293493 14819 ts_tablet_manager.cc:1905] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.295861 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 917a0c0014a741ba879b7fbda8c9cd3d (DEFAULT_TABLE table=default.Table_7c5632f41fd641dabc346cc2c77f2988 [id=3dd47349b939437985e66496f352c6b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.296128 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 917a0c0014a741ba879b7fbda8c9cd3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.297019 14819 ts_tablet_manager.cc:1918] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.297140 14819 log.cc:1198] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/da95daa40c5b4615a0219f6d8e3770f6
I20250115 03:06:19.297468 14819 ts_tablet_manager.cc:1939] T da95daa40c5b4615a0219f6d8e3770f6 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.297683 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_75817b46a6414888988ff7373ea11b27"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:19.297784 14655 tablet_service.cc:1514] Processing DeleteTablet for tablet 98913be4c8e74d7185957b0c4ad6abce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:19.297986 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_75817b46a6414888988ff7373ea11b27 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.298264 14660 tablet_service.cc:1467] Processing CreateTablet for tablet dc8bbf060a644c27bde4b04a72e87c16 (DEFAULT_TABLE table=default.Table_fa2d8bed137445d7a9773e35a10912b2 [id=ebe573c2f94a4a5ea993017fec7b3267]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.298468 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc8bbf060a644c27bde4b04a72e87c16. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.298832 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet da95daa40c5b4615a0219f6d8e3770f6 (table default.Table_4bde77af131940a099eabda16fa2ccb2 [id=4f952e57c86746f6bb1df72e13589832]) successfully deleted
W20250115 03:06:19.299273 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 98913be4c8e74d7185957b0c4ad6abce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 98913be4c8e74d7185957b0c4ad6abce already in progress: creating tablet
I20250115 03:06:19.300693 14823 ts_tablet_manager.cc:1905] T cff95c69fb9045cf9ec0ebb128c9ef6b P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.305066 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_cf48da676b2a490a8903b95bfab1dbe9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.305337 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cf48da676b2a490a8903b95bfab1dbe9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.305516 14820 raft_consensus.cc:2238] T 38c9e64e1f5e4f2394afa876553357bf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.305684 14820 raft_consensus.cc:2267] T 38c9e64e1f5e4f2394afa876553357bf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.301568 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 917a0c0014a741ba879b7fbda8c9cd3d (DEFAULT_TABLE table=default.Table_7c5632f41fd641dabc346cc2c77f2988 [id=3dd47349b939437985e66496f352c6b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.306712 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 917a0c0014a741ba879b7fbda8c9cd3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.307953 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_62faf28a3d6b45698cdbb0e301130a90" } modify_external_catalogs: true
I20250115 03:06:19.308058 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_62faf28a3d6b45698cdbb0e301130a90" } modify_external_catalogs: true
I20250115 03:06:19.308192 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4db09fb126344de28b8a6b6baa54183a" } modify_external_catalogs: true
I20250115 03:06:19.308261 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4db09fb126344de28b8a6b6baa54183a" } modify_external_catalogs: true
I20250115 03:06:19.308696 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0816f2fcac824630b5042becb8b0ed18" } modify_external_catalogs: true
I20250115 03:06:19.308784 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0816f2fcac824630b5042becb8b0ed18" } modify_external_catalogs: true
I20250115 03:06:19.309037 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_9342b800c44944b1bf0997bc3262ab94"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.309321 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9342b800c44944b1bf0997bc3262ab94 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.307974 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8ac3b54226f54715849ee4eb26e7a42f" } modify_external_catalogs: true
I20250115 03:06:19.309648 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8ac3b54226f54715849ee4eb26e7a42f" } modify_external_catalogs: true
I20250115 03:06:19.315214 14820 ts_tablet_manager.cc:1905] T 38c9e64e1f5e4f2394afa876553357bf P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.318907 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet e5b483637c08401d92c1ab63cfe6f3c6
I20250115 03:06:19.319928 14809 tablet_bootstrap.cc:492] T c251960c7c9a44429102a294f35f925c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.320012 14809 ts_tablet_manager.cc:1397] T c251960c7c9a44429102a294f35f925c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.111s user 0.002s sys 0.000s
I20250115 03:06:19.320323 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet e5b483637c08401d92c1ab63cfe6f3c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.320415 14809 raft_consensus.cc:357] T c251960c7c9a44429102a294f35f925c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.320508 14809 raft_consensus.cc:383] T c251960c7c9a44429102a294f35f925c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.320544 14809 raft_consensus.cc:738] T c251960c7c9a44429102a294f35f925c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.320645 14809 consensus_queue.cc:260] T c251960c7c9a44429102a294f35f925c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.320771 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_c9451afa33814f508ab562805101afc8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:19.320899 14809 ts_tablet_manager.cc:1428] T c251960c7c9a44429102a294f35f925c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.321096 14809 tablet_bootstrap.cc:492] T e2c6db404348466ea5211ffa69c8f01c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.322098 14809 tablet_bootstrap.cc:654] T e2c6db404348466ea5211ffa69c8f01c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:19.322183 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c9451afa33814f508ab562805101afc8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:19.326604 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet e5b483637c08401d92c1ab63cfe6f3c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5b483637c08401d92c1ab63cfe6f3c6 already in progress: creating tablet
I20250115 03:06:19.328394 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet c251960c7c9a44429102a294f35f925c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.328430 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet e5b483637c08401d92c1ab63cfe6f3c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.329274 14819 tablet_replica.cc:331] T c251960c7c9a44429102a294f35f925c P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.329449 14819 raft_consensus.cc:2238] T c251960c7c9a44429102a294f35f925c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.329605 14819 raft_consensus.cc:2267] T c251960c7c9a44429102a294f35f925c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.331305 14819 ts_tablet_manager.cc:1905] T c251960c7c9a44429102a294f35f925c P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.340976 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 09f6eb327daa4b69a974db2aeced0603 (DEFAULT_TABLE table=default.Table_48f283b8ffcc4e5c8416b91ca6038d23 [id=ffafce3959ca46b6b2c873ef01209a2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.341217 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09f6eb327daa4b69a974db2aeced0603. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.342341 14518 tablet_service.cc:1467] Processing CreateTablet for tablet e95269dd9a1c4d9a897f3c098628f2c0 (DEFAULT_TABLE table=default.Table_aed56e66118644fa9210464c3953ebdf [id=b57368b0727b4358b6510fae2d06d08e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.343199 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 09f6eb327daa4b69a974db2aeced0603 (DEFAULT_TABLE table=default.Table_48f283b8ffcc4e5c8416b91ca6038d23 [id=ffafce3959ca46b6b2c873ef01209a2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.343482 14812 tablet_bootstrap.cc:492] T 3b9c09b6315345039cdea9eeee3f181a P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:19.343585 14812 ts_tablet_manager.cc:1397] T 3b9c09b6315345039cdea9eeee3f181a P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.141s user 0.002s sys 0.000s
I20250115 03:06:19.343598 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09f6eb327daa4b69a974db2aeced0603. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.344034 14812 raft_consensus.cc:357] T 3b9c09b6315345039cdea9eeee3f181a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.344131 14812 raft_consensus.cc:383] T 3b9c09b6315345039cdea9eeee3f181a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.344162 14812 raft_consensus.cc:738] T 3b9c09b6315345039cdea9eeee3f181a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.344267 14812 consensus_queue.cc:260] T 3b9c09b6315345039cdea9eeee3f181a P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.344359 14819 ts_tablet_manager.cc:1918] T c251960c7c9a44429102a294f35f925c P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.344420 14819 log.cc:1198] T c251960c7c9a44429102a294f35f925c P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/c251960c7c9a44429102a294f35f925c
I20250115 03:06:19.344573 14812 ts_tablet_manager.cc:1428] T 3b9c09b6315345039cdea9eeee3f181a P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.344605 14819 ts_tablet_manager.cc:1939] T c251960c7c9a44429102a294f35f925c P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.344779 14812 tablet_bootstrap.cc:492] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:19.345678 14812 tablet_bootstrap.cc:654] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.348165 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e95269dd9a1c4d9a897f3c098628f2c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.349987 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 015ec42caa1b468ea10f25c8f26e278a
I20250115 03:06:19.350190 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 60997f625d074960ad75aef5ba14b9c2
I20250115 03:06:19.350580 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 980b69b60a354519b0691ecb0bbd6c70
I20250115 03:06:19.350816 14811 tablet_bootstrap.cc:492] T 3007a0506ce14c6c89c600b98a81b168 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.350836 14655 tablet_service.cc:1514] Processing DeleteTablet for tablet 60997f625d074960ad75aef5ba14b9c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.350900 14811 ts_tablet_manager.cc:1397] T 3007a0506ce14c6c89c600b98a81b168 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.060s user 0.000s sys 0.002s
I20250115 03:06:19.351035 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 60997f625d074960ad75aef5ba14b9c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.351423 14655 tablet_service.cc:1514] Processing DeleteTablet for tablet 980b69b60a354519b0691ecb0bbd6c70 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.351454 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 980b69b60a354519b0691ecb0bbd6c70 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.351451 14811 raft_consensus.cc:357] T 3007a0506ce14c6c89c600b98a81b168 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:19.351588 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 015ec42caa1b468ea10f25c8f26e278a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.351681 14811 raft_consensus.cc:383] T 3007a0506ce14c6c89c600b98a81b168 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.351763 14811 raft_consensus.cc:738] T 3007a0506ce14c6c89c600b98a81b168 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.352595 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 015ec42caa1b468ea10f25c8f26e278a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.352746 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b9c09b6315345039cdea9eeee3f181a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.353550 14521 tablet_service.cc:1467] Processing CreateTablet for tablet a9807cb8c17d4b98be44e6fd5dccc150 (DEFAULT_TABLE table=default.Table_e51b0b04bb044b36be347afe90845b14 [id=17ea0c263011486f82895eb86b21fe78]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.353993 14811 consensus_queue.cc:260] T 3007a0506ce14c6c89c600b98a81b168 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:19.354342 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9807cb8c17d4b98be44e6fd5dccc150. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.356379 14809 tablet_bootstrap.cc:492] T e2c6db404348466ea5211ffa69c8f01c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.356465 14809 ts_tablet_manager.cc:1397] T e2c6db404348466ea5211ffa69c8f01c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.035s user 0.002s sys 0.000s
I20250115 03:06:19.354908 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 917a0c0014a741ba879b7fbda8c9cd3d (DEFAULT_TABLE table=default.Table_7c5632f41fd641dabc346cc2c77f2988 [id=3dd47349b939437985e66496f352c6b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.356899 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 917a0c0014a741ba879b7fbda8c9cd3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.356905 14809 raft_consensus.cc:357] T e2c6db404348466ea5211ffa69c8f01c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.357028 14809 raft_consensus.cc:383] T e2c6db404348466ea5211ffa69c8f01c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.357059 14809 raft_consensus.cc:738] T e2c6db404348466ea5211ffa69c8f01c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.357163 14809 consensus_queue.cc:260] T e2c6db404348466ea5211ffa69c8f01c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.357436 14809 ts_tablet_manager.cc:1428] T e2c6db404348466ea5211ffa69c8f01c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:19.355409 15426 tablet.cc:1721] T 5780da67ce984297ba33d3c6901c9e3a P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:19.358035 14811 ts_tablet_manager.cc:1428] T 3007a0506ce14c6c89c600b98a81b168 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.007s user 0.000s sys 0.001s
I20250115 03:06:19.356016 14537 tablet_service.cc:1467] Processing CreateTablet for tablet dc8bbf060a644c27bde4b04a72e87c16 (DEFAULT_TABLE table=default.Table_fa2d8bed137445d7a9773e35a10912b2 [id=ebe573c2f94a4a5ea993017fec7b3267]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.358790 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc8bbf060a644c27bde4b04a72e87c16. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.359881 14811 tablet_bootstrap.cc:492] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.360512 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 98913be4c8e74d7185957b0c4ad6abce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.361896 14811 tablet_bootstrap.cc:654] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.367617 15426 tablet.cc:1721] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:19.367945 15970 tablet.cc:1721] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:19.371714 14820 ts_tablet_manager.cc:1918] T 38c9e64e1f5e4f2394afa876553357bf P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:19.373342 15617 tablet.cc:1721] T 0580a8f7b2984162b9275d74334b1905 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:19.374069 14809 tablet_bootstrap.cc:492] T 62ec9057d85447bdb6b86cdc1f9015ac P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.375028 14809 tablet_bootstrap.cc:654] T 62ec9057d85447bdb6b86cdc1f9015ac P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.372862 14823 ts_tablet_manager.cc:1918] T cff95c69fb9045cf9ec0ebb128c9ef6b P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.377976 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet e2c6db404348466ea5211ffa69c8f01c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.378005 14823 log.cc:1198] T cff95c69fb9045cf9ec0ebb128c9ef6b P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/cff95c69fb9045cf9ec0ebb128c9ef6b
I20250115 03:06:19.378274 14823 ts_tablet_manager.cc:1939] T cff95c69fb9045cf9ec0ebb128c9ef6b P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:19.379161 14823 tablet_replica.cc:331] T 1f2dc6928e2941ef9b5b5a7891376500 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:19.379230 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet cff95c69fb9045cf9ec0ebb128c9ef6b (table default.Table_1ed9a3074bb7436797358399890084b7 [id=8a02676ab3334b7c81ce22069b29551f]) successfully deleted
I20250115 03:06:19.382665 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet c251960c7c9a44429102a294f35f925c (table default.Table_90aedfcdb1274e1497cf36620b980768 [id=9e4f5de45eb24beabef550d15dcad8bf]) successfully deleted
W20250115 03:06:19.382968 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 60997f625d074960ad75aef5ba14b9c2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60997f625d074960ad75aef5ba14b9c2 already in progress: creating tablet
W20250115 03:06:19.383123 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 980b69b60a354519b0691ecb0bbd6c70 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 980b69b60a354519b0691ecb0bbd6c70 already in progress: creating tablet
I20250115 03:06:19.383263 14819 tablet_replica.cc:331] T e2c6db404348466ea5211ffa69c8f01c P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
W20250115 03:06:19.383268 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 015ec42caa1b468ea10f25c8f26e278a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 015ec42caa1b468ea10f25c8f26e278a already in progress: creating tablet
I20250115 03:06:19.383373 14819 raft_consensus.cc:2238] T e2c6db404348466ea5211ffa69c8f01c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.383689 14819 raft_consensus.cc:2267] T e2c6db404348466ea5211ffa69c8f01c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.384608 14819 ts_tablet_manager.cc:1905] T e2c6db404348466ea5211ffa69c8f01c P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.385064 14163 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_3acc4838fabe430994487140caafa10d [id=0790b08615854d3688f940aba20b6520] alter complete (version 3)
I20250115 03:06:19.387981 14819 ts_tablet_manager.cc:1918] T e2c6db404348466ea5211ffa69c8f01c P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.388105 14819 log.cc:1198] T e2c6db404348466ea5211ffa69c8f01c P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/e2c6db404348466ea5211ffa69c8f01c
I20250115 03:06:19.388439 14819 ts_tablet_manager.cc:1939] T e2c6db404348466ea5211ffa69c8f01c P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.389017 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet e2c6db404348466ea5211ffa69c8f01c (table default.Table_ef19dd8e16aa4bf9a97897ebc1c8eef5 [id=38bfb0375c334d16b97c7251d6ddd76b]) successfully deleted
I20250115 03:06:19.391222 14820 log.cc:1198] T 38c9e64e1f5e4f2394afa876553357bf P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/38c9e64e1f5e4f2394afa876553357bf
I20250115 03:06:19.391453 14820 ts_tablet_manager.cc:1939] T 38c9e64e1f5e4f2394afa876553357bf P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.392206 14820 tablet_replica.cc:331] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.392331 14820 raft_consensus.cc:2238] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.392421 14820 raft_consensus.cc:2267] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.392701 14820 ts_tablet_manager.cc:1905] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.394364 14820 ts_tablet_manager.cc:1918] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.394426 14820 log.cc:1198] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/56c600fe18504e05bab2bcde3c89861d
I20250115 03:06:19.394570 14823 raft_consensus.cc:2238] T 1f2dc6928e2941ef9b5b5a7891376500 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.394670 14823 raft_consensus.cc:2267] T 1f2dc6928e2941ef9b5b5a7891376500 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.395346 14823 ts_tablet_manager.cc:1905] T 1f2dc6928e2941ef9b5b5a7891376500 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.378839 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 93fa227c0de74a3fa86778641fdcda3d (table default.Table_4f241c86102a4bbfb509a2d5d76953e9 [id=03b92c08d0b0473d83090c58e450c84b]) successfully deleted
W20250115 03:06:19.397119 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet daea48ecc69d441ab66fcb40ea2e7a66 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet daea48ecc69d441ab66fcb40ea2e7a66 already in progress: creating tablet
I20250115 03:06:19.397415 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 38c9e64e1f5e4f2394afa876553357bf (table default.Table_57e91bade4c54372abe1dbba44434098 [id=91731a4895d4478fad68b1bcbdc45d5d]) successfully deleted
W20250115 03:06:19.397514 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet d2564a20857f45d9a0c7e3b74eb58a91 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d2564a20857f45d9a0c7e3b74eb58a91 already in progress: creating tablet
W20250115 03:06:19.397600 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 0792e52d171446b48a7c9d1f03d37a79 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0792e52d171446b48a7c9d1f03d37a79 already in progress: creating tablet
W20250115 03:06:19.397691 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet da95daa40c5b4615a0219f6d8e3770f6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet da95daa40c5b4615a0219f6d8e3770f6 already in progress: creating tablet
I20250115 03:06:19.398795 14812 tablet_bootstrap.cc:492] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:19.398874 14812 ts_tablet_manager.cc:1397] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.054s user 0.001s sys 0.000s
I20250115 03:06:19.399266 14812 raft_consensus.cc:357] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.399353 14812 raft_consensus.cc:383] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.399381 14812 raft_consensus.cc:738] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.399494 14812 consensus_queue.cc:260] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.399741 14812 ts_tablet_manager.cc:1428] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.400023 14812 tablet_bootstrap.cc:492] T 80aa570e2c154172ad6da29d268bede7 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:19.400954 14812 tablet_bootstrap.cc:654] T 80aa570e2c154172ad6da29d268bede7 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.405243 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet d140be0fab694f59b8a535392970fad0 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:19.405512 14162 master_service.cc:946] ReplaceTablet: received request to replace tablet 3e170ced1612442ebcfdaff6d9826ea3 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:19.405645 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_1df66a4877bb4ba993ae442eb173bf82" } new_table_name: "default.Table_83fe43c0616b4351b2b4457195378f0c" modify_external_catalogs: true
I20250115 03:06:19.405728 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet 9c436975bd034068a6d58d5831db1896 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:19.406373 14167 master_service.cc:946] ReplaceTablet: received request to replace tablet 9bb0b7a8acfd40738abcc951936b1116 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:19.406773 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_162fe16a9dfb414784a702bafb5d9938"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.407070 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_162fe16a9dfb414784a702bafb5d9938 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.408855 14809 tablet_bootstrap.cc:492] T 62ec9057d85447bdb6b86cdc1f9015ac P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.408998 14809 ts_tablet_manager.cc:1397] T 62ec9057d85447bdb6b86cdc1f9015ac P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.035s user 0.001s sys 0.001s
I20250115 03:06:19.409410 14809 raft_consensus.cc:357] T 62ec9057d85447bdb6b86cdc1f9015ac P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.409511 14809 raft_consensus.cc:383] T 62ec9057d85447bdb6b86cdc1f9015ac P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.409543 14809 raft_consensus.cc:738] T 62ec9057d85447bdb6b86cdc1f9015ac P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.409649 14809 consensus_queue.cc:260] T 62ec9057d85447bdb6b86cdc1f9015ac P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.409946 14809 ts_tablet_manager.cc:1428] T 62ec9057d85447bdb6b86cdc1f9015ac P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.411100 14809 tablet_bootstrap.cc:492] T e5a8293f43134a54b3f8dfb5fb99d6ea P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.411392 14823 ts_tablet_manager.cc:1918] T 1f2dc6928e2941ef9b5b5a7891376500 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.412046 14809 tablet_bootstrap.cc:654] T e5a8293f43134a54b3f8dfb5fb99d6ea P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.412165 14823 log.cc:1198] T 1f2dc6928e2941ef9b5b5a7891376500 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/1f2dc6928e2941ef9b5b5a7891376500
I20250115 03:06:19.412317 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_736f17451e344c1e9b3c1e0797cb2a6d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:19.412434 14823 ts_tablet_manager.cc:1939] T 1f2dc6928e2941ef9b5b5a7891376500 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
W20250115 03:06:19.412621 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_736f17451e344c1e9b3c1e0797cb2a6d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.401510 14811 tablet_bootstrap.cc:492] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.413272 14811 ts_tablet_manager.cc:1397] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.053s user 0.001s sys 0.000s
I20250115 03:06:19.413344 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 1f2dc6928e2941ef9b5b5a7891376500 (table default.Table_652f8e06e005491381404a11759aefb1 [id=6fb6e311a0af4f51b190a880f84373d1]) successfully deleted
W20250115 03:06:19.413477 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 98913be4c8e74d7185957b0c4ad6abce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 98913be4c8e74d7185957b0c4ad6abce already in progress: creating tablet
W20250115 03:06:19.413626 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet e5b483637c08401d92c1ab63cfe6f3c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5b483637c08401d92c1ab63cfe6f3c6 already in progress: creating tablet
I20250115 03:06:19.413709 14811 raft_consensus.cc:357] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:19.413820 14811 raft_consensus.cc:383] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.413854 14811 raft_consensus.cc:738] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.413985 14811 consensus_queue.cc:260] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:19.416029 14811 ts_tablet_manager.cc:1428] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:19.416201 14811 tablet_bootstrap.cc:492] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.416983 14811 tablet_bootstrap.cc:654] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:19.418336 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4febfae2f2794669b5997a1fa7ef7412 consensus state has no leader
I20250115 03:06:19.419293 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 62ec9057d85447bdb6b86cdc1f9015ac with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.403585 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet e5b483637c08401d92c1ab63cfe6f3c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.404165 14518 tablet_service.cc:1467] Processing CreateTablet for tablet 09f6eb327daa4b69a974db2aeced0603 (DEFAULT_TABLE table=default.Table_48f283b8ffcc4e5c8416b91ca6038d23 [id=ffafce3959ca46b6b2c873ef01209a2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.420023 14820 ts_tablet_manager.cc:1939] T 56c600fe18504e05bab2bcde3c89861d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.420125 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09f6eb327daa4b69a974db2aeced0603. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.420257 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_543c011b1f2a4240be626d4454882dc8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.420526 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_543c011b1f2a4240be626d4454882dc8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.421157 14819 tablet_replica.cc:331] T 62ec9057d85447bdb6b86cdc1f9015ac P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.421267 14819 raft_consensus.cc:2238] T 62ec9057d85447bdb6b86cdc1f9015ac P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.421573 14819 raft_consensus.cc:2267] T 62ec9057d85447bdb6b86cdc1f9015ac P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.421974 14819 ts_tablet_manager.cc:1905] T 62ec9057d85447bdb6b86cdc1f9015ac P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.422187 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet 623887ec17374f598f48054e17582f3f from {username='slave'} at 127.0.0.1:60432
W20250115 03:06:19.422348 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 60997f625d074960ad75aef5ba14b9c2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60997f625d074960ad75aef5ba14b9c2 already in progress: creating tablet
I20250115 03:06:19.422439 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_c596014f7fe943d3bc6faa31f1a5f974"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.422724 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c596014f7fe943d3bc6faa31f1a5f974 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.422760 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_70d0b700eac34892b4ef769d739481fd" } new_table_name: "default.Table_a2c05ae2d81b4dd190ad94b5212e3bf2" modify_external_catalogs: true
I20250115 03:06:19.404405 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 60997f625d074960ad75aef5ba14b9c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.404978 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet 980b69b60a354519b0691ecb0bbd6c70 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.405393 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 015ec42caa1b468ea10f25c8f26e278a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.424089 14820 tablet_replica.cc:331] T d28ed8538c4a44d18279371b826c22a5 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.424214 14820 raft_consensus.cc:2238] T d28ed8538c4a44d18279371b826c22a5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.424315 14820 raft_consensus.cc:2267] T d28ed8538c4a44d18279371b826c22a5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.424638 14820 ts_tablet_manager.cc:1905] T d28ed8538c4a44d18279371b826c22a5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.425514 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_bfc2bf75532d47a091aeb698f89c36dd" } new_table_name: "default.Table_6cd962b7ee004482b49e10d91eb9ee3f" modify_external_catalogs: true
I20250115 03:06:19.425709 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d1185d5f9bb6408585b742d6b55fec52" } new_table_name: "default.Table_2ad8dffd9f274a8184b5a06c7254b1b5" modify_external_catalogs: true
W20250115 03:06:19.426249 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 980b69b60a354519b0691ecb0bbd6c70 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 980b69b60a354519b0691ecb0bbd6c70 already in progress: creating tablet
I20250115 03:06:19.427127 14171 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 6784e353df554bb2aa10b667c8b488da (table default.Table_91fae283e01841f2a274e89f22ebc882 [id=4d6cb6f5c7354beebee1b3fbe4384d54]). Expected version 1 got 0
I20250115 03:06:19.427354 14819 ts_tablet_manager.cc:1918] T 62ec9057d85447bdb6b86cdc1f9015ac P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.427410 14819 log.cc:1198] T 62ec9057d85447bdb6b86cdc1f9015ac P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/62ec9057d85447bdb6b86cdc1f9015ac
I20250115 03:06:19.427595 14819 ts_tablet_manager.cc:1939] T 62ec9057d85447bdb6b86cdc1f9015ac P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
W20250115 03:06:19.428056 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 015ec42caa1b468ea10f25c8f26e278a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 015ec42caa1b468ea10f25c8f26e278a already in progress: creating tablet
W20250115 03:06:19.428547 14155 catalog_manager.cc:4670] Async tablet task AlterTable RPC for tablet 55538831d8ea47eda40c36d42c19435f (table default.Table_b8b40ffaa9e04cf481788f0d2e8519e9 [id=c99c3573d2394eada9631427ac38322d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 55538831d8ea47eda40c36d42c19435f consensus state has no leader
I20250115 03:06:19.428817 14812 tablet_bootstrap.cc:492] T 80aa570e2c154172ad6da29d268bede7 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:19.428892 14812 ts_tablet_manager.cc:1397] T 80aa570e2c154172ad6da29d268bede7 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20250115 03:06:19.429307 14812 raft_consensus.cc:357] T 80aa570e2c154172ad6da29d268bede7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.429404 14812 raft_consensus.cc:383] T 80aa570e2c154172ad6da29d268bede7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.429436 14812 raft_consensus.cc:738] T 80aa570e2c154172ad6da29d268bede7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.429555 14812 consensus_queue.cc:260] T 80aa570e2c154172ad6da29d268bede7 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.429826 14812 ts_tablet_manager.cc:1428] T 80aa570e2c154172ad6da29d268bede7 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.429987 14812 tablet_bootstrap.cc:492] T cc67fb818b8146629d92f9ac1944dba6 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:19.427963 14823 tablet_replica.cc:331] T 3b9c09b6315345039cdea9eeee3f181a P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:19.430249 14823 raft_consensus.cc:2238] T 3b9c09b6315345039cdea9eeee3f181a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.430338 14823 raft_consensus.cc:2267] T 3b9c09b6315345039cdea9eeee3f181a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.430706 14812 tablet_bootstrap.cc:654] T cc67fb818b8146629d92f9ac1944dba6 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.430724 14823 ts_tablet_manager.cc:1905] T 3b9c09b6315345039cdea9eeee3f181a P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.435125 14823 ts_tablet_manager.cc:1918] T 3b9c09b6315345039cdea9eeee3f181a P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.435240 14823 log.cc:1198] T 3b9c09b6315345039cdea9eeee3f181a P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/3b9c09b6315345039cdea9eeee3f181a
I20250115 03:06:19.437233 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 80aa570e2c154172ad6da29d268bede7 with delete_type TABLET_DATA_DELETED (replaced by tablet 815ef85ce5cf4ae0a56c5734837e03ba) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.437490 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 6618580ae8be4cb7b503d8f9cc5322dc (DEFAULT_TABLE table=default.Table_c9451afa33814f508ab562805101afc8 [id=ff98684fefe64335b07be43e255d3e35]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.437693 14823 ts_tablet_manager.cc:1939] T 3b9c09b6315345039cdea9eeee3f181a P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:19.437928 14655 tablet_service.cc:1467] Processing CreateTablet for tablet 7e23caf72949411e8e401c37c8b29345 (DEFAULT_TABLE table=default.Table_9342b800c44944b1bf0997bc3262ab94 [id=39677fe8c4024c23bfd8b15250f1b4e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.438356 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 9fb51a41e62a4094879887626d199a7c (DEFAULT_TABLE table=default.Table_75817b46a6414888988ff7373ea11b27 [id=dfa93dac2ee14cfd8fe486869349c288]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.438580 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 3b9c09b6315345039cdea9eeee3f181a (table default.Table_561cca4d810b4e73934fcee99a8994c7 [id=270d1081a9a04f09b76df0f45cfbbff6]) successfully deleted
I20250115 03:06:19.438627 14853 raft_consensus.cc:491] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.438694 14853 raft_consensus.cc:513] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:19.439008 14853 leader_election.cc:290] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
W20250115 03:06:19.439662 14493 leader_election.cc:343] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: aa7754f577af4ef6b6fca41e61e5965c
W20250115 03:06:19.439806 14493 leader_election.cc:343] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: aa7754f577af4ef6b6fca41e61e5965c
I20250115 03:06:19.439860 14493 leader_election.cc:304] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:19.437726 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6618580ae8be4cb7b503d8f9cc5322dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.441390 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 9fb51a41e62a4094879887626d199a7c (DEFAULT_TABLE table=default.Table_75817b46a6414888988ff7373ea11b27 [id=dfa93dac2ee14cfd8fe486869349c288]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.441624 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fb51a41e62a4094879887626d199a7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.442018 14654 tablet_service.cc:1467] Processing CreateTablet for tablet 396329e33a40477d9c346db57454bc04 (DEFAULT_TABLE table=default.Table_bfc2bf75532d47a091aeb698f89c36dd [id=6ab0030c03c04d049e912404706df7ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.442226 14654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 396329e33a40477d9c346db57454bc04. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.443734 14390 tablet_service.cc:1467] Processing CreateTablet for tablet 396329e33a40477d9c346db57454bc04 (DEFAULT_TABLE table=default.Table_bfc2bf75532d47a091aeb698f89c36dd [id=6ab0030c03c04d049e912404706df7ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.443967 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 396329e33a40477d9c346db57454bc04. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.444056 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 3dc5eeb98e59478caa3130418d61a1a8 (DEFAULT_TABLE table=default.Table_cf48da676b2a490a8903b95bfab1dbe9 [id=94199a4dea724fc6803af6b5149952b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.444262 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dc5eeb98e59478caa3130418d61a1a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.444780 14809 tablet_bootstrap.cc:492] T e5a8293f43134a54b3f8dfb5fb99d6ea P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.444855 14809 ts_tablet_manager.cc:1397] T e5a8293f43134a54b3f8dfb5fb99d6ea P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.000s
I20250115 03:06:19.445310 14809 raft_consensus.cc:357] T e5a8293f43134a54b3f8dfb5fb99d6ea P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.445399 14809 raft_consensus.cc:383] T e5a8293f43134a54b3f8dfb5fb99d6ea P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.445430 14809 raft_consensus.cc:738] T e5a8293f43134a54b3f8dfb5fb99d6ea P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.445551 14809 consensus_queue.cc:260] T e5a8293f43134a54b3f8dfb5fb99d6ea P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.445811 14809 ts_tablet_manager.cc:1428] T e5a8293f43134a54b3f8dfb5fb99d6ea P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:19.445981 14809 tablet_bootstrap.cc:492] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.446168 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 3dc5eeb98e59478caa3130418d61a1a8 (DEFAULT_TABLE table=default.Table_cf48da676b2a490a8903b95bfab1dbe9 [id=94199a4dea724fc6803af6b5149952b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.446372 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dc5eeb98e59478caa3130418d61a1a8. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:19.446610 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 396329e33a40477d9c346db57454bc04 consensus state has no leader
I20250115 03:06:19.448231 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e23caf72949411e8e401c37c8b29345. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.448354 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 7e23caf72949411e8e401c37c8b29345 (DEFAULT_TABLE table=default.Table_9342b800c44944b1bf0997bc3262ab94 [id=39677fe8c4024c23bfd8b15250f1b4e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.448542 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e23caf72949411e8e401c37c8b29345. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.450145 14853 raft_consensus.cc:2743] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:19.450301 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fb51a41e62a4094879887626d199a7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.452432 14820 ts_tablet_manager.cc:1918] T d28ed8538c4a44d18279371b826c22a5 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:19.452561 14820 log.cc:1198] T d28ed8538c4a44d18279371b826c22a5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/d28ed8538c4a44d18279371b826c22a5
I20250115 03:06:19.452858 14820 ts_tablet_manager.cc:1939] T d28ed8538c4a44d18279371b826c22a5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.453716 14820 tablet_replica.cc:331] T d895f75911b7456a88e7a5159db7d60b P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.453860 14820 raft_consensus.cc:2238] T d895f75911b7456a88e7a5159db7d60b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.453974 14820 raft_consensus.cc:2267] T d895f75911b7456a88e7a5159db7d60b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.454718 14811 tablet_bootstrap.cc:492] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.454819 14811 ts_tablet_manager.cc:1397] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.000s
I20250115 03:06:19.455226 14811 raft_consensus.cc:357] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:19.455286 14809 tablet_bootstrap.cc:654] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.455317 14811 raft_consensus.cc:383] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.455344 14811 raft_consensus.cc:738] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.455444 14811 consensus_queue.cc:260] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:19.457280 14811 ts_tablet_manager.cc:1428] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:19.457415 14811 tablet_bootstrap.cc:492] T 30499e99ab7d4138a376dea40b768c32 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.458225 14811 tablet_bootstrap.cc:654] T 30499e99ab7d4138a376dea40b768c32 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.458542 14552 raft_consensus.cc:3054] T 6784e353df554bb2aa10b667c8b488da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:19.459108 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 62ec9057d85447bdb6b86cdc1f9015ac (table default.Table_5a68d6c8c53b4746a7123860185fe0cc [id=84c18cf488cc4c28aaf93fa72334acb7]) successfully deleted
I20250115 03:06:19.460196 14812 tablet_bootstrap.cc:492] T cc67fb818b8146629d92f9ac1944dba6 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:19.460287 14812 ts_tablet_manager.cc:1397] T cc67fb818b8146629d92f9ac1944dba6 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.000s
I20250115 03:06:19.460700 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 6618580ae8be4cb7b503d8f9cc5322dc (DEFAULT_TABLE table=default.Table_c9451afa33814f508ab562805101afc8 [id=ff98684fefe64335b07be43e255d3e35]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.460698 14812 raft_consensus.cc:357] T cc67fb818b8146629d92f9ac1944dba6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.460805 14812 raft_consensus.cc:383] T cc67fb818b8146629d92f9ac1944dba6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.460847 14812 raft_consensus.cc:738] T cc67fb818b8146629d92f9ac1944dba6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.461033 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6618580ae8be4cb7b503d8f9cc5322dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.460994 14812 consensus_queue.cc:260] T cc67fb818b8146629d92f9ac1944dba6 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.461148 15426 tablet.cc:1721] T 6784e353df554bb2aa10b667c8b488da P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:19.461272 14812 ts_tablet_manager.cc:1428] T cc67fb818b8146629d92f9ac1944dba6 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.461434 14812 tablet_bootstrap.cc:492] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:19.462260 14812 tablet_bootstrap.cc:654] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.462978 14823 tablet_replica.cc:331] T 80aa570e2c154172ad6da29d268bede7 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:19.463131 14823 raft_consensus.cc:2238] T 80aa570e2c154172ad6da29d268bede7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.463177 14812 tablet_bootstrap.cc:492] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:19.463299 14812 ts_tablet_manager.cc:1397] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:19.463707 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 9c436975bd034068a6d58d5831db1896
I20250115 03:06:19.463987 14164 catalog_manager.cc:6596] ReplaceTablet: tablet 9c436975bd034068a6d58d5831db1896 deleted and replaced by tablet 802a007b8d154198b291aaf9530161b9
I20250115 03:06:19.464138 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 3e170ced1612442ebcfdaff6d9826ea3
I20250115 03:06:19.464385 14162 catalog_manager.cc:6596] ReplaceTablet: tablet 3e170ced1612442ebcfdaff6d9826ea3 deleted and replaced by tablet 8e4410ce33f0464eadead04e8116f773
I20250115 03:06:19.464615 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c436975bd034068a6d58d5831db1896 with delete_type TABLET_DATA_DELETED (replaced by tablet 802a007b8d154198b291aaf9530161b9) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.464957 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet d140be0fab694f59b8a535392970fad0
I20250115 03:06:19.465216 14163 catalog_manager.cc:6596] ReplaceTablet: tablet d140be0fab694f59b8a535392970fad0 deleted and replaced by tablet eb787122e6674201bf62d6bc787e8c20
I20250115 03:06:19.465495 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e170ced1612442ebcfdaff6d9826ea3 with delete_type TABLET_DATA_DELETED (replaced by tablet 8e4410ce33f0464eadead04e8116f773) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.465801 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e170ced1612442ebcfdaff6d9826ea3 with delete_type TABLET_DATA_DELETED (replaced by tablet 8e4410ce33f0464eadead04e8116f773) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.465912 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c436975bd034068a6d58d5831db1896 with delete_type TABLET_DATA_DELETED (replaced by tablet 802a007b8d154198b291aaf9530161b9) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.466893 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet d140be0fab694f59b8a535392970fad0 with delete_type TABLET_DATA_DELETED (replaced by tablet eb787122e6674201bf62d6bc787e8c20) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:19.467087 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 3e170ced1612442ebcfdaff6d9826ea3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3e170ced1612442ebcfdaff6d9826ea3 already in progress: creating tablet
W20250115 03:06:19.467222 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b9430861457247d9a324b43bd83fbae0 consensus state has no leader
I20250115 03:06:19.467561 14812 raft_consensus.cc:357] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:19.467234 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 9c436975bd034068a6d58d5831db1896 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9c436975bd034068a6d58d5831db1896 already in progress: creating tablet
I20250115 03:06:19.467882 14812 raft_consensus.cc:383] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.467941 14655 tablet_service.cc:1514] Processing DeleteTablet for tablet cc67fb818b8146629d92f9ac1944dba6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.467962 14812 raft_consensus.cc:738] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.468096 14812 consensus_queue.cc:260] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.468247 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet d140be0fab694f59b8a535392970fad0 with delete_type TABLET_DATA_DELETED (replaced by tablet eb787122e6674201bf62d6bc787e8c20) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:19.468624 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet d140be0fab694f59b8a535392970fad0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d140be0fab694f59b8a535392970fad0 already in progress: creating tablet
I20250115 03:06:19.468677 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 56c600fe18504e05bab2bcde3c89861d (table default.Table_b94f9ef5ede64627b994e94e674b1cfa [id=3f4310bb3227447ba6fcd011f05d6924]) successfully deleted
I20250115 03:06:19.468433 14812 ts_tablet_manager.cc:1428] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:19.469485 14811 tablet_bootstrap.cc:492] T 30499e99ab7d4138a376dea40b768c32 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.469615 14811 ts_tablet_manager.cc:1397] T 30499e99ab7d4138a376dea40b768c32 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
W20250115 03:06:19.469887 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet d891020ed0bb4fc49155bd9f2f6ed35c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d891020ed0bb4fc49155bd9f2f6ed35c already in progress: creating tablet
W20250115 03:06:19.470021 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 31446d60b85d4a878e9afe142011aeb1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 31446d60b85d4a878e9afe142011aeb1 already in progress: creating tablet
I20250115 03:06:19.470209 14811 raft_consensus.cc:357] T 30499e99ab7d4138a376dea40b768c32 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.470357 14811 raft_consensus.cc:383] T 30499e99ab7d4138a376dea40b768c32 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.470433 14811 raft_consensus.cc:738] T 30499e99ab7d4138a376dea40b768c32 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.470111 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet d28ed8538c4a44d18279371b826c22a5 (table default.Table_b8b40ffaa9e04cf481788f0d2e8519e9 [id=c99c3573d2394eada9631427ac38322d]) successfully deleted
W20250115 03:06:19.470605 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 82bf1d7ac9fe4a2eb7b57cf1ab627c3d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 82bf1d7ac9fe4a2eb7b57cf1ab627c3d already in progress: creating tablet
I20250115 03:06:19.470654 14811 consensus_queue.cc:260] T 30499e99ab7d4138a376dea40b768c32 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.471096 14811 ts_tablet_manager.cc:1428] T 30499e99ab7d4138a376dea40b768c32 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.471312 14811 tablet_bootstrap.cc:492] T 8b0f0539530643f78207e528c39530af P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.472240 14811 tablet_bootstrap.cc:654] T 8b0f0539530643f78207e528c39530af P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.473825 14812 tablet_bootstrap.cc:492] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:19.474749 14812 tablet_bootstrap.cc:654] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.476404 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3acc4838fabe430994487140caafa10d" } new_table_name: "default.Table_def871c9b5dc4a4b8fdc40fe12ccaf7a" modify_external_catalogs: true
I20250115 03:06:19.476619 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 75854bea6dec40ac9f205fec0ae66f39 (DEFAULT_TABLE table=default.Table_736f17451e344c1e9b3c1e0797cb2a6d [id=b4ff85ac73d3493da83cc2f88227e638]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.476845 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75854bea6dec40ac9f205fec0ae66f39. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.477574 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 91edfeb70a964909bf9049aa1c9fd8a5 (DEFAULT_TABLE table=default.Table_162fe16a9dfb414784a702bafb5d9938 [id=96ef5713dddc43eab4af43b8b6b60680]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.477816 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91edfeb70a964909bf9049aa1c9fd8a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.478952 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 4283bf9f53c54e7095e270fa04fc4f12 (DEFAULT_TABLE table=default.Table_543c011b1f2a4240be626d4454882dc8 [id=61210b6b042d4c04b32b6ff818232e1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.479040 14823 raft_consensus.cc:2267] T 80aa570e2c154172ad6da29d268bede7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.479156 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4283bf9f53c54e7095e270fa04fc4f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.479458 14823 ts_tablet_manager.cc:1905] T 80aa570e2c154172ad6da29d268bede7 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.480726 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 4283bf9f53c54e7095e270fa04fc4f12 (DEFAULT_TABLE table=default.Table_543c011b1f2a4240be626d4454882dc8 [id=61210b6b042d4c04b32b6ff818232e1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.480988 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4283bf9f53c54e7095e270fa04fc4f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.481063 14654 tablet_service.cc:1467] Processing CreateTablet for tablet 30ed5d31dc08401da8853d3c2318d878 (DEFAULT_TABLE table=default.Table_c596014f7fe943d3bc6faa31f1a5f974 [id=e19abce4af3f485a9c2ccd0d35002301]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.481263 14654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30ed5d31dc08401da8853d3c2318d878. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.481644 14823 ts_tablet_manager.cc:1918] T 80aa570e2c154172ad6da29d268bede7 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.481698 14823 log.cc:1198] T 80aa570e2c154172ad6da29d268bede7 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/80aa570e2c154172ad6da29d268bede7
I20250115 03:06:19.481889 14823 ts_tablet_manager.cc:1939] T 80aa570e2c154172ad6da29d268bede7 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:19.482420 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 80aa570e2c154172ad6da29d268bede7 (table default.Table_24cbe6b962b248a9b340506b63c41b20 [id=9866275e65ab4dfcbfee6bf209c3654e]) successfully deleted
W20250115 03:06:19.482824 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 9c436975bd034068a6d58d5831db1896 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9c436975bd034068a6d58d5831db1896 already in progress: creating tablet
I20250115 03:06:19.483127 14655 tablet_service.cc:1467] Processing CreateTablet for tablet 91edfeb70a964909bf9049aa1c9fd8a5 (DEFAULT_TABLE table=default.Table_162fe16a9dfb414784a702bafb5d9938 [id=96ef5713dddc43eab4af43b8b6b60680]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.483248 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 75854bea6dec40ac9f205fec0ae66f39 (DEFAULT_TABLE table=default.Table_736f17451e344c1e9b3c1e0797cb2a6d [id=b4ff85ac73d3493da83cc2f88227e638]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.483336 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91edfeb70a964909bf9049aa1c9fd8a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.483523 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75854bea6dec40ac9f205fec0ae66f39. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:19.483515 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 3e170ced1612442ebcfdaff6d9826ea3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3e170ced1612442ebcfdaff6d9826ea3 already in progress: creating tablet
I20250115 03:06:19.483690 14823 tablet_replica.cc:331] T cc67fb818b8146629d92f9ac1944dba6 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:19.483805 14823 raft_consensus.cc:2238] T cc67fb818b8146629d92f9ac1944dba6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.483883 14823 raft_consensus.cc:2267] T cc67fb818b8146629d92f9ac1944dba6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.484257 14820 ts_tablet_manager.cc:1905] T d895f75911b7456a88e7a5159db7d60b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.485281 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 30ed5d31dc08401da8853d3c2318d878 (DEFAULT_TABLE table=default.Table_c596014f7fe943d3bc6faa31f1a5f974 [id=e19abce4af3f485a9c2ccd0d35002301]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.485487 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30ed5d31dc08401da8853d3c2318d878. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.486536 14820 ts_tablet_manager.cc:1918] T d895f75911b7456a88e7a5159db7d60b P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:19.486593 14820 log.cc:1198] T d895f75911b7456a88e7a5159db7d60b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/d895f75911b7456a88e7a5159db7d60b
I20250115 03:06:19.486807 14820 ts_tablet_manager.cc:1939] T d895f75911b7456a88e7a5159db7d60b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.490188 14809 tablet_bootstrap.cc:492] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.490283 14809 ts_tablet_manager.cc:1397] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.044s user 0.000s sys 0.002s
I20250115 03:06:19.490655 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 9fb51a41e62a4094879887626d199a7c (DEFAULT_TABLE table=default.Table_75817b46a6414888988ff7373ea11b27 [id=dfa93dac2ee14cfd8fe486869349c288]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.490737 14809 raft_consensus.cc:357] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.490855 14809 raft_consensus.cc:383] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.490895 14809 raft_consensus.cc:738] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.490896 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fb51a41e62a4094879887626d199a7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.491016 14809 consensus_queue.cc:260] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.491307 14809 ts_tablet_manager.cc:1428] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:19.491480 14809 tablet_bootstrap.cc:492] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.491775 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 6618580ae8be4cb7b503d8f9cc5322dc (DEFAULT_TABLE table=default.Table_c9451afa33814f508ab562805101afc8 [id=ff98684fefe64335b07be43e255d3e35]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.491978 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6618580ae8be4cb7b503d8f9cc5322dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.492311 14809 tablet_bootstrap.cc:654] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.492885 14518 tablet_service.cc:1467] Processing CreateTablet for tablet 396329e33a40477d9c346db57454bc04 (DEFAULT_TABLE table=default.Table_bfc2bf75532d47a091aeb698f89c36dd [id=6ab0030c03c04d049e912404706df7ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.493106 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 396329e33a40477d9c346db57454bc04. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.493885 14534 tablet_service.cc:1467] Processing CreateTablet for tablet 7e23caf72949411e8e401c37c8b29345 (DEFAULT_TABLE table=default.Table_9342b800c44944b1bf0997bc3262ab94 [id=39677fe8c4024c23bfd8b15250f1b4e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.494092 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e23caf72949411e8e401c37c8b29345. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.495211 14809 tablet_bootstrap.cc:492] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.495281 14809 ts_tablet_manager.cc:1397] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:19.495628 14526 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c436975bd034068a6d58d5831db1896 with delete_type TABLET_DATA_DELETED (replaced by tablet 802a007b8d154198b291aaf9530161b9) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.495746 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e170ced1612442ebcfdaff6d9826ea3 with delete_type TABLET_DATA_DELETED (replaced by tablet 8e4410ce33f0464eadead04e8116f773) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.495851 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet d140be0fab694f59b8a535392970fad0 with delete_type TABLET_DATA_DELETED (replaced by tablet eb787122e6674201bf62d6bc787e8c20) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.495911 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet 3007a0506ce14c6c89c600b98a81b168 with delete_type TABLET_DATA_DELETED (replaced by tablet 1fd348b3ae2a45dfa05c52f02536a201) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.495694 14809 raft_consensus.cc:357] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.495980 15401 raft_consensus.cc:491] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.496002 14809 raft_consensus.cc:383] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.496045 15401 raft_consensus.cc:513] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.496347 15617 tablet.cc:1721] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:19.496415 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet aa7754f577af4ef6b6fca41e61e5965c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.496462 14812 tablet_bootstrap.cc:492] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:19.496551 14812 ts_tablet_manager.cc:1397] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s
I20250115 03:06:19.496670 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 75854bea6dec40ac9f205fec0ae66f39 (DEFAULT_TABLE table=default.Table_736f17451e344c1e9b3c1e0797cb2a6d [id=b4ff85ac73d3493da83cc2f88227e638]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.496865 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75854bea6dec40ac9f205fec0ae66f39. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.496989 14812 raft_consensus.cc:357] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.497076 14812 raft_consensus.cc:383] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.497102 14812 raft_consensus.cc:738] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.497201 14812 consensus_queue.cc:260] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.497442 14812 ts_tablet_manager.cc:1428] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.498555 14812 tablet_bootstrap.cc:492] T ca2093980c8e480e99ae0cd79606f318 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:19.500140 14812 tablet_bootstrap.cc:654] T ca2093980c8e480e99ae0cd79606f318 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.500108 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15fbf8d5de904fc4b284af664f3bd60f" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:19.500727 14356 leader_election.cc:343] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.501168 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15fbf8d5de904fc4b284af664f3bd60f" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:19.496356 15401 leader_election.cc:290] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:19.501479 14358 leader_election.cc:343] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.501541 14358 leader_election.cc:304] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:19.501758 15401 raft_consensus.cc:2743] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:19.503037 15426 tablet.cc:1721] T 13f36a563f13403eab372deb10b67451 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:19.505204 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 3dc5eeb98e59478caa3130418d61a1a8 (DEFAULT_TABLE table=default.Table_cf48da676b2a490a8903b95bfab1dbe9 [id=94199a4dea724fc6803af6b5149952b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.505540 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dc5eeb98e59478caa3130418d61a1a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.508459 14655 tablet_service.cc:1467] Processing CreateTablet for tablet 802a007b8d154198b291aaf9530161b9 (DEFAULT_TABLE table=default.Table_6e2e63a19a64494884fc71e4b3364978 [id=522c89cadb084de68368a97199ea66db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.496045 14809 raft_consensus.cc:738] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.508702 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 802a007b8d154198b291aaf9530161b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.508734 14809 consensus_queue.cc:260] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.509918 14809 ts_tablet_manager.cc:1428] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.015s user 0.001s sys 0.000s
I20250115 03:06:19.510067 14809 tablet_bootstrap.cc:492] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.510668 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 802a007b8d154198b291aaf9530161b9 (DEFAULT_TABLE table=default.Table_6e2e63a19a64494884fc71e4b3364978 [id=522c89cadb084de68368a97199ea66db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.510876 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 802a007b8d154198b291aaf9530161b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.510924 14809 tablet_bootstrap.cc:654] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.512806 14654 tablet_service.cc:1467] Processing CreateTablet for tablet eb787122e6674201bf62d6bc787e8c20 (DEFAULT_TABLE table=default.Table_6df09ae8a979496abf73d384649b81b5 [id=e30f9e0f1aef45b7a1668a6d0f5f42bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.513041 14654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb787122e6674201bf62d6bc787e8c20. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.514887 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 8e4410ce33f0464eadead04e8116f773 (DEFAULT_TABLE table=default.Table_77d4ac27a2c44250a2dcc62121f202c7 [id=6af8f6f562ac47b484eeb4d67ac6c7df]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.515071 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e4410ce33f0464eadead04e8116f773. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.515463 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 8e4410ce33f0464eadead04e8116f773 (DEFAULT_TABLE table=default.Table_77d4ac27a2c44250a2dcc62121f202c7 [id=6af8f6f562ac47b484eeb4d67ac6c7df]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.515683 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e4410ce33f0464eadead04e8116f773. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.518009 14396 tablet_service.cc:1467] Processing CreateTablet for tablet eb787122e6674201bf62d6bc787e8c20 (DEFAULT_TABLE table=default.Table_6df09ae8a979496abf73d384649b81b5 [id=e30f9e0f1aef45b7a1668a6d0f5f42bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.518224 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb787122e6674201bf62d6bc787e8c20. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.518301 14164 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 15fbf8d5de904fc4b284af664f3bd60f (table default.Table_655e9d02aacf4b88ba521f0f57a5f522 [id=6ed01835736b40f8b4caeca7378770d8]). Expected version 1 got 0
I20250115 03:06:19.519065 14811 tablet_bootstrap.cc:492] T 8b0f0539530643f78207e528c39530af P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.519137 14811 ts_tablet_manager.cc:1397] T 8b0f0539530643f78207e528c39530af P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.048s user 0.001s sys 0.000s
I20250115 03:06:19.519156 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 91edfeb70a964909bf9049aa1c9fd8a5 (DEFAULT_TABLE table=default.Table_162fe16a9dfb414784a702bafb5d9938 [id=96ef5713dddc43eab4af43b8b6b60680]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.519361 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91edfeb70a964909bf9049aa1c9fd8a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.519542 14811 raft_consensus.cc:357] T 8b0f0539530643f78207e528c39530af P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:19.519635 14811 raft_consensus.cc:383] T 8b0f0539530643f78207e528c39530af P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.519665 14811 raft_consensus.cc:738] T 8b0f0539530643f78207e528c39530af P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.519768 14811 consensus_queue.cc:260] T 8b0f0539530643f78207e528c39530af P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:19.521607 14811 ts_tablet_manager.cc:1428] T 8b0f0539530643f78207e528c39530af P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:19.521813 14811 tablet_bootstrap.cc:492] T 7c503b29cb2843338fb1870a89aee443 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.521872 14820 tablet_replica.cc:331] T 3007a0506ce14c6c89c600b98a81b168 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.521977 14820 raft_consensus.cc:2238] T 3007a0506ce14c6c89c600b98a81b168 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.522068 14820 raft_consensus.cc:2267] T 3007a0506ce14c6c89c600b98a81b168 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.522630 14809 tablet_bootstrap.cc:492] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.522706 14809 ts_tablet_manager.cc:1397] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20250115 03:06:19.522697 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 4283bf9f53c54e7095e270fa04fc4f12 (DEFAULT_TABLE table=default.Table_543c011b1f2a4240be626d4454882dc8 [id=61210b6b042d4c04b32b6ff818232e1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.522853 14164 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_a2c05ae2d81b4dd190ad94b5212e3bf2 [id=61f3625fed2b4a70b5331d6c6ac73f1c] alter complete (version 2)
I20250115 03:06:19.522919 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4283bf9f53c54e7095e270fa04fc4f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.523099 14809 raft_consensus.cc:357] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.523193 14809 raft_consensus.cc:383] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.523218 14809 raft_consensus.cc:738] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.524236 14811 tablet_bootstrap.cc:654] T 7c503b29cb2843338fb1870a89aee443 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.523344 14809 consensus_queue.cc:260] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.525710 14166 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet b56279bb16054869a97006d88574e381 (table default.Table_23165257844448ce93a97233b487ddc7 [id=080d0a4dfa174d3ba2979bb850d0604a]). Expected version 1 got 0
I20250115 03:06:19.526249 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 30ed5d31dc08401da8853d3c2318d878 (DEFAULT_TABLE table=default.Table_c596014f7fe943d3bc6faa31f1a5f974 [id=e19abce4af3f485a9c2ccd0d35002301]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.526454 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30ed5d31dc08401da8853d3c2318d878. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.527729 14809 ts_tablet_manager.cc:1428] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:19.527891 14809 tablet_bootstrap.cc:492] T 312dcdda8cf042648db9fdf6228062d9 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.528837 14166 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 6bcfde5fe3d4425e98edb4dc54096718 (table default.Table_a63d6344914343c4af56eef21f467c6f [id=f45053c781fc460ab55f592eed6fb946]). Expected version 1 got 0
I20250115 03:06:19.530941 14809 tablet_bootstrap.cc:654] T 312dcdda8cf042648db9fdf6228062d9 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.532588 14166 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 6bcfde5fe3d4425e98edb4dc54096718 (table default.Table_a63d6344914343c4af56eef21f467c6f [id=f45053c781fc460ab55f592eed6fb946]). Expected version 1 got 0
I20250115 03:06:19.533697 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 802a007b8d154198b291aaf9530161b9 (DEFAULT_TABLE table=default.Table_6e2e63a19a64494884fc71e4b3364978 [id=522c89cadb084de68368a97199ea66db]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.533720 15426 tablet.cc:1721] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:19.534366 14823 ts_tablet_manager.cc:1905] T cc67fb818b8146629d92f9ac1944dba6 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.534338 15970 tablet.cc:1721] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:19.534727 15617 tablet.cc:1721] T 0580a8f7b2984162b9275d74334b1905 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:19.538610 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 802a007b8d154198b291aaf9530161b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.539665 14823 ts_tablet_manager.cc:1918] T cc67fb818b8146629d92f9ac1944dba6 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.539793 14823 log.cc:1198] T cc67fb818b8146629d92f9ac1944dba6 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/cc67fb818b8146629d92f9ac1944dba6
I20250115 03:06:19.540030 14823 ts_tablet_manager.cc:1939] T cc67fb818b8146629d92f9ac1944dba6 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
W20250115 03:06:19.540433 14356 consensus_peers.cc:487] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:19.540676 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet cc67fb818b8146629d92f9ac1944dba6 (table default.Table_3fab7f59cc324dcf811b852f62ba1f9e [id=0a2e80b35ce148ff85d040843c7a1a2d]) successfully deleted
W20250115 03:06:19.540836 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet d140be0fab694f59b8a535392970fad0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d140be0fab694f59b8a535392970fad0 already in progress: creating tablet
I20250115 03:06:19.542939 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_def871c9b5dc4a4b8fdc40fe12ccaf7a [id=0790b08615854d3688f940aba20b6520] alter complete (version 4)
I20250115 03:06:19.545300 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet d895f75911b7456a88e7a5159db7d60b (table default.Table_6bb3ca6745554454a044c329e6b7e8d3 [id=4b6ce1381c024498a78d91e346f488f2]) successfully deleted
W20250115 03:06:19.545433 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet d6cf1a0b9c81413b8f72de0cafc76963 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d6cf1a0b9c81413b8f72de0cafc76963 already in progress: creating tablet
I20250115 03:06:19.545449 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 8e4410ce33f0464eadead04e8116f773 (DEFAULT_TABLE table=default.Table_77d4ac27a2c44250a2dcc62121f202c7 [id=6af8f6f562ac47b484eeb4d67ac6c7df]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:19.545545 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 0e1ce43146ed4e4d82a7c134216e2872 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e1ce43146ed4e4d82a7c134216e2872 already in progress: creating tablet
W20250115 03:06:19.545640 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet f72c9bf04a3942ceacc37fd407bff033 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f72c9bf04a3942ceacc37fd407bff033 already in progress: creating tablet
I20250115 03:06:19.545650 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e4410ce33f0464eadead04e8116f773. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:19.545728 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 074fdc09e4e24078ad0bb588b32ff24f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 074fdc09e4e24078ad0bb588b32ff24f already in progress: creating tablet
W20250115 03:06:19.545825 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet dcb99303cec64a889c3057554f90561e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dcb99303cec64a889c3057554f90561e already in progress: creating tablet
W20250115 03:06:19.545934 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 98913be4c8e74d7185957b0c4ad6abce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 98913be4c8e74d7185957b0c4ad6abce already in progress: creating tablet
W20250115 03:06:19.546041 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet e5b483637c08401d92c1ab63cfe6f3c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5b483637c08401d92c1ab63cfe6f3c6 already in progress: creating tablet
W20250115 03:06:19.546149 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 60997f625d074960ad75aef5ba14b9c2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60997f625d074960ad75aef5ba14b9c2 already in progress: creating tablet
I20250115 03:06:19.546221 14820 ts_tablet_manager.cc:1905] T 3007a0506ce14c6c89c600b98a81b168 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:19.546245 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 980b69b60a354519b0691ecb0bbd6c70 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 980b69b60a354519b0691ecb0bbd6c70 already in progress: creating tablet
W20250115 03:06:19.546335 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 015ec42caa1b468ea10f25c8f26e278a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 015ec42caa1b468ea10f25c8f26e278a already in progress: creating tablet
W20250115 03:06:19.546738 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 9c436975bd034068a6d58d5831db1896 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9c436975bd034068a6d58d5831db1896 already in progress: creating tablet
W20250115 03:06:19.546857 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 3e170ced1612442ebcfdaff6d9826ea3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3e170ced1612442ebcfdaff6d9826ea3 already in progress: creating tablet
W20250115 03:06:19.546955 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet d140be0fab694f59b8a535392970fad0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d140be0fab694f59b8a535392970fad0 already in progress: creating tablet
I20250115 03:06:19.547436 14530 tablet_service.cc:1467] Processing CreateTablet for tablet eb787122e6674201bf62d6bc787e8c20 (DEFAULT_TABLE table=default.Table_6df09ae8a979496abf73d384649b81b5 [id=e30f9e0f1aef45b7a1668a6d0f5f42bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.547631 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb787122e6674201bf62d6bc787e8c20. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.548277 14820 ts_tablet_manager.cc:1918] T 3007a0506ce14c6c89c600b98a81b168 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.548394 14820 log.cc:1198] T 3007a0506ce14c6c89c600b98a81b168 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/3007a0506ce14c6c89c600b98a81b168
I20250115 03:06:19.548642 14820 ts_tablet_manager.cc:1939] T 3007a0506ce14c6c89c600b98a81b168 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.549309 14820 tablet_replica.cc:331] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.549471 14820 raft_consensus.cc:2238] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.549628 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 3007a0506ce14c6c89c600b98a81b168 (table default.Table_cf7f3fbd11804e94848a7c5357169a86 [id=d64515775f264a419768662b8ce0a854]) successfully deleted
I20250115 03:06:19.550109 15970 tablet.cc:1721] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:19.553874 14820 raft_consensus.cc:2267] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.553925 14811 tablet_bootstrap.cc:492] T 7c503b29cb2843338fb1870a89aee443 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.553988 14811 ts_tablet_manager.cc:1397] T 7c503b29cb2843338fb1870a89aee443 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.000s
I20250115 03:06:19.554388 14811 raft_consensus.cc:357] T 7c503b29cb2843338fb1870a89aee443 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.554486 14811 raft_consensus.cc:383] T 7c503b29cb2843338fb1870a89aee443 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.554529 14811 raft_consensus.cc:738] T 7c503b29cb2843338fb1870a89aee443 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.554628 14811 consensus_queue.cc:260] T 7c503b29cb2843338fb1870a89aee443 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.554857 14811 ts_tablet_manager.cc:1428] T 7c503b29cb2843338fb1870a89aee443 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:19.556391 14812 tablet_bootstrap.cc:492] T ca2093980c8e480e99ae0cd79606f318 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:19.556473 14812 ts_tablet_manager.cc:1397] T ca2093980c8e480e99ae0cd79606f318 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.058s user 0.001s sys 0.000s
I20250115 03:06:19.557035 14812 raft_consensus.cc:357] T ca2093980c8e480e99ae0cd79606f318 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.557135 14812 raft_consensus.cc:383] T ca2093980c8e480e99ae0cd79606f318 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.557165 14812 raft_consensus.cc:738] T ca2093980c8e480e99ae0cd79606f318 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.557271 14812 consensus_queue.cc:260] T ca2093980c8e480e99ae0cd79606f318 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.557538 14812 ts_tablet_manager.cc:1428] T ca2093980c8e480e99ae0cd79606f318 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.557624 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 8b0f0539530643f78207e528c39530af (table default.Table_d7d37f44d845457e8a3656b4f7deabf2 [id=fe357a2a64fc4531870b305eb2480df1]). Expected version 2 got 0
I20250115 03:06:19.557693 14812 tablet_bootstrap.cc:492] T 62ec9057d85447bdb6b86cdc1f9015ac P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:19.558382 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_34987b0302b64565abe50420f8c86a38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:19.558559 14812 tablet_bootstrap.cc:654] T 62ec9057d85447bdb6b86cdc1f9015ac P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:19.558706 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_34987b0302b64565abe50420f8c86a38 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.559651 14820 ts_tablet_manager.cc:1905] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.563725 14809 tablet_bootstrap.cc:492] T 312dcdda8cf042648db9fdf6228062d9 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.563827 14809 ts_tablet_manager.cc:1397] T 312dcdda8cf042648db9fdf6228062d9 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.036s user 0.001s sys 0.000s
I20250115 03:06:19.564257 14809 raft_consensus.cc:357] T 312dcdda8cf042648db9fdf6228062d9 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.564365 14809 raft_consensus.cc:383] T 312dcdda8cf042648db9fdf6228062d9 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.564409 14809 raft_consensus.cc:738] T 312dcdda8cf042648db9fdf6228062d9 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.564339 14820 ts_tablet_manager.cc:1918] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.564532 14820 log.cc:1198] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/aa7754f577af4ef6b6fca41e61e5965c
I20250115 03:06:19.564555 14809 consensus_queue.cc:260] T 312dcdda8cf042648db9fdf6228062d9 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.564744 14820 ts_tablet_manager.cc:1939] T aa7754f577af4ef6b6fca41e61e5965c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.564862 14809 ts_tablet_manager.cc:1428] T 312dcdda8cf042648db9fdf6228062d9 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.566674 14654 tablet_service.cc:1514] Processing DeleteTablet for tablet ca2093980c8e480e99ae0cd79606f318 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.567471 14823 tablet_replica.cc:331] T ca2093980c8e480e99ae0cd79606f318 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:19.567652 14823 raft_consensus.cc:2238] T ca2093980c8e480e99ae0cd79606f318 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.567804 14823 raft_consensus.cc:2267] T ca2093980c8e480e99ae0cd79606f318 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.568266 14823 ts_tablet_manager.cc:1905] T ca2093980c8e480e99ae0cd79606f318 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.569684 14811 tablet_bootstrap.cc:492] T d2220e7ec26b4708b2c7a2540c053dcf P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.570369 14809 tablet_bootstrap.cc:492] T 98913be4c8e74d7185957b0c4ad6abce P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.571740 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet aa7754f577af4ef6b6fca41e61e5965c (table default.Table_b175cba7965344979476c3cb6d789eca [id=8ca6a98158114c7bb6bf49940d846cc6]) successfully deleted
I20250115 03:06:19.572125 14809 tablet_bootstrap.cc:654] T 98913be4c8e74d7185957b0c4ad6abce P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.572093 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a2c05ae2d81b4dd190ad94b5212e3bf2" } modify_external_catalogs: true
I20250115 03:06:19.572189 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a2c05ae2d81b4dd190ad94b5212e3bf2" } modify_external_catalogs: true
I20250115 03:06:19.572393 14823 ts_tablet_manager.cc:1918] T ca2093980c8e480e99ae0cd79606f318 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.572508 14823 log.cc:1198] T ca2093980c8e480e99ae0cd79606f318 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/ca2093980c8e480e99ae0cd79606f318
I20250115 03:06:19.572613 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 7c503b29cb2843338fb1870a89aee443 (table default.Table_dfcccd3390fa4814a0a04bb1ff5e43fa [id=f0a7068682b9481a82fa34cb54ed528e]). Expected version 1 got 0
I20250115 03:06:19.571177 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 312dcdda8cf042648db9fdf6228062d9 with delete_type TABLET_DATA_DELETED (replaced by tablet 7b86e66428504217af08f667f660d304) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.572813 14823 ts_tablet_manager.cc:1939] T ca2093980c8e480e99ae0cd79606f318 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:19.572856 14819 tablet_replica.cc:331] T 312dcdda8cf042648db9fdf6228062d9 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.572988 14819 raft_consensus.cc:2238] T 312dcdda8cf042648db9fdf6228062d9 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.573071 14819 raft_consensus.cc:2267] T 312dcdda8cf042648db9fdf6228062d9 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.573477 14819 ts_tablet_manager.cc:1905] T 312dcdda8cf042648db9fdf6228062d9 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.574755 14811 tablet_bootstrap.cc:654] T d2220e7ec26b4708b2c7a2540c053dcf P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.574983 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet ca2093980c8e480e99ae0cd79606f318 (table default.Table_0aecd3e601c04f0497cefafda025e6e1 [id=bde9696f4c534fe58bb1de4fd7e00201]) successfully deleted
I20250115 03:06:19.575469 14819 ts_tablet_manager.cc:1918] T 312dcdda8cf042648db9fdf6228062d9 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.575525 14819 log.cc:1198] T 312dcdda8cf042648db9fdf6228062d9 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/312dcdda8cf042648db9fdf6228062d9
I20250115 03:06:19.575541 14812 tablet_bootstrap.cc:492] T 62ec9057d85447bdb6b86cdc1f9015ac P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:19.575620 14812 ts_tablet_manager.cc:1397] T 62ec9057d85447bdb6b86cdc1f9015ac P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20250115 03:06:19.575713 14819 ts_tablet_manager.cc:1939] T 312dcdda8cf042648db9fdf6228062d9 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.576052 14812 raft_consensus.cc:357] T 62ec9057d85447bdb6b86cdc1f9015ac P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.576144 14812 raft_consensus.cc:383] T 62ec9057d85447bdb6b86cdc1f9015ac P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.576176 14812 raft_consensus.cc:738] T 62ec9057d85447bdb6b86cdc1f9015ac P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.576555 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 312dcdda8cf042648db9fdf6228062d9 (table default.Table_341a3cd1c8c94ff4b05fc21960a4e9b4 [id=c096ce170874405ea740b9ea829f886e]) successfully deleted
I20250115 03:06:19.576300 14812 consensus_queue.cc:260] T 62ec9057d85447bdb6b86cdc1f9015ac P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.576800 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 13f36a563f13403eab372deb10b67451
I20250115 03:06:19.578603 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 13f36a563f13403eab372deb10b67451 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.578610 14858 raft_consensus.cc:491] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.578689 14858 raft_consensus.cc:513] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:19.579010 14858 leader_election.cc:290] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:19.579093 14812 ts_tablet_manager.cc:1428] T 62ec9057d85447bdb6b86cdc1f9015ac P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
W20250115 03:06:19.579739 14493 leader_election.cc:343] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 5d27310f8433411a85bf88d05a81bcf0
W20250115 03:06:19.579952 14493 leader_election.cc:343] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 5d27310f8433411a85bf88d05a81bcf0
I20250115 03:06:19.580003 14493 leader_election.cc:304] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:19.580080 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet 13f36a563f13403eab372deb10b67451 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.580174 14812 tablet_bootstrap.cc:492] T e2c6db404348466ea5211ffa69c8f01c P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:19.580199 14654 tablet_service.cc:1514] Processing DeleteTablet for tablet 13f36a563f13403eab372deb10b67451 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.581055 14812 tablet_bootstrap.cc:654] T e2c6db404348466ea5211ffa69c8f01c P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.581085 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_47fd03c657214fea893d4881cbbff5f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.581423 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_47fd03c657214fea893d4881cbbff5f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.581190 14823 tablet_replica.cc:331] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:19.581786 14823 raft_consensus.cc:2238] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.581892 14823 raft_consensus.cc:2267] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.582329 14823 ts_tablet_manager.cc:1905] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.583451 14820 tablet_replica.cc:331] T 13f36a563f13403eab372deb10b67451 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.583707 14819 tablet_replica.cc:331] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.583825 14819 raft_consensus.cc:2238] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:19.583992 14819 raft_consensus.cc:2267] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.584331 14820 raft_consensus.cc:2238] T 13f36a563f13403eab372deb10b67451 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.584461 14819 ts_tablet_manager.cc:1905] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.584501 14820 raft_consensus.cc:2267] T 13f36a563f13403eab372deb10b67451 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.585101 14820 ts_tablet_manager.cc:1905] T 13f36a563f13403eab372deb10b67451 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.585537 14823 ts_tablet_manager.cc:1918] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:19.586383 14819 ts_tablet_manager.cc:1918] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:19.586505 14819 log.cc:1198] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/13f36a563f13403eab372deb10b67451
I20250115 03:06:19.586571 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_9fffa7d4b0834e3e85a07968b118ac3a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:19.586786 14819 ts_tablet_manager.cc:1939] T 13f36a563f13403eab372deb10b67451 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
W20250115 03:06:19.586884 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9fffa7d4b0834e3e85a07968b118ac3a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.587347 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 13f36a563f13403eab372deb10b67451 (table default.Table_a2c05ae2d81b4dd190ad94b5212e3bf2 [id=61f3625fed2b4a70b5331d6c6ac73f1c]) successfully deleted
I20250115 03:06:19.587498 14823 log.cc:1198] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/13f36a563f13403eab372deb10b67451
I20250115 03:06:19.587724 14823 ts_tablet_manager.cc:1939] T 13f36a563f13403eab372deb10b67451 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:19.588337 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 13f36a563f13403eab372deb10b67451 (table default.Table_a2c05ae2d81b4dd190ad94b5212e3bf2 [id=61f3625fed2b4a70b5331d6c6ac73f1c]) successfully deleted
I20250115 03:06:19.589330 14654 tablet_service.cc:1514] Processing DeleteTablet for tablet 62ec9057d85447bdb6b86cdc1f9015ac with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.589798 14858 raft_consensus.cc:2743] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:19.592979 14823 tablet_replica.cc:331] T 62ec9057d85447bdb6b86cdc1f9015ac P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:19.593108 14823 raft_consensus.cc:2238] T 62ec9057d85447bdb6b86cdc1f9015ac P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.593210 14823 raft_consensus.cc:2267] T 62ec9057d85447bdb6b86cdc1f9015ac P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.594125 14820 ts_tablet_manager.cc:1918] T 13f36a563f13403eab372deb10b67451 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:19.594189 14820 log.cc:1198] T 13f36a563f13403eab372deb10b67451 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/13f36a563f13403eab372deb10b67451
I20250115 03:06:19.594358 14823 ts_tablet_manager.cc:1905] T 62ec9057d85447bdb6b86cdc1f9015ac P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.595961 16001 raft_consensus.cc:491] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.596035 16001 raft_consensus.cc:513] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.596336 16001 leader_election.cc:290] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:19.596653 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5655935aabd48a19d0ef03739e089a4" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:19.596897 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5655935aabd48a19d0ef03739e089a4" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:19.597030 14422 raft_consensus.cc:2463] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
W20250115 03:06:19.597270 14626 leader_election.cc:343] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.596664 14820 ts_tablet_manager.cc:1939] T 13f36a563f13403eab372deb10b67451 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.597373 14628 leader_election.cc:304] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:19.597501 16001 raft_consensus.cc:2798] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:19.597551 16001 raft_consensus.cc:491] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:19.597595 16001 raft_consensus.cc:3054] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:19.598315 16001 raft_consensus.cc:513] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.598672 16001 leader_election.cc:290] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:19.598881 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5655935aabd48a19d0ef03739e089a4" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:19.598978 14422 raft_consensus.cc:3054] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:19.598986 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0c02079a9e2a4cb2ac71a7f8ea1b2cc9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.599279 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c02079a9e2a4cb2ac71a7f8ea1b2cc9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.599574 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 13f36a563f13403eab372deb10b67451 (table default.Table_a2c05ae2d81b4dd190ad94b5212e3bf2 [id=61f3625fed2b4a70b5331d6c6ac73f1c]) successfully deleted
I20250115 03:06:19.599772 14422 raft_consensus.cc:2463] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:19.599965 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5655935aabd48a19d0ef03739e089a4" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:19.600041 14628 leader_election.cc:304] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:19.600178 16001 raft_consensus.cc:2798] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:19.600257 16001 raft_consensus.cc:695] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:19.600365 16001 consensus_queue.cc:237] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.602645 14812 tablet_bootstrap.cc:492] T e2c6db404348466ea5211ffa69c8f01c P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:19.602730 14812 ts_tablet_manager.cc:1397] T e2c6db404348466ea5211ffa69c8f01c P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s
W20250115 03:06:19.602808 14626 leader_election.cc:343] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.603156 14812 raft_consensus.cc:357] T e2c6db404348466ea5211ffa69c8f01c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.603240 14812 raft_consensus.cc:383] T e2c6db404348466ea5211ffa69c8f01c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.603269 14812 raft_consensus.cc:738] T e2c6db404348466ea5211ffa69c8f01c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.603367 14812 consensus_queue.cc:260] T e2c6db404348466ea5211ffa69c8f01c P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.604247 14812 ts_tablet_manager.cc:1428] T e2c6db404348466ea5211ffa69c8f01c P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.605654 14812 tablet_bootstrap.cc:492] T da95daa40c5b4615a0219f6d8e3770f6 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:19.606675 14812 tablet_bootstrap.cc:654] T da95daa40c5b4615a0219f6d8e3770f6 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.607014 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_bfdb42e52734403c9ba4b455ea0e6e44"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.607389 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bfdb42e52734403c9ba4b455ea0e6e44 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.608153 14823 ts_tablet_manager.cc:1918] T 62ec9057d85447bdb6b86cdc1f9015ac P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.608206 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_fe5a8443c8e54cb6a0f436c90a2b5b91"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:19.608304 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b8b40ffaa9e04cf481788f0d2e8519e9" } modify_external_catalogs: true
I20250115 03:06:19.608363 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_48f283b8ffcc4e5c8416b91ca6038d23" } modify_external_catalogs: true
I20250115 03:06:19.608394 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b8b40ffaa9e04cf481788f0d2e8519e9" } modify_external_catalogs: true
I20250115 03:06:19.608424 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_48f283b8ffcc4e5c8416b91ca6038d23" } modify_external_catalogs: true
W20250115 03:06:19.608484 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe5a8443c8e54cb6a0f436c90a2b5b91 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.608662 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_511db1d50b7246e9b6dc2d6ed50da4d0" } modify_external_catalogs: true
I20250115 03:06:19.608722 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_511db1d50b7246e9b6dc2d6ed50da4d0" } modify_external_catalogs: true
I20250115 03:06:19.553385 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d27310f8433411a85bf88d05a81bcf0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.553422 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 30499e99ab7d4138a376dea40b768c32 with delete_type TABLET_DATA_DELETED (replaced by tablet 80ede4e9696d491aa385b375fcb5cf3b) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.609135 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f7de4dae6ab44600a02d5b8f92eafedd" } modify_external_catalogs: true
I20250115 03:06:19.609205 14820 tablet_replica.cc:331] T 30499e99ab7d4138a376dea40b768c32 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.609225 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f7de4dae6ab44600a02d5b8f92eafedd" } modify_external_catalogs: true
I20250115 03:06:19.609320 14820 raft_consensus.cc:2238] T 30499e99ab7d4138a376dea40b768c32 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.609407 14820 raft_consensus.cc:2267] T 30499e99ab7d4138a376dea40b768c32 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.610229 14820 ts_tablet_manager.cc:1905] T 30499e99ab7d4138a376dea40b768c32 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.611375 14823 log.cc:1198] T 62ec9057d85447bdb6b86cdc1f9015ac P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/62ec9057d85447bdb6b86cdc1f9015ac
I20250115 03:06:19.611570 14823 ts_tablet_manager.cc:1939] T 62ec9057d85447bdb6b86cdc1f9015ac P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:19.612067 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 62ec9057d85447bdb6b86cdc1f9015ac (table default.Table_5a68d6c8c53b4746a7123860185fe0cc [id=84c18cf488cc4c28aaf93fa72334acb7]) successfully deleted
I20250115 03:06:19.613420 14820 ts_tablet_manager.cc:1918] T 30499e99ab7d4138a376dea40b768c32 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.613483 14820 log.cc:1198] T 30499e99ab7d4138a376dea40b768c32 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/30499e99ab7d4138a376dea40b768c32
I20250115 03:06:19.613638 14820 ts_tablet_manager.cc:1939] T 30499e99ab7d4138a376dea40b768c32 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.614075 14820 tablet_replica.cc:331] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.614153 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 30499e99ab7d4138a376dea40b768c32 (table default.Table_75cd7b01fc534652975eb593912b88bc [id=111fc67094164d3f99965c246ae12b24]) successfully deleted
I20250115 03:06:19.614825 14811 tablet_bootstrap.cc:492] T d2220e7ec26b4708b2c7a2540c053dcf P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.617070 14811 ts_tablet_manager.cc:1397] T d2220e7ec26b4708b2c7a2540c053dcf P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.047s user 0.001s sys 0.000s
I20250115 03:06:19.618010 14811 raft_consensus.cc:357] T d2220e7ec26b4708b2c7a2540c053dcf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.618396 14811 raft_consensus.cc:383] T d2220e7ec26b4708b2c7a2540c053dcf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.618450 14811 raft_consensus.cc:738] T d2220e7ec26b4708b2c7a2540c053dcf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.619731 14811 consensus_queue.cc:260] T d2220e7ec26b4708b2c7a2540c053dcf P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.620167 14811 ts_tablet_manager.cc:1428] T d2220e7ec26b4708b2c7a2540c053dcf P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:19.620452 14820 raft_consensus.cc:2238] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.620555 14820 raft_consensus.cc:2267] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.621116 14809 tablet_bootstrap.cc:492] T 98913be4c8e74d7185957b0c4ad6abce P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.621202 14809 ts_tablet_manager.cc:1397] T 98913be4c8e74d7185957b0c4ad6abce P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.052s user 0.001s sys 0.000s
I20250115 03:06:19.621632 14809 raft_consensus.cc:357] T 98913be4c8e74d7185957b0c4ad6abce P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.622570 14809 raft_consensus.cc:383] T 98913be4c8e74d7185957b0c4ad6abce P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.622623 14809 raft_consensus.cc:738] T 98913be4c8e74d7185957b0c4ad6abce P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.622766 14809 consensus_queue.cc:260] T 98913be4c8e74d7185957b0c4ad6abce P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.624784 14811 tablet_bootstrap.cc:492] T 074fdc09e4e24078ad0bb588b32ff24f P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.625735 14811 tablet_bootstrap.cc:654] T 074fdc09e4e24078ad0bb588b32ff24f P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.626541 14809 ts_tablet_manager.cc:1428] T 98913be4c8e74d7185957b0c4ad6abce P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:19.627537 14809 tablet_bootstrap.cc:492] T 623887ec17374f598f48054e17582f3f P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.628365 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 98913be4c8e74d7185957b0c4ad6abce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.628509 14819 tablet_replica.cc:331] T 98913be4c8e74d7185957b0c4ad6abce P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.625908 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet d2220e7ec26b4708b2c7a2540c053dcf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.629606 14819 raft_consensus.cc:2238] T 98913be4c8e74d7185957b0c4ad6abce P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.630005 14819 raft_consensus.cc:2267] T 98913be4c8e74d7185957b0c4ad6abce P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.630388 14819 ts_tablet_manager.cc:1905] T 98913be4c8e74d7185957b0c4ad6abce P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.630561 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6fadef5b204546918bd04347978ae2c2" } new_table_name: "default.Table_60d8dde0d2c04c069ae9502e04496ffa" modify_external_catalogs: true
I20250115 03:06:19.630754 14812 tablet_bootstrap.cc:492] T da95daa40c5b4615a0219f6d8e3770f6 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:19.630842 14812 ts_tablet_manager.cc:1397] T da95daa40c5b4615a0219f6d8e3770f6 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20250115 03:06:19.630841 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_9e4202d4703b406e8c5255ab8a0cc41f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.631119 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9e4202d4703b406e8c5255ab8a0cc41f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.631270 14812 raft_consensus.cc:357] T da95daa40c5b4615a0219f6d8e3770f6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.631400 14812 raft_consensus.cc:383] T da95daa40c5b4615a0219f6d8e3770f6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.631433 14812 raft_consensus.cc:738] T da95daa40c5b4615a0219f6d8e3770f6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.631536 14812 consensus_queue.cc:260] T da95daa40c5b4615a0219f6d8e3770f6 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.631807 14812 ts_tablet_manager.cc:1428] T da95daa40c5b4615a0219f6d8e3770f6 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.631975 14812 tablet_bootstrap.cc:492] T c251960c7c9a44429102a294f35f925c P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:19.632820 14812 tablet_bootstrap.cc:654] T c251960c7c9a44429102a294f35f925c P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.634927 14166 catalog_manager.cc:5526] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:19.635524 14819 ts_tablet_manager.cc:1918] T 98913be4c8e74d7185957b0c4ad6abce P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.635597 14819 log.cc:1198] T 98913be4c8e74d7185957b0c4ad6abce P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/98913be4c8e74d7185957b0c4ad6abce
I20250115 03:06:19.635780 14819 ts_tablet_manager.cc:1939] T 98913be4c8e74d7185957b0c4ad6abce P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.636142 14654 tablet_service.cc:1467] Processing CreateTablet for tablet 0687887b68b848d78f19f5674f269fb3 (DEFAULT_TABLE table=default.Table_34987b0302b64565abe50420f8c86a38 [id=07a15fb8d76e43c89f7e05eccc1dccba]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.636374 14654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0687887b68b848d78f19f5674f269fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.638993 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 0687887b68b848d78f19f5674f269fb3 (DEFAULT_TABLE table=default.Table_34987b0302b64565abe50420f8c86a38 [id=07a15fb8d76e43c89f7e05eccc1dccba]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.639279 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0687887b68b848d78f19f5674f269fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.639535 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_9b10ca11a6584a9da1f16a9bc803131b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.639827 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9b10ca11a6584a9da1f16a9bc803131b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.640376 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 0687887b68b848d78f19f5674f269fb3 (DEFAULT_TABLE table=default.Table_34987b0302b64565abe50420f8c86a38 [id=07a15fb8d76e43c89f7e05eccc1dccba]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.640611 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0687887b68b848d78f19f5674f269fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.640738 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_c1c66695a4cf4e7d8c148ac48b3f932b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:19.628710 14809 tablet_bootstrap.cc:654] T 623887ec17374f598f48054e17582f3f P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.642046 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 98913be4c8e74d7185957b0c4ad6abce (table default.Table_71fadacb26bd4e4a8ddcd48e403350bb [id=529f6a8366ca4afd8e51cc396a8bb7a1]) successfully deleted
W20250115 03:06:19.642247 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c1c66695a4cf4e7d8c148ac48b3f932b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.642834 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 09f6eb327daa4b69a974db2aeced0603
I20250115 03:06:19.642922 14809 tablet_bootstrap.cc:492] T 623887ec17374f598f48054e17582f3f P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.642995 14809 ts_tablet_manager.cc:1397] T 623887ec17374f598f48054e17582f3f P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20250115 03:06:19.643388 14809 raft_consensus.cc:357] T 623887ec17374f598f48054e17582f3f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.643558 14809 raft_consensus.cc:383] T 623887ec17374f598f48054e17582f3f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.643637 14809 raft_consensus.cc:738] T 623887ec17374f598f48054e17582f3f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.643815 14809 consensus_queue.cc:260] T 623887ec17374f598f48054e17582f3f P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.648289 14654 tablet_service.cc:1514] Processing DeleteTablet for tablet 09f6eb327daa4b69a974db2aeced0603 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:19.648681 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 09f6eb327daa4b69a974db2aeced0603 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09f6eb327daa4b69a974db2aeced0603 already in progress: creating tablet
I20250115 03:06:19.648852 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet 09f6eb327daa4b69a974db2aeced0603 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.650869 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 623887ec17374f598f48054e17582f3f with delete_type TABLET_DATA_DELETED (replaced by tablet 12678da7ffee4a46b4690ac3e94cb19b) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.651063 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 09f6eb327daa4b69a974db2aeced0603 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:19.652628 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 623887ec17374f598f48054e17582f3f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 623887ec17374f598f48054e17582f3f already in progress: creating tablet
W20250115 03:06:19.652768 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 09f6eb327daa4b69a974db2aeced0603 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09f6eb327daa4b69a974db2aeced0603 already in progress: creating tablet
I20250115 03:06:19.653856 14809 ts_tablet_manager.cc:1428] T 623887ec17374f598f48054e17582f3f P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.011s user 0.001s sys 0.000s
I20250115 03:06:19.657234 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 12b78b3abfe84714a97ecc89f2fb9e33
I20250115 03:06:19.657488 14820 ts_tablet_manager.cc:1905] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.657617 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 2b0e3df3c79c40ba958b6e2453fa666f
I20250115 03:06:19.657994 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 55538831d8ea47eda40c36d42c19435f
I20250115 03:06:19.659047 14885 consensus_queue.cc:1035] T 907301ca08e0467f9f29c6dcb36bdd5e P e8df7247ee9c43e0aa18216da2661f75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:19.659291 14670 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b0e3df3c79c40ba958b6e2453fa666f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.659435 14655 tablet_service.cc:1514] Processing DeleteTablet for tablet 55538831d8ea47eda40c36d42c19435f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.659539 14654 tablet_service.cc:1514] Processing DeleteTablet for tablet 12b78b3abfe84714a97ecc89f2fb9e33 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.659909 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 55538831d8ea47eda40c36d42c19435f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:19.659961 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 2b0e3df3c79c40ba958b6e2453fa666f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2b0e3df3c79c40ba958b6e2453fa666f already in progress: creating tablet
I20250115 03:06:19.660036 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 12b78b3abfe84714a97ecc89f2fb9e33 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:19.660099 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 55538831d8ea47eda40c36d42c19435f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55538831d8ea47eda40c36d42c19435f already in progress: creating tablet
I20250115 03:06:19.660663 14809 tablet_bootstrap.cc:492] T fbd161e290914d9d8671c10becfeaee1 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.660995 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b0e3df3c79c40ba958b6e2453fa666f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.662583 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet cd6f136b92ba4fc7b956d52aa0c20f53 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:19.663842 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b0e3df3c79c40ba958b6e2453fa666f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.664000 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet 55538831d8ea47eda40c36d42c19435f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.664870 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet 12b78b3abfe84714a97ecc89f2fb9e33 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.666267 16001 raft_consensus.cc:491] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.666349 16001 raft_consensus.cc:513] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:19.666384 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 55538831d8ea47eda40c36d42c19435f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55538831d8ea47eda40c36d42c19435f already in progress: creating tablet
W20250115 03:06:19.666514 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 12b78b3abfe84714a97ecc89f2fb9e33 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12b78b3abfe84714a97ecc89f2fb9e33 already in progress: creating tablet
W20250115 03:06:19.666635 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 2b0e3df3c79c40ba958b6e2453fa666f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2b0e3df3c79c40ba958b6e2453fa666f already in progress: creating tablet
I20250115 03:06:19.666699 16001 leader_election.cc:290] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
W20250115 03:06:19.666750 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 12b78b3abfe84714a97ecc89f2fb9e33 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12b78b3abfe84714a97ecc89f2fb9e33 already in progress: creating tablet
I20250115 03:06:19.666837 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 623887ec17374f598f48054e17582f3f with delete_type TABLET_DATA_DELETED (replaced by tablet 12678da7ffee4a46b4690ac3e94cb19b) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.666978 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5a8293f43134a54b3f8dfb5fb99d6ea" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:19.667079 14422 raft_consensus.cc:2463] T e5a8293f43134a54b3f8dfb5fb99d6ea P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:19.667090 14655 tablet_service.cc:1514] Processing DeleteTablet for tablet e2c6db404348466ea5211ffa69c8f01c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.667232 14654 tablet_service.cc:1514] Processing DeleteTablet for tablet da95daa40c5b4615a0219f6d8e3770f6 with delete_type TABLET_DATA_DELETED (replaced by tablet 678b4453675b42ab98832a20f1b0271b) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.667307 14819 tablet_replica.cc:331] T 623887ec17374f598f48054e17582f3f P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.667333 14823 tablet_replica.cc:331] T e2c6db404348466ea5211ffa69c8f01c P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:19.667416 14819 raft_consensus.cc:2238] T 623887ec17374f598f48054e17582f3f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.667446 14823 raft_consensus.cc:2238] T e2c6db404348466ea5211ffa69c8f01c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.667493 14819 raft_consensus.cc:2267] T 623887ec17374f598f48054e17582f3f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.667541 14823 raft_consensus.cc:2267] T e2c6db404348466ea5211ffa69c8f01c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.668080 14823 ts_tablet_manager.cc:1905] T e2c6db404348466ea5211ffa69c8f01c P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.669124 14628 leader_election.cc:304] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:19.669277 16001 raft_consensus.cc:2798] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:19.669317 16001 raft_consensus.cc:491] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:19.669351 16001 raft_consensus.cc:3054] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:19.670094 14823 ts_tablet_manager.cc:1918] T e2c6db404348466ea5211ffa69c8f01c P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.670148 14823 log.cc:1198] T e2c6db404348466ea5211ffa69c8f01c P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/e2c6db404348466ea5211ffa69c8f01c
I20250115 03:06:19.670257 16001 raft_consensus.cc:513] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.670322 14823 ts_tablet_manager.cc:1939] T e2c6db404348466ea5211ffa69c8f01c P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:19.670616 16001 leader_election.cc:290] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:19.671043 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5a8293f43134a54b3f8dfb5fb99d6ea" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:19.671145 14420 raft_consensus.cc:3054] T e5a8293f43134a54b3f8dfb5fb99d6ea P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:19.671188 14823 tablet_replica.cc:331] T da95daa40c5b4615a0219f6d8e3770f6 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:19.671319 14823 raft_consensus.cc:2238] T da95daa40c5b4615a0219f6d8e3770f6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.671375 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5a8293f43134a54b3f8dfb5fb99d6ea" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:19.671809 14820 ts_tablet_manager.cc:1918] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.671871 14820 log.cc:1198] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/5d27310f8433411a85bf88d05a81bcf0
I20250115 03:06:19.671401 14823 raft_consensus.cc:2267] T da95daa40c5b4615a0219f6d8e3770f6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.672058 14820 ts_tablet_manager.cc:1939] T 5d27310f8433411a85bf88d05a81bcf0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.672142 14420 raft_consensus.cc:2463] T e5a8293f43134a54b3f8dfb5fb99d6ea P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:19.672338 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5a8293f43134a54b3f8dfb5fb99d6ea" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:19.672319 14820 tablet_replica.cc:331] T d2220e7ec26b4708b2c7a2540c053dcf P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.672529 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet cd6f136b92ba4fc7b956d52aa0c20f53
I20250115 03:06:19.672634 14820 raft_consensus.cc:2238] T d2220e7ec26b4708b2c7a2540c053dcf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.672725 14820 raft_consensus.cc:2267] T d2220e7ec26b4708b2c7a2540c053dcf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.672760 14168 catalog_manager.cc:6596] ReplaceTablet: tablet cd6f136b92ba4fc7b956d52aa0c20f53 deleted and replaced by tablet e558fc096deb44bf9203248d98bea5cc
I20250115 03:06:19.673173 14812 tablet_bootstrap.cc:492] T c251960c7c9a44429102a294f35f925c P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:19.673223 14628 leader_election.cc:304] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:19.673251 14812 ts_tablet_manager.cc:1397] T c251960c7c9a44429102a294f35f925c P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.000s
I20250115 03:06:19.673717 14820 ts_tablet_manager.cc:1905] T d2220e7ec26b4708b2c7a2540c053dcf P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.673753 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet e2c6db404348466ea5211ffa69c8f01c (table default.Table_ef19dd8e16aa4bf9a97897ebc1c8eef5 [id=38bfb0375c334d16b97c7251d6ddd76b]) successfully deleted
I20250115 03:06:19.673789 14812 raft_consensus.cc:357] T c251960c7c9a44429102a294f35f925c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.673904 14812 raft_consensus.cc:383] T c251960c7c9a44429102a294f35f925c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.673945 14812 raft_consensus.cc:738] T c251960c7c9a44429102a294f35f925c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.674062 14812 consensus_queue.cc:260] T c251960c7c9a44429102a294f35f925c P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.674328 14812 ts_tablet_manager.cc:1428] T c251960c7c9a44429102a294f35f925c P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:19.674525 14812 tablet_bootstrap.cc:492] T 312dcdda8cf042648db9fdf6228062d9 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:19.674808 15153 raft_consensus.cc:2798] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:19.674888 15153 raft_consensus.cc:695] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:19.674991 15153 consensus_queue.cc:237] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.675444 14812 tablet_bootstrap.cc:654] T 312dcdda8cf042648db9fdf6228062d9 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.675680 14654 tablet_service.cc:1514] Processing DeleteTablet for tablet cd6f136b92ba4fc7b956d52aa0c20f53 with delete_type TABLET_DATA_DELETED (replaced by tablet e558fc096deb44bf9203248d98bea5cc) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:19.676518 14626 leader_election.cc:343] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:19.676682 14626 leader_election.cc:343] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.678123 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 5d27310f8433411a85bf88d05a81bcf0 (table default.Table_eec42b37d6de49ccad194b13edd928e3 [id=5d934a52e6704fa68f7c685ae3c72081]) successfully deleted
I20250115 03:06:19.678500 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet cd6f136b92ba4fc7b956d52aa0c20f53 with delete_type TABLET_DATA_DELETED (replaced by tablet e558fc096deb44bf9203248d98bea5cc) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.678656 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_174d0415fa854efd9c7dd3405d04a414" } new_table_name: "default.Table_94c5e41f3cf24fc983cb6108737fb277" modify_external_catalogs: true
I20250115 03:06:19.678685 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_3f9710271c4c45a586f35f80a9e271c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:19.678886 14823 ts_tablet_manager.cc:1905] T da95daa40c5b4615a0219f6d8e3770f6 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:19.678990 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3f9710271c4c45a586f35f80a9e271c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.679615 14171 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0c984dbf685d4dad9338caa1119c5e44" } new_table_name: "default.Table_5e72c1c96c004205bf7304be210f76f1" modify_external_catalogs: true
I20250115 03:06:19.680262 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet cd6f136b92ba4fc7b956d52aa0c20f53 with delete_type TABLET_DATA_DELETED (replaced by tablet e558fc096deb44bf9203248d98bea5cc) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.680778 14823 ts_tablet_manager.cc:1918] T da95daa40c5b4615a0219f6d8e3770f6 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.681780 14809 tablet_bootstrap.cc:654] T fbd161e290914d9d8671c10becfeaee1 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.682401 14823 log.cc:1198] T da95daa40c5b4615a0219f6d8e3770f6 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/da95daa40c5b4615a0219f6d8e3770f6
I20250115 03:06:19.682670 14823 ts_tablet_manager.cc:1939] T da95daa40c5b4615a0219f6d8e3770f6 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:19.683264 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet da95daa40c5b4615a0219f6d8e3770f6 (table default.Table_4bde77af131940a099eabda16fa2ccb2 [id=4f952e57c86746f6bb1df72e13589832]) successfully deleted
W20250115 03:06:19.683531 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet cd6f136b92ba4fc7b956d52aa0c20f53 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd6f136b92ba4fc7b956d52aa0c20f53 already in progress: creating tablet
I20250115 03:06:19.683612 14819 ts_tablet_manager.cc:1905] T 623887ec17374f598f48054e17582f3f P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.685645 14819 ts_tablet_manager.cc:1918] T 623887ec17374f598f48054e17582f3f P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.685706 14819 log.cc:1198] T 623887ec17374f598f48054e17582f3f P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/623887ec17374f598f48054e17582f3f
I20250115 03:06:19.685894 14819 ts_tablet_manager.cc:1939] T 623887ec17374f598f48054e17582f3f P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.686589 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_543c011b1f2a4240be626d4454882dc8" } new_table_name: "default.Table_108b2e9d9cd745e99201b20cb34eb6f7" modify_external_catalogs: true
I20250115 03:06:19.687613 16185 tablet.cc:1721] T fc056b9dbc374d26bf3feaf524f4d834 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:19.688167 15970 tablet.cc:1721] T fc056b9dbc374d26bf3feaf524f4d834 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:19.688695 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 623887ec17374f598f48054e17582f3f (table default.Table_0337ad43f9b646a48c61c66c15f1844a [id=7f5506ea0a5e465cba8de1ba2f5e75c6]) successfully deleted
W20250115 03:06:19.689128 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet cd6f136b92ba4fc7b956d52aa0c20f53 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd6f136b92ba4fc7b956d52aa0c20f53 already in progress: creating tablet
I20250115 03:06:19.692196 14812 tablet_bootstrap.cc:492] T 312dcdda8cf042648db9fdf6228062d9 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:19.692302 14812 ts_tablet_manager.cc:1397] T 312dcdda8cf042648db9fdf6228062d9 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20250115 03:06:19.692752 14812 raft_consensus.cc:357] T 312dcdda8cf042648db9fdf6228062d9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:19.695010 14171 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 72abf6acd9d54f3b983b7098aaf2691d consensus state has no leader
W20250115 03:06:19.695263 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d1c4e127c99241da950af20754e6667c consensus state has no leader
I20250115 03:06:19.695768 14166 catalog_manager.cc:5526] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:19.696638 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 87fea2193b584f2898171517362a2cf8 (DEFAULT_TABLE table=default.Table_bfdb42e52734403c9ba4b455ea0e6e44 [id=1e8cf5f2a3aa44fc91270918559dea21]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.696895 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87fea2193b584f2898171517362a2cf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.697002 14820 ts_tablet_manager.cc:1918] T d2220e7ec26b4708b2c7a2540c053dcf P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.697067 14820 log.cc:1198] T d2220e7ec26b4708b2c7a2540c053dcf P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/d2220e7ec26b4708b2c7a2540c053dcf
I20250115 03:06:19.697256 14820 ts_tablet_manager.cc:1939] T d2220e7ec26b4708b2c7a2540c053dcf P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.700035 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 772f665935f845f58d5a2b6095bcd564 (DEFAULT_TABLE table=default.Table_9fffa7d4b0834e3e85a07968b118ac3a [id=92be405981194051bde5f9108ba14027]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.700245 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 772f665935f845f58d5a2b6095bcd564. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.700045 14390 tablet_service.cc:1467] Processing CreateTablet for tablet 6f53ad530584487e946120a724f99cfa (DEFAULT_TABLE table=default.Table_47fd03c657214fea893d4881cbbff5f1 [id=b4b9e2dd19f1416ca06cd5f53ffa5109]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.700448 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f53ad530584487e946120a724f99cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.701140 14400 tablet_service.cc:1467] Processing CreateTablet for tablet dcc1c999a9d34006a9bd4ec2bec02bdd (DEFAULT_TABLE table=default.Table_fe5a8443c8e54cb6a0f436c90a2b5b91 [id=deb35d72fb8349f4866b49e2963740ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.701334 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcc1c999a9d34006a9bd4ec2bec02bdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.702698 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 35aee3e2a8234005989ff61a25703ad2 (DEFAULT_TABLE table=default.Table_0c02079a9e2a4cb2ac71a7f8ea1b2cc9 [id=404b7a3602814f9aaf1f2199bcf9adea]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.702919 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35aee3e2a8234005989ff61a25703ad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.703469 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 87fea2193b584f2898171517362a2cf8 (DEFAULT_TABLE table=default.Table_bfdb42e52734403c9ba4b455ea0e6e44 [id=1e8cf5f2a3aa44fc91270918559dea21]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.703536 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 772f665935f845f58d5a2b6095bcd564 (DEFAULT_TABLE table=default.Table_9fffa7d4b0834e3e85a07968b118ac3a [id=92be405981194051bde5f9108ba14027]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.703686 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87fea2193b584f2898171517362a2cf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.703831 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 772f665935f845f58d5a2b6095bcd564. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.705994 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 6f53ad530584487e946120a724f99cfa (DEFAULT_TABLE table=default.Table_47fd03c657214fea893d4881cbbff5f1 [id=b4b9e2dd19f1416ca06cd5f53ffa5109]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.706218 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f53ad530584487e946120a724f99cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.706187 14530 tablet_service.cc:1467] Processing CreateTablet for tablet dcc1c999a9d34006a9bd4ec2bec02bdd (DEFAULT_TABLE table=default.Table_fe5a8443c8e54cb6a0f436c90a2b5b91 [id=deb35d72fb8349f4866b49e2963740ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.706439 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 35aee3e2a8234005989ff61a25703ad2 (DEFAULT_TABLE table=default.Table_0c02079a9e2a4cb2ac71a7f8ea1b2cc9 [id=404b7a3602814f9aaf1f2199bcf9adea]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.696523 14654 tablet_service.cc:1467] Processing CreateTablet for tablet 87fea2193b584f2898171517362a2cf8 (DEFAULT_TABLE table=default.Table_bfdb42e52734403c9ba4b455ea0e6e44 [id=1e8cf5f2a3aa44fc91270918559dea21]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.706811 14654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87fea2193b584f2898171517362a2cf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.707824 14554 raft_consensus.cc:3054] T 8b0f0539530643f78207e528c39530af P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:19.707895 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcc1c999a9d34006a9bd4ec2bec02bdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.699594 14655 tablet_service.cc:1467] Processing CreateTablet for tablet 35aee3e2a8234005989ff61a25703ad2 (DEFAULT_TABLE table=default.Table_0c02079a9e2a4cb2ac71a7f8ea1b2cc9 [id=404b7a3602814f9aaf1f2199bcf9adea]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.708895 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35aee3e2a8234005989ff61a25703ad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.709663 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35aee3e2a8234005989ff61a25703ad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.710044 16189 raft_consensus.cc:491] T 58130f2b27b745d08c0002ac96af6d72 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.710119 16189 raft_consensus.cc:513] T 58130f2b27b745d08c0002ac96af6d72 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.710436 16189 leader_election.cc:290] T 58130f2b27b745d08c0002ac96af6d72 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:19.710543 16185 tablet.cc:1721] T 8b0f0539530643f78207e528c39530af P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:19.699704 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 772f665935f845f58d5a2b6095bcd564 (DEFAULT_TABLE table=default.Table_9fffa7d4b0834e3e85a07968b118ac3a [id=92be405981194051bde5f9108ba14027]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.710937 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 772f665935f845f58d5a2b6095bcd564. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.711241 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58130f2b27b745d08c0002ac96af6d72" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:19.711609 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58130f2b27b745d08c0002ac96af6d72" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:19.712185 14493 leader_election.cc:304] T 58130f2b27b745d08c0002ac96af6d72 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:19.712392 14858 raft_consensus.cc:2743] T 58130f2b27b745d08c0002ac96af6d72 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:19.699793 14660 tablet_service.cc:1467] Processing CreateTablet for tablet dcc1c999a9d34006a9bd4ec2bec02bdd (DEFAULT_TABLE table=default.Table_fe5a8443c8e54cb6a0f436c90a2b5b91 [id=deb35d72fb8349f4866b49e2963740ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.713009 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcc1c999a9d34006a9bd4ec2bec02bdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.699885 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 6f53ad530584487e946120a724f99cfa (DEFAULT_TABLE table=default.Table_47fd03c657214fea893d4881cbbff5f1 [id=b4b9e2dd19f1416ca06cd5f53ffa5109]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.715224 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f53ad530584487e946120a724f99cfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.692909 14812 raft_consensus.cc:383] T 312dcdda8cf042648db9fdf6228062d9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.717118 14812 raft_consensus.cc:738] T 312dcdda8cf042648db9fdf6228062d9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.717361 14812 consensus_queue.cc:260] T 312dcdda8cf042648db9fdf6228062d9 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.717964 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet d2220e7ec26b4708b2c7a2540c053dcf (table default.Table_807e7c2704994c25aa03b1267a20f794 [id=fb7018f8e1ac4d6db5c0c114b0e73a07]) successfully deleted
W20250115 03:06:19.718098 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 09f6eb327daa4b69a974db2aeced0603 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09f6eb327daa4b69a974db2aeced0603 already in progress: creating tablet
W20250115 03:06:19.718191 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 2b0e3df3c79c40ba958b6e2453fa666f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2b0e3df3c79c40ba958b6e2453fa666f already in progress: creating tablet
W20250115 03:06:19.718268 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 55538831d8ea47eda40c36d42c19435f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 55538831d8ea47eda40c36d42c19435f already in progress: creating tablet
W20250115 03:06:19.718408 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 12b78b3abfe84714a97ecc89f2fb9e33 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12b78b3abfe84714a97ecc89f2fb9e33 already in progress: creating tablet
W20250115 03:06:19.718493 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet cd6f136b92ba4fc7b956d52aa0c20f53 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cd6f136b92ba4fc7b956d52aa0c20f53 already in progress: creating tablet
W20250115 03:06:19.721015 14167 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4283bf9f53c54e7095e270fa04fc4f12 consensus state has no leader
I20250115 03:06:19.722731 14812 ts_tablet_manager.cc:1428] T 312dcdda8cf042648db9fdf6228062d9 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.030s user 0.000s sys 0.001s
I20250115 03:06:19.728344 14812 tablet_bootstrap.cc:492] T 98913be4c8e74d7185957b0c4ad6abce P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:19.730662 14812 tablet_bootstrap.cc:654] T 98913be4c8e74d7185957b0c4ad6abce P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.730909 14167 master_service.cc:946] ReplaceTablet: received request to replace tablet 78b5e2e16442407880b75d256e6b2614 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:19.734732 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a8a91cdcd75a40dea727b3c6890804c3" } new_table_name: "default.Table_0f3709600e57401499c5004118dc8ef3" modify_external_catalogs: true
I20250115 03:06:19.735476 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet 2bffef32339a4aa6b2ef7fb5661d035c from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:19.738163 14170 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_60d8dde0d2c04c069ae9502e04496ffa [id=3bcfb6a13f2b4d56b801fcf210764fea] alter complete (version 3)
I20250115 03:06:19.739792 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet c251960c7c9a44429102a294f35f925c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.739936 14823 tablet_replica.cc:331] T c251960c7c9a44429102a294f35f925c P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:19.740043 14823 raft_consensus.cc:2238] T c251960c7c9a44429102a294f35f925c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.740123 14823 raft_consensus.cc:2267] T c251960c7c9a44429102a294f35f925c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.740552 14823 ts_tablet_manager.cc:1905] T c251960c7c9a44429102a294f35f925c P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.742370 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 312dcdda8cf042648db9fdf6228062d9 with delete_type TABLET_DATA_DELETED (replaced by tablet 7b86e66428504217af08f667f660d304) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.742714 14823 ts_tablet_manager.cc:1918] T c251960c7c9a44429102a294f35f925c P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.742772 14823 log.cc:1198] T c251960c7c9a44429102a294f35f925c P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/c251960c7c9a44429102a294f35f925c
I20250115 03:06:19.742964 14823 ts_tablet_manager.cc:1939] T c251960c7c9a44429102a294f35f925c P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:19.743438 14823 tablet_replica.cc:331] T 312dcdda8cf042648db9fdf6228062d9 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:19.743554 14823 raft_consensus.cc:2238] T 312dcdda8cf042648db9fdf6228062d9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.743643 14823 raft_consensus.cc:2267] T 312dcdda8cf042648db9fdf6228062d9 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.743690 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet c251960c7c9a44429102a294f35f925c (table default.Table_90aedfcdb1274e1497cf36620b980768 [id=9e4f5de45eb24beabef550d15dcad8bf]) successfully deleted
I20250115 03:06:19.744083 14823 ts_tablet_manager.cc:1905] T 312dcdda8cf042648db9fdf6228062d9 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:19.745823 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dda418ed197e4af2aa511d3f1851663e consensus state has no leader
I20250115 03:06:19.746073 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 2bffef32339a4aa6b2ef7fb5661d035c
I20250115 03:06:19.746335 14171 catalog_manager.cc:6596] ReplaceTablet: tablet 2bffef32339a4aa6b2ef7fb5661d035c deleted and replaced by tablet 34bd3886562742bc980c571ca76230fe
I20250115 03:06:19.746518 14823 ts_tablet_manager.cc:1918] T 312dcdda8cf042648db9fdf6228062d9 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.746640 14823 log.cc:1198] T 312dcdda8cf042648db9fdf6228062d9 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/312dcdda8cf042648db9fdf6228062d9
I20250115 03:06:19.746819 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bffef32339a4aa6b2ef7fb5661d035c with delete_type TABLET_DATA_DELETED (replaced by tablet 34bd3886562742bc980c571ca76230fe) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.746970 14819 tablet_replica.cc:331] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.747031 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 78b5e2e16442407880b75d256e6b2614
I20250115 03:06:19.747092 14819 raft_consensus.cc:2238] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:19.747256 14167 catalog_manager.cc:6596] ReplaceTablet: tablet 78b5e2e16442407880b75d256e6b2614 deleted and replaced by tablet ec1a8d0326f84ca0a87851010c1e68b7
I20250115 03:06:19.747252 14819 raft_consensus.cc:2267] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.747591 14819 ts_tablet_manager.cc:1905] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.749308 14811 tablet_bootstrap.cc:492] T 074fdc09e4e24078ad0bb588b32ff24f P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.749364 14809 tablet_bootstrap.cc:492] T fbd161e290914d9d8671c10becfeaee1 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.749387 14811 ts_tablet_manager.cc:1397] T 074fdc09e4e24078ad0bb588b32ff24f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.125s user 0.001s sys 0.000s
I20250115 03:06:19.749444 14809 ts_tablet_manager.cc:1397] T fbd161e290914d9d8671c10becfeaee1 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.089s user 0.002s sys 0.000s
I20250115 03:06:19.749850 14811 raft_consensus.cc:357] T 074fdc09e4e24078ad0bb588b32ff24f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.749886 14809 raft_consensus.cc:357] T fbd161e290914d9d8671c10becfeaee1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.749959 14811 raft_consensus.cc:383] T 074fdc09e4e24078ad0bb588b32ff24f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.749981 14809 raft_consensus.cc:383] T fbd161e290914d9d8671c10becfeaee1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.749994 14811 raft_consensus.cc:738] T 074fdc09e4e24078ad0bb588b32ff24f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.750010 14809 raft_consensus.cc:738] T fbd161e290914d9d8671c10becfeaee1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.750102 14811 consensus_queue.cc:260] T 074fdc09e4e24078ad0bb588b32ff24f P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.750114 14809 consensus_queue.cc:260] T fbd161e290914d9d8671c10becfeaee1 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.750269 15617 tablet.cc:1721] T fc056b9dbc374d26bf3feaf524f4d834 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:19.750355 14811 ts_tablet_manager.cc:1428] T 074fdc09e4e24078ad0bb588b32ff24f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.750373 14809 ts_tablet_manager.cc:1428] T fbd161e290914d9d8671c10becfeaee1 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.750524 14811 tablet_bootstrap.cc:492] T c84127dd93bc47ada551f93295ead47b P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.750536 14809 tablet_bootstrap.cc:492] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.750561 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 78b5e2e16442407880b75d256e6b2614 with delete_type TABLET_DATA_DELETED (replaced by tablet ec1a8d0326f84ca0a87851010c1e68b7) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.750882 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bffef32339a4aa6b2ef7fb5661d035c with delete_type TABLET_DATA_DELETED (replaced by tablet 34bd3886562742bc980c571ca76230fe) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.751446 14809 tablet_bootstrap.cc:654] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.751473 14811 tablet_bootstrap.cc:654] T c84127dd93bc47ada551f93295ead47b P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.754215 14554 raft_consensus.cc:3054] T 58130f2b27b745d08c0002ac96af6d72 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:19.754730 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 78b5e2e16442407880b75d256e6b2614 with delete_type TABLET_DATA_DELETED (replaced by tablet ec1a8d0326f84ca0a87851010c1e68b7) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.755761 14552 raft_consensus.cc:3054] T 7c503b29cb2843338fb1870a89aee443 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:19.757409 16058 mvcc.cc:204] Tried to move back new op lower bound from 7114384912191311872 to 7114384908797235200. Current Snapshot: MvccSnapshot[applied={T|T < 7114384912191311872}]
I20250115 03:06:19.757630 16185 tablet.cc:1721] T 58130f2b27b745d08c0002ac96af6d72 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:19.757881 15426 tablet.cc:1721] T 7c503b29cb2843338fb1870a89aee443 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:19.760735 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bffef32339a4aa6b2ef7fb5661d035c with delete_type TABLET_DATA_DELETED (replaced by tablet 34bd3886562742bc980c571ca76230fe) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.760946 15401 raft_consensus.cc:491] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.761019 15401 raft_consensus.cc:513] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.761126 14820 tablet_replica.cc:331] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.761233 14820 raft_consensus.cc:2238] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.761317 14820 raft_consensus.cc:2267] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.761356 15401 leader_election.cc:290] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:19.761422 14819 ts_tablet_manager.cc:1918] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:19.761473 14819 log.cc:1198] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/2bffef32339a4aa6b2ef7fb5661d035c
I20250115 03:06:19.761613 14820 ts_tablet_manager.cc:1905] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.761699 14819 ts_tablet_manager.cc:1939] T 2bffef32339a4aa6b2ef7fb5661d035c P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.762180 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56279bb16054869a97006d88574e381" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:19.762516 14358 leader_election.cc:343] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.762526 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 2bffef32339a4aa6b2ef7fb5661d035c (table default.Table_e303383ddf3845828de6837219313014 [id=654dfdc4477b476f80e6af5b066713c2]) successfully deleted
I20250115 03:06:19.762878 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet 78b5e2e16442407880b75d256e6b2614 with delete_type TABLET_DATA_DELETED (replaced by tablet ec1a8d0326f84ca0a87851010c1e68b7) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:19.762912 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 78b5e2e16442407880b75d256e6b2614 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78b5e2e16442407880b75d256e6b2614 already in progress: creating tablet
I20250115 03:06:19.763314 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet 75c581f11be84e6293a3d14966fd584f from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:19.763546 14820 ts_tablet_manager.cc:1918] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:19.763551 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet 0792e52d171446b48a7c9d1f03d37a79 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:19.763599 14820 log.cc:1198] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/2bffef32339a4aa6b2ef7fb5661d035c
I20250115 03:06:19.763809 14820 ts_tablet_manager.cc:1939] T 2bffef32339a4aa6b2ef7fb5661d035c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.764245 14823 ts_tablet_manager.cc:1939] T 312dcdda8cf042648db9fdf6228062d9 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:19.764398 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56279bb16054869a97006d88574e381" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:19.764693 14356 leader_election.cc:343] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.764762 14356 leader_election.cc:304] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:19.764900 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 312dcdda8cf042648db9fdf6228062d9 (table default.Table_341a3cd1c8c94ff4b05fc21960a4e9b4 [id=c096ce170874405ea740b9ea829f886e]) successfully deleted
I20250115 03:06:19.764963 15401 raft_consensus.cc:2743] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:19.765280 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 78b5e2e16442407880b75d256e6b2614 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78b5e2e16442407880b75d256e6b2614 already in progress: creating tablet
I20250115 03:06:19.765455 14823 tablet_replica.cc:331] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:19.765578 14823 raft_consensus.cc:2238] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.765667 14823 raft_consensus.cc:2267] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.766012 14823 ts_tablet_manager.cc:1905] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.766883 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 2bffef32339a4aa6b2ef7fb5661d035c (table default.Table_e303383ddf3845828de6837219313014 [id=654dfdc4477b476f80e6af5b066713c2]) successfully deleted
W20250115 03:06:19.767010 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 78b5e2e16442407880b75d256e6b2614 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78b5e2e16442407880b75d256e6b2614 already in progress: creating tablet
I20250115 03:06:19.768060 14823 ts_tablet_manager.cc:1918] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:19.768115 14823 log.cc:1198] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/2bffef32339a4aa6b2ef7fb5661d035c
I20250115 03:06:19.768329 14823 ts_tablet_manager.cc:1939] T 2bffef32339a4aa6b2ef7fb5661d035c P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:19.768870 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 2bffef32339a4aa6b2ef7fb5661d035c (table default.Table_e303383ddf3845828de6837219313014 [id=654dfdc4477b476f80e6af5b066713c2]) successfully deleted
I20250115 03:06:19.769613 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet 074fdc09e4e24078ad0bb588b32ff24f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.769801 14820 tablet_replica.cc:331] T 074fdc09e4e24078ad0bb588b32ff24f P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.769907 14820 raft_consensus.cc:2238] T 074fdc09e4e24078ad0bb588b32ff24f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.769982 14820 raft_consensus.cc:2267] T 074fdc09e4e24078ad0bb588b32ff24f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.770414 14820 ts_tablet_manager.cc:1905] T 074fdc09e4e24078ad0bb588b32ff24f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.772387 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_64d33ab8807d44a996a1f0fcc2deb7e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.772763 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_64d33ab8807d44a996a1f0fcc2deb7e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.773672 14820 ts_tablet_manager.cc:1918] T 074fdc09e4e24078ad0bb588b32ff24f P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.773806 14820 log.cc:1198] T 074fdc09e4e24078ad0bb588b32ff24f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/074fdc09e4e24078ad0bb588b32ff24f
I20250115 03:06:19.774032 14820 ts_tablet_manager.cc:1939] T 074fdc09e4e24078ad0bb588b32ff24f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.774402 14811 tablet_bootstrap.cc:492] T c84127dd93bc47ada551f93295ead47b P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.774494 14811 ts_tablet_manager.cc:1397] T c84127dd93bc47ada551f93295ead47b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.000s
I20250115 03:06:19.774920 14811 raft_consensus.cc:357] T c84127dd93bc47ada551f93295ead47b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.775024 14811 raft_consensus.cc:383] T c84127dd93bc47ada551f93295ead47b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.775065 14811 raft_consensus.cc:738] T c84127dd93bc47ada551f93295ead47b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.775182 14811 consensus_queue.cc:260] T c84127dd93bc47ada551f93295ead47b P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.775444 14811 ts_tablet_manager.cc:1428] T c84127dd93bc47ada551f93295ead47b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.775630 14811 tablet_bootstrap.cc:492] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.776528 14811 tablet_bootstrap.cc:654] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.779111 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 074fdc09e4e24078ad0bb588b32ff24f (table default.Table_87c206116f514479ab9de8cd4a05817f [id=eaddac40b10648b1933b121cc70d68e8]) successfully deleted
I20250115 03:06:19.779737 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet c84127dd93bc47ada551f93295ead47b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.780351 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ad994d4298c546df9e4808628545682a" } modify_external_catalogs: true
I20250115 03:06:19.780450 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ad994d4298c546df9e4808628545682a" } modify_external_catalogs: true
I20250115 03:06:19.781795 14820 tablet_replica.cc:331] T c84127dd93bc47ada551f93295ead47b P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.781944 14820 raft_consensus.cc:2238] T c84127dd93bc47ada551f93295ead47b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.783718 14820 raft_consensus.cc:2267] T c84127dd93bc47ada551f93295ead47b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.784070 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet f8c3406d44a74b20a4ef4b42e8d04ad0
I20250115 03:06:19.784440 14820 ts_tablet_manager.cc:1905] T c84127dd93bc47ada551f93295ead47b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.787118 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet f8c3406d44a74b20a4ef4b42e8d04ad0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.787128 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet f8c3406d44a74b20a4ef4b42e8d04ad0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.787187 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet f8c3406d44a74b20a4ef4b42e8d04ad0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.787277 14823 tablet_replica.cc:331] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:19.787407 14823 raft_consensus.cc:2238] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.787529 14823 raft_consensus.cc:2267] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.787868 14823 ts_tablet_manager.cc:1905] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.787989 14820 ts_tablet_manager.cc:1918] T c84127dd93bc47ada551f93295ead47b P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.788085 14820 log.cc:1198] T c84127dd93bc47ada551f93295ead47b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/c84127dd93bc47ada551f93295ead47b
I20250115 03:06:19.788175 14819 tablet_replica.cc:331] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.788308 14819 raft_consensus.cc:2238] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:19.788479 14819 raft_consensus.cc:2267] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.788818 14819 ts_tablet_manager.cc:1905] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.789892 14820 ts_tablet_manager.cc:1939] T c84127dd93bc47ada551f93295ead47b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.790385 14820 tablet_replica.cc:331] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.790482 14820 raft_consensus.cc:2238] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.790557 14820 raft_consensus.cc:2267] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.790853 14820 ts_tablet_manager.cc:1905] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.790895 14819 ts_tablet_manager.cc:1918] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:19.790958 14819 log.cc:1198] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/f8c3406d44a74b20a4ef4b42e8d04ad0
I20250115 03:06:19.791064 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet c84127dd93bc47ada551f93295ead47b (table default.Table_121a1b9da2674287a03165a67467717f [id=9391b8f0b8a144f592e4aeaef70fbeef]) successfully deleted
I20250115 03:06:19.791179 14819 ts_tablet_manager.cc:1939] T f8c3406d44a74b20a4ef4b42e8d04ad0 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.792061 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet f8c3406d44a74b20a4ef4b42e8d04ad0 (table default.Table_ad994d4298c546df9e4808628545682a [id=f88b8c3027ae47ceacd99aa8a2811628]) successfully deleted
I20250115 03:06:19.792680 14820 ts_tablet_manager.cc:1918] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:19.793195 14809 tablet_bootstrap.cc:492] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.793267 14809 ts_tablet_manager.cc:1397] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.043s user 0.001s sys 0.001s
I20250115 03:06:19.793661 14809 raft_consensus.cc:357] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.793768 14809 raft_consensus.cc:383] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.793802 14809 raft_consensus.cc:738] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.793889 14809 consensus_queue.cc:260] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.794133 14809 ts_tablet_manager.cc:1428] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:19.794296 14809 tablet_bootstrap.cc:492] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.794651 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b45108a844cb400cbc6220e82aaafb9e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.794958 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b45108a844cb400cbc6220e82aaafb9e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.795145 14809 tablet_bootstrap.cc:654] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.795509 14812 tablet_bootstrap.cc:492] T 98913be4c8e74d7185957b0c4ad6abce P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:19.795588 14812 ts_tablet_manager.cc:1397] T 98913be4c8e74d7185957b0c4ad6abce P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.067s user 0.001s sys 0.001s
I20250115 03:06:19.796013 14812 raft_consensus.cc:357] T 98913be4c8e74d7185957b0c4ad6abce P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.796115 14812 raft_consensus.cc:383] T 98913be4c8e74d7185957b0c4ad6abce P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.796149 14812 raft_consensus.cc:738] T 98913be4c8e74d7185957b0c4ad6abce P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.796280 14812 consensus_queue.cc:260] T 98913be4c8e74d7185957b0c4ad6abce P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.796563 14812 ts_tablet_manager.cc:1428] T 98913be4c8e74d7185957b0c4ad6abce P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.799788 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 98913be4c8e74d7185957b0c4ad6abce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.800495 14809 tablet_bootstrap.cc:492] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.800581 14809 ts_tablet_manager.cc:1397] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20250115 03:06:19.801021 14809 raft_consensus.cc:357] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.801112 14809 raft_consensus.cc:383] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.801143 14809 raft_consensus.cc:738] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.801352 14809 consensus_queue.cc:260] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.801661 14809 ts_tablet_manager.cc:1428] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.801852 14809 tablet_bootstrap.cc:492] T 4102586f32d94a998cf2807434ee0ebf P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.802884 14809 tablet_bootstrap.cc:654] T 4102586f32d94a998cf2807434ee0ebf P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.803033 14812 tablet_bootstrap.cc:492] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:19.803893 15970 tablet.cc:1721] T 5cb34cb2006f4716b87bef67f3a6dd8e P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:19.807358 15617 tablet.cc:1721] T 5cb34cb2006f4716b87bef67f3a6dd8e P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:19.808441 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_85da758784964b0c9135d0abba13c4cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.808779 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_85da758784964b0c9135d0abba13c4cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.803951 14812 tablet_bootstrap.cc:654] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.810712 14809 tablet_bootstrap.cc:492] T 4102586f32d94a998cf2807434ee0ebf P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.810789 14809 ts_tablet_manager.cc:1397] T 4102586f32d94a998cf2807434ee0ebf P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:19.811159 14809 raft_consensus.cc:357] T 4102586f32d94a998cf2807434ee0ebf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.811255 14809 raft_consensus.cc:383] T 4102586f32d94a998cf2807434ee0ebf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.811304 14809 raft_consensus.cc:738] T 4102586f32d94a998cf2807434ee0ebf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.811403 14809 consensus_queue.cc:260] T 4102586f32d94a998cf2807434ee0ebf P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.811655 14809 ts_tablet_manager.cc:1428] T 4102586f32d94a998cf2807434ee0ebf P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.811795 14809 tablet_bootstrap.cc:492] T c7e9a1d1ada941b68688e0ed6742aa99 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.812621 14809 tablet_bootstrap.cc:654] T c7e9a1d1ada941b68688e0ed6742aa99 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.815356 14168 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_77b3fb817e8b4502af7fae7bcb0124a5 [id=ed28aa2aaacd4ba58dcb3216a7156a91] alter complete (version 1)
I20250115 03:06:19.818677 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2ec77dbcffce476aa7f1d98daa698e84"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.819012 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ec77dbcffce476aa7f1d98daa698e84 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.801981 14811 tablet_bootstrap.cc:492] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.820902 14811 ts_tablet_manager.cc:1397] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.045s user 0.002s sys 0.000s
I20250115 03:06:19.821409 14811 raft_consensus.cc:357] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.821508 14811 raft_consensus.cc:383] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.821534 14811 raft_consensus.cc:738] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.821638 14811 consensus_queue.cc:260] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.821906 14811 ts_tablet_manager.cc:1428] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:19.822082 14811 tablet_bootstrap.cc:492] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.822970 14811 tablet_bootstrap.cc:654] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.825991 14820 log.cc:1198] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/f8c3406d44a74b20a4ef4b42e8d04ad0
I20250115 03:06:19.827556 14820 ts_tablet_manager.cc:1939] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.833477 14811 tablet_bootstrap.cc:492] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.833552 14811 ts_tablet_manager.cc:1397] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.002s
I20250115 03:06:19.833950 14811 raft_consensus.cc:357] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:19.834036 14811 raft_consensus.cc:383] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.834061 14811 raft_consensus.cc:738] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.834156 14811 consensus_queue.cc:260] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:19.834388 14811 ts_tablet_manager.cc:1428] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:19.834530 14811 tablet_bootstrap.cc:492] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.834647 14170 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 3efc2fb8427f493ea933347bec34f88f (table default.Table_ad04661b1b684f879f1c0cd4f94e5458 [id=364a7905f3f044f684e9c75927f7efda]). Expected version 1 got 0
I20250115 03:06:19.835399 14811 tablet_bootstrap.cc:654] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.835956 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_4ec1365f1a874eb49b19337126347443"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.836297 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4ec1365f1a874eb49b19337126347443 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.838940 14809 tablet_bootstrap.cc:492] T c7e9a1d1ada941b68688e0ed6742aa99 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.839016 14809 ts_tablet_manager.cc:1397] T c7e9a1d1ada941b68688e0ed6742aa99 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.002s
I20250115 03:06:19.839397 14809 raft_consensus.cc:357] T c7e9a1d1ada941b68688e0ed6742aa99 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.839483 14809 raft_consensus.cc:383] T c7e9a1d1ada941b68688e0ed6742aa99 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.839535 14809 raft_consensus.cc:738] T c7e9a1d1ada941b68688e0ed6742aa99 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.839668 14809 consensus_queue.cc:260] T c7e9a1d1ada941b68688e0ed6742aa99 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.839912 14809 ts_tablet_manager.cc:1428] T c7e9a1d1ada941b68688e0ed6742aa99 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:19.840049 14809 tablet_bootstrap.cc:492] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.841013 14809 tablet_bootstrap.cc:654] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.841468 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_fa0e3b934db54f57b3b6565e1a6abb38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.841856 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fa0e3b934db54f57b3b6565e1a6abb38 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.844513 14811 tablet_bootstrap.cc:492] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.844594 14811 ts_tablet_manager.cc:1397] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20250115 03:06:19.844772 14809 tablet_bootstrap.cc:492] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.844839 14809 ts_tablet_manager.cc:1397] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20250115 03:06:19.845037 14811 raft_consensus.cc:357] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:19.845129 14811 raft_consensus.cc:383] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.845160 14811 raft_consensus.cc:738] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.845248 14809 raft_consensus.cc:357] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.845265 14811 consensus_queue.cc:260] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:19.845330 14809 raft_consensus.cc:383] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.845358 14809 raft_consensus.cc:738] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.845458 14809 consensus_queue.cc:260] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.845507 14811 ts_tablet_manager.cc:1428] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.845675 14811 tablet_bootstrap.cc:492] T 01433c76a398462bb6708a02997047fa P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.845705 14809 ts_tablet_manager.cc:1428] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:19.845845 14809 tablet_bootstrap.cc:492] T 2c69daff62144976ae9082c89cc58eca P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.846510 14811 tablet_bootstrap.cc:654] T 01433c76a398462bb6708a02997047fa P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.846671 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet f8c3406d44a74b20a4ef4b42e8d04ad0 (table default.Table_ad994d4298c546df9e4808628545682a [id=f88b8c3027ae47ceacd99aa8a2811628]) successfully deleted
I20250115 03:06:19.847070 14809 tablet_bootstrap.cc:654] T 2c69daff62144976ae9082c89cc58eca P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.850976 14812 tablet_bootstrap.cc:492] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:19.851058 14812 ts_tablet_manager.cc:1397] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.048s user 0.001s sys 0.001s
I20250115 03:06:19.851480 14812 raft_consensus.cc:357] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.851573 14812 raft_consensus.cc:383] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.851599 14812 raft_consensus.cc:738] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.851709 14812 consensus_queue.cc:260] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.851950 14812 ts_tablet_manager.cc:1428] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:19.852121 14812 tablet_bootstrap.cc:492] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:19.853957 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 75c581f11be84e6293a3d14966fd584f
I20250115 03:06:19.854228 14171 catalog_manager.cc:6596] ReplaceTablet: tablet 75c581f11be84e6293a3d14966fd584f deleted and replaced by tablet 5c5e30d9b2ce4537b00b216ef1f2dd1c
I20250115 03:06:19.854686 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 75c581f11be84e6293a3d14966fd584f with delete_type TABLET_DATA_DELETED (replaced by tablet 5c5e30d9b2ce4537b00b216ef1f2dd1c) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.854986 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 75c581f11be84e6293a3d14966fd584f with delete_type TABLET_DATA_DELETED (replaced by tablet 5c5e30d9b2ce4537b00b216ef1f2dd1c) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.855140 14812 tablet_bootstrap.cc:654] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:19.855248 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 75c581f11be84e6293a3d14966fd584f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75c581f11be84e6293a3d14966fd584f already in progress: creating tablet
I20250115 03:06:19.856065 14170 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 15fbf8d5de904fc4b284af664f3bd60f (table default.Table_655e9d02aacf4b88ba521f0f57a5f522 [id=6ed01835736b40f8b4caeca7378770d8]). Expected version 1 got 0
I20250115 03:06:19.857075 14858 raft_consensus.cc:491] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.857153 14858 raft_consensus.cc:513] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:19.857466 14858 leader_election.cc:290] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
W20250115 03:06:19.858060 14493 leader_election.cc:343] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 69533c30b57e45dab0b00239dfd90f13
W20250115 03:06:19.858484 14493 leader_election.cc:343] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 69533c30b57e45dab0b00239dfd90f13
I20250115 03:06:19.858546 14493 leader_election.cc:304] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:19.858882 14858 raft_consensus.cc:2743] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:19.858930 14170 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 15fbf8d5de904fc4b284af664f3bd60f (table default.Table_655e9d02aacf4b88ba521f0f57a5f522 [id=6ed01835736b40f8b4caeca7378770d8]). Expected version 1 got 0
I20250115 03:06:19.859941 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_867788c99c794aadbb2dc125669537dc" } modify_external_catalogs: true
I20250115 03:06:19.860204 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_867788c99c794aadbb2dc125669537dc" } modify_external_catalogs: true
I20250115 03:06:19.866187 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 6132d15b17ab4d0aaa10a7ea04cda7ac
I20250115 03:06:19.866379 14811 tablet_bootstrap.cc:492] T 01433c76a398462bb6708a02997047fa P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.866452 14811 ts_tablet_manager.cc:1397] T 01433c76a398462bb6708a02997047fa P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.000s
I20250115 03:06:19.866727 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 6132d15b17ab4d0aaa10a7ea04cda7ac with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.866871 14811 raft_consensus.cc:357] T 01433c76a398462bb6708a02997047fa P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:19.866957 14811 raft_consensus.cc:383] T 01433c76a398462bb6708a02997047fa P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.866982 14811 raft_consensus.cc:738] T 01433c76a398462bb6708a02997047fa P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.867074 14811 consensus_queue.cc:260] T 01433c76a398462bb6708a02997047fa P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:19.867193 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 6132d15b17ab4d0aaa10a7ea04cda7ac with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.825568 14823 ts_tablet_manager.cc:1918] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:19.867282 14811 ts_tablet_manager.cc:1428] T 01433c76a398462bb6708a02997047fa P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.867326 14823 log.cc:1198] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/f8c3406d44a74b20a4ef4b42e8d04ad0
I20250115 03:06:19.867422 14811 tablet_bootstrap.cc:492] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.867604 14823 ts_tablet_manager.cc:1939] T f8c3406d44a74b20a4ef4b42e8d04ad0 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
W20250115 03:06:19.867590 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 6132d15b17ab4d0aaa10a7ea04cda7ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6132d15b17ab4d0aaa10a7ea04cda7ac already in progress: creating tablet
I20250115 03:06:19.868217 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet f8c3406d44a74b20a4ef4b42e8d04ad0 (table default.Table_ad994d4298c546df9e4808628545682a [id=f88b8c3027ae47ceacd99aa8a2811628]) successfully deleted
I20250115 03:06:19.868254 14811 tablet_bootstrap.cc:654] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.869817 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet 75c581f11be84e6293a3d14966fd584f with delete_type TABLET_DATA_DELETED (replaced by tablet 5c5e30d9b2ce4537b00b216ef1f2dd1c) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.870570 15153 raft_consensus.cc:491] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.870656 15153 raft_consensus.cc:513] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.871001 15153 leader_election.cc:290] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:19.871393 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5737f10aa50e4059a814585019346872" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:19.871524 14422 raft_consensus.cc:2463] T 5737f10aa50e4059a814585019346872 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:19.871588 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet 6132d15b17ab4d0aaa10a7ea04cda7ac with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.871765 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet 69533c30b57e45dab0b00239dfd90f13 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.871744 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5737f10aa50e4059a814585019346872" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:19.872123 14628 leader_election.cc:304] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:19.872525 15153 raft_consensus.cc:2798] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:19.872584 15153 raft_consensus.cc:491] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:19.872622 15153 raft_consensus.cc:3054] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:19.874073 14820 tablet_replica.cc:331] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.874194 14820 raft_consensus.cc:2238] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.874276 14820 raft_consensus.cc:2267] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.874601 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 4102586f32d94a998cf2807434ee0ebf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.874686 14820 ts_tablet_manager.cc:1905] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.874665 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet c7e9a1d1ada941b68688e0ed6742aa99 with delete_type TABLET_DATA_DELETED (replaced by tablet 238e68300f304266aaed94bc3b76a01d) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.874747 14819 tablet_replica.cc:331] T 4102586f32d94a998cf2807434ee0ebf P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.874790 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet d2564a20857f45d9a0c7e3b74eb58a91 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.874876 14819 raft_consensus.cc:2238] T 4102586f32d94a998cf2807434ee0ebf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.874969 14819 raft_consensus.cc:2267] T 4102586f32d94a998cf2807434ee0ebf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.875222 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 9f88a8cc2995427eb2082117e553812c (DEFAULT_TABLE table=default.Table_3f9710271c4c45a586f35f80a9e271c1 [id=23aab4c5c5eb407aa10e5a0eec3cd38d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.875381 14819 ts_tablet_manager.cc:1905] T 4102586f32d94a998cf2807434ee0ebf P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.875456 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f88a8cc2995427eb2082117e553812c. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:19.877022 14626 leader_election.cc:343] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.877242 15153 raft_consensus.cc:513] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.877286 14820 ts_tablet_manager.cc:1918] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.877347 14820 log.cc:1198] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/69533c30b57e45dab0b00239dfd90f13
I20250115 03:06:19.877523 14820 ts_tablet_manager.cc:1939] T 69533c30b57e45dab0b00239dfd90f13 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:19.877548 14819 ts_tablet_manager.cc:1918] T 4102586f32d94a998cf2807434ee0ebf P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.877554 15153 leader_election.cc:290] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:19.877601 14819 log.cc:1198] T 4102586f32d94a998cf2807434ee0ebf P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/4102586f32d94a998cf2807434ee0ebf
I20250115 03:06:19.877763 14819 ts_tablet_manager.cc:1939] T 4102586f32d94a998cf2807434ee0ebf P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.878064 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5737f10aa50e4059a814585019346872" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:19.878161 14422 raft_consensus.cc:3054] T 5737f10aa50e4059a814585019346872 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:19.878398 14819 tablet_replica.cc:331] T c7e9a1d1ada941b68688e0ed6742aa99 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.878528 14819 raft_consensus.cc:2238] T c7e9a1d1ada941b68688e0ed6742aa99 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.878523 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 4102586f32d94a998cf2807434ee0ebf (table default.Table_a1c9e28bac9c491e948bc3fc9b173bad [id=2fb8c68c76304deebd009b01eb078616]) successfully deleted
I20250115 03:06:19.878614 14819 raft_consensus.cc:2267] T c7e9a1d1ada941b68688e0ed6742aa99 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.878973 14422 raft_consensus.cc:2463] T 5737f10aa50e4059a814585019346872 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:19.879268 14628 leader_election.cc:304] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:19.879292 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 9f88a8cc2995427eb2082117e553812c (DEFAULT_TABLE table=default.Table_3f9710271c4c45a586f35f80a9e271c1 [id=23aab4c5c5eb407aa10e5a0eec3cd38d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.879491 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f88a8cc2995427eb2082117e553812c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.879973 14399 tablet_service.cc:1467] Processing CreateTablet for tablet e558fc096deb44bf9203248d98bea5cc (DEFAULT_TABLE table=default.Table_f2ff78f7081d4b56a30785f4d6ae8e71 [id=7a4016b430b049578159e70d6e2e237c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.880175 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e558fc096deb44bf9203248d98bea5cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.880369 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5737f10aa50e4059a814585019346872" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:19.880717 14653 tablet_service.cc:1467] Processing CreateTablet for tablet c4509d58adae4c81a6cb58e392f8fc9c (DEFAULT_TABLE table=default.Table_9b10ca11a6584a9da1f16a9bc803131b [id=089b4c05876844b2bf1c2d8f8f23efb2]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:19.880908 14626 leader_election.cc:343] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.881172 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4509d58adae4c81a6cb58e392f8fc9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.881513 14396 tablet_service.cc:1467] Processing CreateTablet for tablet c4509d58adae4c81a6cb58e392f8fc9c (DEFAULT_TABLE table=default.Table_9b10ca11a6584a9da1f16a9bc803131b [id=089b4c05876844b2bf1c2d8f8f23efb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.881744 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4509d58adae4c81a6cb58e392f8fc9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.881994 14858 raft_consensus.cc:491] T 01433c76a398462bb6708a02997047fa P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.882062 14858 raft_consensus.cc:513] T 01433c76a398462bb6708a02997047fa P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:19.882254 15390 raft_consensus.cc:491] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.882346 14858 leader_election.cc:290] T 01433c76a398462bb6708a02997047fa P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:19.882325 15390 raft_consensus.cc:513] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.882616 15390 leader_election.cc:290] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:19.882858 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01433c76a398462bb6708a02997047fa" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:19.882921 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2564a20857f45d9a0c7e3b74eb58a91" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:19.882917 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2564a20857f45d9a0c7e3b74eb58a91" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:19.883224 14358 leader_election.cc:343] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:19.883312 14356 leader_election.cc:343] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.883350 14356 leader_election.cc:304] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:19.883435 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01433c76a398462bb6708a02997047fa" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:19.883522 15390 raft_consensus.cc:2743] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:19.883869 14493 leader_election.cc:304] T 01433c76a398462bb6708a02997047fa P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:19.884064 14858 raft_consensus.cc:2743] T 01433c76a398462bb6708a02997047fa P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:19.884229 15153 raft_consensus.cc:2798] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:19.884307 15153 raft_consensus.cc:695] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:19.884243 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_eca9fde960bd4324a474bbda7f6e3317"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:19.884353 14819 ts_tablet_manager.cc:1905] T c7e9a1d1ada941b68688e0ed6742aa99 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.884411 15153 consensus_queue.cc:237] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:19.884578 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eca9fde960bd4324a474bbda7f6e3317 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.886157 14819 ts_tablet_manager.cc:1918] T c7e9a1d1ada941b68688e0ed6742aa99 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.886215 14819 log.cc:1198] T c7e9a1d1ada941b68688e0ed6742aa99 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/c7e9a1d1ada941b68688e0ed6742aa99
I20250115 03:06:19.886472 14819 ts_tablet_manager.cc:1939] T c7e9a1d1ada941b68688e0ed6742aa99 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.887068 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet c7e9a1d1ada941b68688e0ed6742aa99 (table default.Table_941290b0882541e185286447fca362b9 [id=a2689bcea23843c9bb9e1f08e335e58f]) successfully deleted
I20250115 03:06:19.887252 14819 tablet_replica.cc:331] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.887351 14819 raft_consensus.cc:2238] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.887428 14819 raft_consensus.cc:2267] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.888017 14809 tablet_bootstrap.cc:492] T 2c69daff62144976ae9082c89cc58eca P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.888106 14809 ts_tablet_manager.cc:1397] T 2c69daff62144976ae9082c89cc58eca P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.042s user 0.001s sys 0.001s
I20250115 03:06:19.888543 14809 raft_consensus.cc:357] T 2c69daff62144976ae9082c89cc58eca P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.888648 14809 raft_consensus.cc:383] T 2c69daff62144976ae9082c89cc58eca P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.888685 14809 raft_consensus.cc:738] T 2c69daff62144976ae9082c89cc58eca P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.888808 14809 consensus_queue.cc:260] T 2c69daff62144976ae9082c89cc58eca P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.889106 14809 ts_tablet_manager.cc:1428] T 2c69daff62144976ae9082c89cc58eca P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.889096 14171 catalog_manager.cc:5526] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:19.889297 14809 tablet_bootstrap.cc:492] T bfc34dc76ff94b8eaeb5c2822e6498da P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.890146 14809 tablet_bootstrap.cc:654] T bfc34dc76ff94b8eaeb5c2822e6498da P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.891309 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_83fe43c0616b4351b2b4457195378f0c" } modify_external_catalogs: true
I20250115 03:06:19.891462 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_83fe43c0616b4351b2b4457195378f0c" } modify_external_catalogs: true
I20250115 03:06:19.891556 14811 tablet_bootstrap.cc:492] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.891655 14811 ts_tablet_manager.cc:1397] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.000s
I20250115 03:06:19.892071 14811 raft_consensus.cc:357] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.892189 14811 raft_consensus.cc:383] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.892231 14811 raft_consensus.cc:738] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.892344 14811 consensus_queue.cc:260] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.892642 14811 ts_tablet_manager.cc:1428] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.892822 14811 tablet_bootstrap.cc:492] T 51b412277e634ea39bb6d2b90a43b875 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.893116 14858 raft_consensus.cc:491] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.893188 14858 raft_consensus.cc:513] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.893465 14858 leader_election.cc:290] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:19.894038 14823 tablet_replica.cc:331] T 98913be4c8e74d7185957b0c4ad6abce P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:19.894266 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_5a9c638d77a84fefb016a56854ac897f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.894568 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a9c638d77a84fefb016a56854ac897f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.895098 14819 ts_tablet_manager.cc:1905] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.895640 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c69daff62144976ae9082c89cc58eca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.896385 14823 raft_consensus.cc:2238] T 98913be4c8e74d7185957b0c4ad6abce P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.896984 14166 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_72e82b3e776e4d398fa5d8240afde936" } modify_external_catalogs: true
I20250115 03:06:19.897079 14166 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_72e82b3e776e4d398fa5d8240afde936" } modify_external_catalogs: true
I20250115 03:06:19.897871 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3efc2fb8427f493ea933347bec34f88f" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:19.897881 14819 ts_tablet_manager.cc:1918] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.898547 14819 log.cc:1198] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/d2564a20857f45d9a0c7e3b74eb58a91
I20250115 03:06:19.898568 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_c1f030f5a1ef49a8826714b1b730459a" } modify_external_catalogs: true
I20250115 03:06:19.898715 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_c1f030f5a1ef49a8826714b1b730459a" } modify_external_catalogs: true
I20250115 03:06:19.899127 14819 ts_tablet_manager.cc:1939] T d2564a20857f45d9a0c7e3b74eb58a91 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.899720 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet d2564a20857f45d9a0c7e3b74eb58a91 (table default.Table_7c8f638a9380432c844dc355d43df492 [id=6a9388b294f24f358a5d5f34568e8a13]) successfully deleted
I20250115 03:06:19.902352 14819 tablet_replica.cc:331] T 2c69daff62144976ae9082c89cc58eca P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.902518 14819 raft_consensus.cc:2238] T 2c69daff62144976ae9082c89cc58eca P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.902664 14819 raft_consensus.cc:2267] T 2c69daff62144976ae9082c89cc58eca P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.904140 14809 tablet_bootstrap.cc:492] T bfc34dc76ff94b8eaeb5c2822e6498da P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.904256 14809 ts_tablet_manager.cc:1397] T bfc34dc76ff94b8eaeb5c2822e6498da P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20250115 03:06:19.904676 14809 raft_consensus.cc:357] T bfc34dc76ff94b8eaeb5c2822e6498da P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.904825 14809 raft_consensus.cc:383] T bfc34dc76ff94b8eaeb5c2822e6498da P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.904891 14809 raft_consensus.cc:738] T bfc34dc76ff94b8eaeb5c2822e6498da P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.905084 14809 consensus_queue.cc:260] T bfc34dc76ff94b8eaeb5c2822e6498da P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.905386 14809 ts_tablet_manager.cc:1428] T bfc34dc76ff94b8eaeb5c2822e6498da P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.905594 14809 tablet_bootstrap.cc:492] T a73d45bf459c4137af6961b680d282ac P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.906114 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet d65bf3745b4f4d92a346b4dd18652432
I20250115 03:06:19.906576 14809 tablet_bootstrap.cc:654] T a73d45bf459c4137af6961b680d282ac P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.906764 15390 raft_consensus.cc:491] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.906844 15390 raft_consensus.cc:513] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.907152 15390 leader_election.cc:290] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:19.907301 14811 tablet_bootstrap.cc:654] T 51b412277e634ea39bb6d2b90a43b875 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.906775 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet d65bf3745b4f4d92a346b4dd18652432 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.907851 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada45621f8864caab9189039fae0fae2" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:19.908176 14819 ts_tablet_manager.cc:1905] T 2c69daff62144976ae9082c89cc58eca P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.908250 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet d643bd0947c54bcfa79f15d88e1e53e4
I20250115 03:06:19.908269 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet bfc34dc76ff94b8eaeb5c2822e6498da with delete_type TABLET_DATA_DELETED (replaced by tablet f2ca06c402cd46dc8311c8a47da92846) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:19.908643 14356 leader_election.cc:343] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.909570 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet d643bd0947c54bcfa79f15d88e1e53e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.910058 14819 ts_tablet_manager.cc:1918] T 2c69daff62144976ae9082c89cc58eca P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.910166 14819 log.cc:1198] T 2c69daff62144976ae9082c89cc58eca P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/2c69daff62144976ae9082c89cc58eca
I20250115 03:06:19.910346 14819 ts_tablet_manager.cc:1939] T 2c69daff62144976ae9082c89cc58eca P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.901096 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3efc2fb8427f493ea933347bec34f88f" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:19.912917 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 2c69daff62144976ae9082c89cc58eca (table default.Table_24b2f1f4bbb14d2fa7cba35e089f6707 [id=9c79f5b643664aebbc02a5d815485f61]) successfully deleted
I20250115 03:06:19.907588 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada45621f8864caab9189039fae0fae2" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:19.913080 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet d65bf3745b4f4d92a346b4dd18652432 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d65bf3745b4f4d92a346b4dd18652432 already in progress: creating tablet
I20250115 03:06:19.913345 14493 leader_election.cc:304] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:19.913569 14853 raft_consensus.cc:2743] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:19.913645 14358 leader_election.cc:343] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.913700 14358 leader_election.cc:304] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:19.908100 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet d65bf3745b4f4d92a346b4dd18652432 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.910526 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet d643bd0947c54bcfa79f15d88e1e53e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.912950 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 4febfae2f2794669b5997a1fa7ef7412
I20250115 03:06:19.913995 14885 raft_consensus.cc:2743] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:19.914254 14819 tablet_replica.cc:331] T bfc34dc76ff94b8eaeb5c2822e6498da P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.914372 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 4febfae2f2794669b5997a1fa7ef7412 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.914485 14823 raft_consensus.cc:2267] T 98913be4c8e74d7185957b0c4ad6abce P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.914570 14819 raft_consensus.cc:2238] T bfc34dc76ff94b8eaeb5c2822e6498da P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.914669 14819 raft_consensus.cc:2267] T bfc34dc76ff94b8eaeb5c2822e6498da P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.915040 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2e3e4b87e2e04feaab40ea415807cd06" } new_table_name: "default.Table_83b1fd83225a4454b29b7c61bc4639d4" modify_external_catalogs: true
I20250115 03:06:19.915355 14819 ts_tablet_manager.cc:1905] T bfc34dc76ff94b8eaeb5c2822e6498da P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.915809 14811 tablet_bootstrap.cc:492] T 51b412277e634ea39bb6d2b90a43b875 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.915884 14811 ts_tablet_manager.cc:1397] T 51b412277e634ea39bb6d2b90a43b875 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s
I20250115 03:06:19.916298 14811 raft_consensus.cc:357] T 51b412277e634ea39bb6d2b90a43b875 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.916399 14811 raft_consensus.cc:383] T 51b412277e634ea39bb6d2b90a43b875 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.916429 14811 raft_consensus.cc:738] T 51b412277e634ea39bb6d2b90a43b875 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.916538 14811 consensus_queue.cc:260] T 51b412277e634ea39bb6d2b90a43b875 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.916802 14811 ts_tablet_manager.cc:1428] T 51b412277e634ea39bb6d2b90a43b875 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.916981 14811 tablet_bootstrap.cc:492] T 5cb34cb2006f4716b87bef67f3a6dd8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.917815 14811 tablet_bootstrap.cc:654] T 5cb34cb2006f4716b87bef67f3a6dd8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.919901 14819 ts_tablet_manager.cc:1918] T bfc34dc76ff94b8eaeb5c2822e6498da P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.919967 14819 log.cc:1198] T bfc34dc76ff94b8eaeb5c2822e6498da P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/bfc34dc76ff94b8eaeb5c2822e6498da
I20250115 03:06:19.920141 14819 ts_tablet_manager.cc:1939] T bfc34dc76ff94b8eaeb5c2822e6498da P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.925081 14809 tablet_bootstrap.cc:492] T a73d45bf459c4137af6961b680d282ac P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.925150 14809 ts_tablet_manager.cc:1397] T a73d45bf459c4137af6961b680d282ac P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20250115 03:06:19.925536 14809 raft_consensus.cc:357] T a73d45bf459c4137af6961b680d282ac P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.925619 14809 raft_consensus.cc:383] T a73d45bf459c4137af6961b680d282ac P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.925643 14809 raft_consensus.cc:738] T a73d45bf459c4137af6961b680d282ac P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.925747 14809 consensus_queue.cc:260] T a73d45bf459c4137af6961b680d282ac P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.926020 14809 ts_tablet_manager.cc:1428] T a73d45bf459c4137af6961b680d282ac P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.926201 14809 tablet_bootstrap.cc:492] T 5589b497b0e24740ad49d1392f5a737b P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
W20250115 03:06:19.927173 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6bd4e4200f9b489ea5dd2154aa6f382d consensus state has no leader
I20250115 03:06:19.927362 14809 tablet_bootstrap.cc:654] T 5589b497b0e24740ad49d1392f5a737b P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.930461 14811 tablet_bootstrap.cc:492] T 5cb34cb2006f4716b87bef67f3a6dd8e P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.930541 14811 ts_tablet_manager.cc:1397] T 5cb34cb2006f4716b87bef67f3a6dd8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20250115 03:06:19.930747 14809 tablet_bootstrap.cc:492] T 5589b497b0e24740ad49d1392f5a737b P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.930814 14809 ts_tablet_manager.cc:1397] T 5589b497b0e24740ad49d1392f5a737b P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:19.930959 14811 raft_consensus.cc:357] T 5cb34cb2006f4716b87bef67f3a6dd8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.931047 14811 raft_consensus.cc:383] T 5cb34cb2006f4716b87bef67f3a6dd8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.931077 14811 raft_consensus.cc:738] T 5cb34cb2006f4716b87bef67f3a6dd8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.931167 14809 raft_consensus.cc:357] T 5589b497b0e24740ad49d1392f5a737b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.931181 14811 consensus_queue.cc:260] T 5cb34cb2006f4716b87bef67f3a6dd8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.931243 14809 raft_consensus.cc:383] T 5589b497b0e24740ad49d1392f5a737b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.931270 14809 raft_consensus.cc:738] T 5589b497b0e24740ad49d1392f5a737b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.931392 14809 consensus_queue.cc:260] T 5589b497b0e24740ad49d1392f5a737b P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.931474 14811 ts_tablet_manager.cc:1428] T 5cb34cb2006f4716b87bef67f3a6dd8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.931627 14811 tablet_bootstrap.cc:492] T 0f3bf849316a4ebd841c3345ca17087e P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.931646 14809 ts_tablet_manager.cc:1428] T 5589b497b0e24740ad49d1392f5a737b P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.932313 14811 tablet_bootstrap.cc:654] T 0f3bf849316a4ebd841c3345ca17087e P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.932408 14809 tablet_bootstrap.cc:492] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.933938 14811 tablet_bootstrap.cc:492] T 0f3bf849316a4ebd841c3345ca17087e P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.934013 14811 ts_tablet_manager.cc:1397] T 0f3bf849316a4ebd841c3345ca17087e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:19.934540 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet 9fb51a41e62a4094879887626d199a7c from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:19.934983 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet a73d45bf459c4137af6961b680d282ac with delete_type TABLET_DATA_DELETED (replaced by tablet 15263a3b32be418288770ce348c082d7) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.935184 14809 tablet_bootstrap.cc:654] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.936239 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2868f144930b418199ddf15146d94c13"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:19.936395 14811 raft_consensus.cc:357] T 0f3bf849316a4ebd841c3345ca17087e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.936484 14811 raft_consensus.cc:383] T 0f3bf849316a4ebd841c3345ca17087e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.936514 14811 raft_consensus.cc:738] T 0f3bf849316a4ebd841c3345ca17087e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
W20250115 03:06:19.936582 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2868f144930b418199ddf15146d94c13 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.936651 14811 consensus_queue.cc:260] T 0f3bf849316a4ebd841c3345ca17087e P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.936900 14811 ts_tablet_manager.cc:1428] T 0f3bf849316a4ebd841c3345ca17087e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:19.937095 14811 tablet_bootstrap.cc:492] T d12d1f964d2b486bb703ed98cb2e7949 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.937837 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet bfc34dc76ff94b8eaeb5c2822e6498da (table default.Table_a1547d1d923f4f4da05aaeb519f4cdf0 [id=e929ffaba567498c815393836b264a12]) successfully deleted
I20250115 03:06:19.937892 14819 tablet_replica.cc:331] T a73d45bf459c4137af6961b680d282ac P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
W20250115 03:06:19.937987 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet d643bd0947c54bcfa79f15d88e1e53e4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d643bd0947c54bcfa79f15d88e1e53e4 already in progress: creating tablet
I20250115 03:06:19.938001 14819 raft_consensus.cc:2238] T a73d45bf459c4137af6961b680d282ac P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.938087 14819 raft_consensus.cc:2267] T a73d45bf459c4137af6961b680d282ac P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:19.938131 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 4febfae2f2794669b5997a1fa7ef7412 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4febfae2f2794669b5997a1fa7ef7412 already in progress: creating tablet
I20250115 03:06:19.938165 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d7d37f44d845457e8a3656b4f7deabf2" } new_table_name: "default.Table_b49f54dbd6184a01a6565d15d71e14f1" modify_external_catalogs: true
I20250115 03:06:19.938482 14819 ts_tablet_manager.cc:1905] T a73d45bf459c4137af6961b680d282ac P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.939882 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 5589b497b0e24740ad49d1392f5a737b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.940501 14811 tablet_bootstrap.cc:654] T d12d1f964d2b486bb703ed98cb2e7949 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.941462 14809 tablet_bootstrap.cc:492] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:19.941589 14809 ts_tablet_manager.cc:1397] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:19.941998 14809 raft_consensus.cc:357] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.942135 14809 raft_consensus.cc:383] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.942199 14809 raft_consensus.cc:738] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.942359 14809 consensus_queue.cc:260] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.942646 14809 ts_tablet_manager.cc:1428] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.942905 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 9fb51a41e62a4094879887626d199a7c
I20250115 03:06:19.943125 14164 catalog_manager.cc:6596] ReplaceTablet: tablet 9fb51a41e62a4094879887626d199a7c deleted and replaced by tablet 9476eae94d664f5eb94865db2c95dca6
I20250115 03:06:19.944180 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fb51a41e62a4094879887626d199a7c with delete_type TABLET_DATA_DELETED (replaced by tablet 9476eae94d664f5eb94865db2c95dca6) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:19.945185 14164 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 9febec144bb343a8a21817e6d3655ad7 (table default.Table_57fbe77148284b4a9666630570f69321 [id=c9ed0b9c92ef40efa5c51cfdd8389325]). Expected version 2 got 0
I20250115 03:06:19.946770 14809 tablet_bootstrap.cc:492] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:19.947929 14809 tablet_bootstrap.cc:654] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.949745 14819 ts_tablet_manager.cc:1918] T a73d45bf459c4137af6961b680d282ac P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.949873 14819 log.cc:1198] T a73d45bf459c4137af6961b680d282ac P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/a73d45bf459c4137af6961b680d282ac
I20250115 03:06:19.950110 14819 ts_tablet_manager.cc:1939] T a73d45bf459c4137af6961b680d282ac P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.950341 14166 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 8b0f0539530643f78207e528c39530af (table default.Table_b49f54dbd6184a01a6565d15d71e14f1 [id=fe357a2a64fc4531870b305eb2480df1]). Expected version 3 got 2
I20250115 03:06:19.952184 14167 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 9febec144bb343a8a21817e6d3655ad7 (table default.Table_57fbe77148284b4a9666630570f69321 [id=c9ed0b9c92ef40efa5c51cfdd8389325]). Expected version 2 got 0
I20250115 03:06:19.953205 14396 tablet_service.cc:1467] Processing CreateTablet for tablet ef81ff8719614ae4a71f933d8538f21b (DEFAULT_TABLE table=default.Table_c1c66695a4cf4e7d8c148ac48b3f932b [id=0549020617024d74b3273662dde31867]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.953429 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef81ff8719614ae4a71f933d8538f21b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.953738 14811 tablet_bootstrap.cc:492] T d12d1f964d2b486bb703ed98cb2e7949 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.953845 14811 ts_tablet_manager.cc:1397] T d12d1f964d2b486bb703ed98cb2e7949 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20250115 03:06:19.954257 14811 raft_consensus.cc:357] T d12d1f964d2b486bb703ed98cb2e7949 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.954366 14811 raft_consensus.cc:383] T d12d1f964d2b486bb703ed98cb2e7949 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.954404 14811 raft_consensus.cc:738] T d12d1f964d2b486bb703ed98cb2e7949 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.954520 14811 consensus_queue.cc:260] T d12d1f964d2b486bb703ed98cb2e7949 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.956219 14819 tablet_replica.cc:331] T 5589b497b0e24740ad49d1392f5a737b P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:19.956326 14819 raft_consensus.cc:2238] T 5589b497b0e24740ad49d1392f5a737b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.956393 14819 raft_consensus.cc:2267] T 5589b497b0e24740ad49d1392f5a737b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.956544 14811 ts_tablet_manager.cc:1428] T d12d1f964d2b486bb703ed98cb2e7949 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:19.956691 14811 tablet_bootstrap.cc:492] T 907301ca08e0467f9f29c6dcb36bdd5e P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.957403 14811 tablet_bootstrap.cc:654] T 907301ca08e0467f9f29c6dcb36bdd5e P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.958323 14811 tablet_bootstrap.cc:492] T 907301ca08e0467f9f29c6dcb36bdd5e P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:19.958323 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet a73d45bf459c4137af6961b680d282ac (table default.Table_70287f2d5a024cc88244993789eb747b [id=88146f846aa74ecfb3977f01ea45df2f]) successfully deleted
I20250115 03:06:19.958403 14811 ts_tablet_manager.cc:1397] T 907301ca08e0467f9f29c6dcb36bdd5e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:19.958366 14396 tablet_service.cc:1467] Processing CreateTablet for tablet e24cd437a72f4333a234791c7240324a (DEFAULT_TABLE table=default.Table_9e4202d4703b406e8c5255ab8a0cc41f [id=50b13f1be1b34650ba2d4a2cfe1faad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.958595 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e24cd437a72f4333a234791c7240324a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.914867 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 4febfae2f2794669b5997a1fa7ef7412 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.943519 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fb51a41e62a4094879887626d199a7c with delete_type TABLET_DATA_DELETED (replaced by tablet 9476eae94d664f5eb94865db2c95dca6) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:19.953997 14655 tablet_service.cc:1467] Processing CreateTablet for tablet ef81ff8719614ae4a71f933d8538f21b (DEFAULT_TABLE table=default.Table_c1c66695a4cf4e7d8c148ac48b3f932b [id=0549020617024d74b3273662dde31867]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.958966 14811 raft_consensus.cc:357] T 907301ca08e0467f9f29c6dcb36bdd5e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.959053 14811 raft_consensus.cc:383] T 907301ca08e0467f9f29c6dcb36bdd5e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:19.959079 14811 raft_consensus.cc:738] T 907301ca08e0467f9f29c6dcb36bdd5e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:19.959187 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef81ff8719614ae4a71f933d8538f21b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.959200 14811 consensus_queue.cc:260] T 907301ca08e0467f9f29c6dcb36bdd5e P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.959429 14811 ts_tablet_manager.cc:1428] T 907301ca08e0467f9f29c6dcb36bdd5e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:19.960633 14811 tablet_bootstrap.cc:492] T 7d894471f7b94aec8a19df043906d7e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:19.961078 14819 ts_tablet_manager.cc:1905] T 5589b497b0e24740ad49d1392f5a737b P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.961544 14811 tablet_bootstrap.cc:654] T 7d894471f7b94aec8a19df043906d7e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:19.954249 14670 tablet_service.cc:1467] Processing CreateTablet for tablet e558fc096deb44bf9203248d98bea5cc (DEFAULT_TABLE table=default.Table_f2ff78f7081d4b56a30785f4d6ae8e71 [id=7a4016b430b049578159e70d6e2e237c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.962280 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e558fc096deb44bf9203248d98bea5cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.963989 14819 ts_tablet_manager.cc:1918] T 5589b497b0e24740ad49d1392f5a737b P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.964252 14819 log.cc:1198] T 5589b497b0e24740ad49d1392f5a737b P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/5589b497b0e24740ad49d1392f5a737b
I20250115 03:06:19.964445 14819 ts_tablet_manager.cc:1939] T 5589b497b0e24740ad49d1392f5a737b P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:19.958690 15153 raft_consensus.cc:491] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.958817 14654 tablet_service.cc:1467] Processing CreateTablet for tablet e24cd437a72f4333a234791c7240324a (DEFAULT_TABLE table=default.Table_9e4202d4703b406e8c5255ab8a0cc41f [id=50b13f1be1b34650ba2d4a2cfe1faad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.964720 15153 raft_consensus.cc:513] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.964864 14654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e24cd437a72f4333a234791c7240324a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.965065 15153 leader_election.cc:290] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:19.965977 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_68d93433744a4618ba532d361af70f09" } new_table_name: "default.Table_660872e60c104e56be9287f66d74c387" modify_external_catalogs: true
I20250115 03:06:19.966017 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 5589b497b0e24740ad49d1392f5a737b (table default.Table_8daa0abab4d1464488bec21bb7ad375b [id=37d965d8a5eb408ea3f33313bd22c152]) successfully deleted
W20250115 03:06:19.966424 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 9fb51a41e62a4094879887626d199a7c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9fb51a41e62a4094879887626d199a7c already in progress: creating tablet
I20250115 03:06:19.967173 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a92db72c6c4c4f8280f5329dc717930a" } new_table_name: "default.Table_f2fc8911242b4bc0af1b441861a4ff0a" modify_external_catalogs: true
I20250115 03:06:19.969086 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15fbf8d5de904fc4b284af664f3bd60f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:19.969211 14419 raft_consensus.cc:2463] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:19.969516 14628 leader_election.cc:304] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:19.969820 15153 raft_consensus.cc:2798] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:19.969841 14853 raft_consensus.cc:491] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.969874 15153 raft_consensus.cc:491] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:19.969908 15153 raft_consensus.cc:3054] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:19.969924 14853 raft_consensus.cc:513] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:19.970291 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15fbf8d5de904fc4b284af664f3bd60f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:19.970316 14853 leader_election.cc:290] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:19.970741 15153 raft_consensus.cc:513] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.971040 15153 leader_election.cc:290] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
W20250115 03:06:19.971194 14493 leader_election.cc:343] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 31446d60b85d4a878e9afe142011aeb1
W20250115 03:06:19.971773 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bfeafbf380df41738ebbcc11c19a7f85 consensus state has no leader
I20250115 03:06:19.972347 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15fbf8d5de904fc4b284af664f3bd60f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:19.972465 14419 raft_consensus.cc:3054] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:19.972451 14493 leader_election.cc:343] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 31446d60b85d4a878e9afe142011aeb1
I20250115 03:06:19.972519 14493 leader_election.cc:304] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:19.972412 14823 ts_tablet_manager.cc:1905] T 98913be4c8e74d7185957b0c4ad6abce P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.972800 14853 raft_consensus.cc:2743] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:19.973469 14419 raft_consensus.cc:2463] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:19.973773 14628 leader_election.cc:304] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:19.973922 16001 raft_consensus.cc:2798] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:19.974010 16001 raft_consensus.cc:695] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:19.974130 16001 consensus_queue.cc:237] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.974603 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15fbf8d5de904fc4b284af664f3bd60f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:19.977669 14823 ts_tablet_manager.cc:1918] T 98913be4c8e74d7185957b0c4ad6abce P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.977869 14823 log.cc:1198] T 98913be4c8e74d7185957b0c4ad6abce P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/98913be4c8e74d7185957b0c4ad6abce
I20250115 03:06:19.978250 15390 raft_consensus.cc:491] T fbd161e290914d9d8671c10becfeaee1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.978336 15390 raft_consensus.cc:513] T fbd161e290914d9d8671c10becfeaee1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.978613 15390 leader_election.cc:290] T fbd161e290914d9d8671c10becfeaee1 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:19.978871 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd161e290914d9d8671c10becfeaee1" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:19.978989 14626 leader_election.cc:343] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.979099 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd161e290914d9d8671c10becfeaee1" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:19.979399 14358 leader_election.cc:343] T fbd161e290914d9d8671c10becfeaee1 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:19.979480 14356 leader_election.cc:343] T fbd161e290914d9d8671c10becfeaee1 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.979543 14356 leader_election.cc:304] T fbd161e290914d9d8671c10becfeaee1 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:19.980021 15390 raft_consensus.cc:2743] T fbd161e290914d9d8671c10becfeaee1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:19.980224 14626 leader_election.cc:343] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.982316 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ab37ce65c0ae4e06b3e72a0e843d8f21" } new_table_name: "default.Table_c920b2b2aef445488633aa45f8868aa8" modify_external_catalogs: true
W20250115 03:06:19.982710 14167 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f972872b022344418e9517131c27a22d consensus state has no leader
I20250115 03:06:19.983542 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 9f88a8cc2995427eb2082117e553812c (DEFAULT_TABLE table=default.Table_3f9710271c4c45a586f35f80a9e271c1 [id=23aab4c5c5eb407aa10e5a0eec3cd38d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.983776 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f88a8cc2995427eb2082117e553812c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.985846 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet d65bf3745b4f4d92a346b4dd18652432 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.985877 14535 tablet_service.cc:1467] Processing CreateTablet for tablet c4509d58adae4c81a6cb58e392f8fc9c (DEFAULT_TABLE table=default.Table_9b10ca11a6584a9da1f16a9bc803131b [id=089b4c05876844b2bf1c2d8f8f23efb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.986099 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4509d58adae4c81a6cb58e392f8fc9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.986377 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet d643bd0947c54bcfa79f15d88e1e53e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.986711 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet 4febfae2f2794669b5997a1fa7ef7412 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.987107 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet 9fb51a41e62a4094879887626d199a7c with delete_type TABLET_DATA_DELETED (replaced by tablet 9476eae94d664f5eb94865db2c95dca6) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.987828 14532 tablet_service.cc:1467] Processing CreateTablet for tablet e24cd437a72f4333a234791c7240324a (DEFAULT_TABLE table=default.Table_9e4202d4703b406e8c5255ab8a0cc41f [id=50b13f1be1b34650ba2d4a2cfe1faad2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.988037 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e24cd437a72f4333a234791c7240324a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.988191 14530 tablet_service.cc:1467] Processing CreateTablet for tablet ef81ff8719614ae4a71f933d8538f21b (DEFAULT_TABLE table=default.Table_c1c66695a4cf4e7d8c148ac48b3f932b [id=0549020617024d74b3273662dde31867]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.988399 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef81ff8719614ae4a71f933d8538f21b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.988770 14168 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 01433c76a398462bb6708a02997047fa (table default.Table_3ca292c29f184579b5fd9fe897150035 [id=960713d5552d4e339aa8c94edbe9c7b7]). Expected version 1 got 0
W20250115 03:06:19.989616 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 75c581f11be84e6293a3d14966fd584f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75c581f11be84e6293a3d14966fd584f already in progress: creating tablet
W20250115 03:06:19.989743 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 6132d15b17ab4d0aaa10a7ea04cda7ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6132d15b17ab4d0aaa10a7ea04cda7ac already in progress: creating tablet
I20250115 03:06:19.989838 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 69533c30b57e45dab0b00239dfd90f13 (table default.Table_d76368b5971a4d08a167c7e7bac8144e [id=3968ef424a944c9c83aacaea3ccc7c0d]) successfully deleted
W20250115 03:06:19.990000 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet d65bf3745b4f4d92a346b4dd18652432 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d65bf3745b4f4d92a346b4dd18652432 already in progress: creating tablet
W20250115 03:06:19.990104 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet d643bd0947c54bcfa79f15d88e1e53e4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d643bd0947c54bcfa79f15d88e1e53e4 already in progress: creating tablet
W20250115 03:06:19.990206 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 4febfae2f2794669b5997a1fa7ef7412 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4febfae2f2794669b5997a1fa7ef7412 already in progress: creating tablet
W20250115 03:06:19.990301 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 9fb51a41e62a4094879887626d199a7c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9fb51a41e62a4094879887626d199a7c already in progress: creating tablet
I20250115 03:06:19.990413 15390 raft_consensus.cc:491] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:19.990403 14521 tablet_service.cc:1467] Processing CreateTablet for tablet e558fc096deb44bf9203248d98bea5cc (DEFAULT_TABLE table=default.Table_f2ff78f7081d4b56a30785f4d6ae8e71 [id=7a4016b430b049578159e70d6e2e237c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:19.990494 15390 raft_consensus.cc:513] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:19.990599 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e558fc096deb44bf9203248d98bea5cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:19.990826 15390 leader_election.cc:290] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:19.991032 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bcfde5fe3d4425e98edb4dc54096718" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:19.991257 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bcfde5fe3d4425e98edb4dc54096718" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:19.991534 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet cff95c69fb9045cf9ec0ebb128c9ef6b from {username='slave'} at 127.0.0.1:60432
W20250115 03:06:19.991565 14358 leader_election.cc:343] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:19.991650 14356 leader_election.cc:343] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:19.991701 14356 leader_election.cc:304] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:19.991917 15390 raft_consensus.cc:2743] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:19.992050 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_db820168492749099144481f4895d059" } modify_external_catalogs: true
I20250115 03:06:19.992120 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 31446d60b85d4a878e9afe142011aeb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:19.992141 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_db820168492749099144481f4895d059" } modify_external_catalogs: true
I20250115 03:06:19.992246 14820 tablet_replica.cc:331] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:19.992362 14820 raft_consensus.cc:2238] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:19.992471 14820 raft_consensus.cc:2267] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:19.992408 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_472a764a0bfe4cf7ad316cc6de021448"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.992724 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_472a764a0bfe4cf7ad316cc6de021448 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.992885 14820 ts_tablet_manager.cc:1905] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:19.992973 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_3386c15b41014720beb4e38c88d64b08"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:19.993252 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3386c15b41014720beb4e38c88d64b08 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:19.994982 14820 ts_tablet_manager.cc:1918] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:19.995029 14820 log.cc:1198] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/31446d60b85d4a878e9afe142011aeb1
I20250115 03:06:19.995158 14820 ts_tablet_manager.cc:1939] T 31446d60b85d4a878e9afe142011aeb1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
W20250115 03:06:19.997515 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet af712cc03750455087ecd381cc97d464 consensus state has no leader
I20250115 03:06:20.000654 14812 tablet_bootstrap.cc:492] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.000762 14812 ts_tablet_manager.cc:1397] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.149s user 0.000s sys 0.002s
I20250115 03:06:20.001277 14812 raft_consensus.cc:357] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.001375 14812 raft_consensus.cc:383] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.001405 14812 raft_consensus.cc:738] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.001530 14812 consensus_queue.cc:260] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.001829 14812 ts_tablet_manager.cc:1428] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:20.002013 14812 tablet_bootstrap.cc:492] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.002924 14812 tablet_bootstrap.cc:654] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.003655 14823 ts_tablet_manager.cc:1939] T 98913be4c8e74d7185957b0c4ad6abce P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:20.006424 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 98913be4c8e74d7185957b0c4ad6abce (table default.Table_71fadacb26bd4e4a8ddcd48e403350bb [id=529f6a8366ca4afd8e51cc396a8bb7a1]) successfully deleted
W20250115 03:06:20.006551 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 75c581f11be84e6293a3d14966fd584f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75c581f11be84e6293a3d14966fd584f already in progress: creating tablet
W20250115 03:06:20.006635 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 6132d15b17ab4d0aaa10a7ea04cda7ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6132d15b17ab4d0aaa10a7ea04cda7ac already in progress: creating tablet
W20250115 03:06:20.006704 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet d643bd0947c54bcfa79f15d88e1e53e4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d643bd0947c54bcfa79f15d88e1e53e4 already in progress: creating tablet
W20250115 03:06:20.006783 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet d65bf3745b4f4d92a346b4dd18652432 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d65bf3745b4f4d92a346b4dd18652432 already in progress: creating tablet
W20250115 03:06:20.006851 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 9fb51a41e62a4094879887626d199a7c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9fb51a41e62a4094879887626d199a7c already in progress: creating tablet
W20250115 03:06:20.006942 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 4febfae2f2794669b5997a1fa7ef7412 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4febfae2f2794669b5997a1fa7ef7412 already in progress: creating tablet
I20250115 03:06:20.008347 15390 raft_consensus.cc:491] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.008435 15390 raft_consensus.cc:513] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.008754 15390 leader_election.cc:290] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:20.009279 14171 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 5cb34cb2006f4716b87bef67f3a6dd8e (table default.Table_77b3fb817e8b4502af7fae7bcb0124a5 [id=ed28aa2aaacd4ba58dcb3216a7156a91]). Expected version 1 got 0
I20250115 03:06:20.009352 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21524aef80f74a6c899ea66b9b022389" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:20.009358 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21524aef80f74a6c899ea66b9b022389" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:20.009701 14356 leader_election.cc:343] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:20.010258 14358 leader_election.cc:343] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.010337 14358 leader_election.cc:304] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:20.010941 15390 raft_consensus.cc:2743] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.011253 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet f66defba523f4df996929f98c848f95a from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:20.011303 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 31446d60b85d4a878e9afe142011aeb1 (table default.Table_717056a529de4818b9c40f37ccb2083b [id=fa31aef04ed04e58888ba86469303e0b]) successfully deleted
I20250115 03:06:20.011662 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet 82bf1d7ac9fe4a2eb7b57cf1ab627c3d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.011828 14820 tablet_replica.cc:331] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.011961 14820 raft_consensus.cc:2238] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.011997 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet d12d1f964d2b486bb703ed98cb2e7949 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.012056 14820 raft_consensus.cc:2267] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.011986 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3bf849316a4ebd841c3345ca17087e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.012502 14820 ts_tablet_manager.cc:1905] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.014467 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet f66defba523f4df996929f98c848f95a
I20250115 03:06:20.014703 14820 ts_tablet_manager.cc:1918] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.014762 14820 log.cc:1198] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/82bf1d7ac9fe4a2eb7b57cf1ab627c3d
I20250115 03:06:20.014786 14171 catalog_manager.cc:6596] ReplaceTablet: tablet f66defba523f4df996929f98c848f95a deleted and replaced by tablet b9979224e3874a7982e1bada8395c921
I20250115 03:06:20.014950 14820 ts_tablet_manager.cc:1939] T 82bf1d7ac9fe4a2eb7b57cf1ab627c3d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.015503 14820 tablet_replica.cc:331] T d12d1f964d2b486bb703ed98cb2e7949 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.015627 14820 raft_consensus.cc:2238] T d12d1f964d2b486bb703ed98cb2e7949 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.015723 14820 raft_consensus.cc:2267] T d12d1f964d2b486bb703ed98cb2e7949 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.016219 14820 ts_tablet_manager.cc:1905] T d12d1f964d2b486bb703ed98cb2e7949 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.017009 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet f66defba523f4df996929f98c848f95a with delete_type TABLET_DATA_DELETED (replaced by tablet b9979224e3874a7982e1bada8395c921) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.017431 15153 raft_consensus.cc:491] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.017510 15153 raft_consensus.cc:513] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.017867 15153 leader_election.cc:290] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:20.018141 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bcfde5fe3d4425e98edb4dc54096718" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:20.018152 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bcfde5fe3d4425e98edb4dc54096718" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:20.018268 14419 raft_consensus.cc:2463] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:20.018276 14820 ts_tablet_manager.cc:1918] T d12d1f964d2b486bb703ed98cb2e7949 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.018327 14820 log.cc:1198] T d12d1f964d2b486bb703ed98cb2e7949 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/d12d1f964d2b486bb703ed98cb2e7949
W20250115 03:06:20.018386 14626 leader_election.cc:343] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.018476 14654 tablet_service.cc:1514] Processing DeleteTablet for tablet f66defba523f4df996929f98c848f95a with delete_type TABLET_DATA_DELETED (replaced by tablet b9979224e3874a7982e1bada8395c921) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.018577 14820 ts_tablet_manager.cc:1939] T d12d1f964d2b486bb703ed98cb2e7949 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.018577 14628 leader_election.cc:304] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:20.019032 14820 tablet_replica.cc:331] T 0f3bf849316a4ebd841c3345ca17087e P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
W20250115 03:06:20.019033 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet f66defba523f4df996929f98c848f95a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f66defba523f4df996929f98c848f95a already in progress: creating tablet
I20250115 03:06:20.019160 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 82bf1d7ac9fe4a2eb7b57cf1ab627c3d (table default.Table_0ab5bfc343e6431890e7f64f444e1c05 [id=2f9b0aa8377744488a89ba41fb20d5c3]) successfully deleted
I20250115 03:06:20.019183 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet c2afa048b3fd4362b50c8a6da306c25a from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:20.019275 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet d12d1f964d2b486bb703ed98cb2e7949 (table default.Table_0e5754dbfde84156b74d86bba62a0046 [id=739ec7394ea24ccb849d2486e451496b]) successfully deleted
I20250115 03:06:20.019284 15153 raft_consensus.cc:2798] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:20.019330 15153 raft_consensus.cc:491] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:20.019372 15153 raft_consensus.cc:3054] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.019205 14820 raft_consensus.cc:2238] T 0f3bf849316a4ebd841c3345ca17087e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.019675 14820 raft_consensus.cc:2267] T 0f3bf849316a4ebd841c3345ca17087e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.020133 14820 ts_tablet_manager.cc:1905] T 0f3bf849316a4ebd841c3345ca17087e P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.020220 15153 raft_consensus.cc:513] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.020545 15153 leader_election.cc:290] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:20.020983 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bcfde5fe3d4425e98edb4dc54096718" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:20.020982 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bcfde5fe3d4425e98edb4dc54096718" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:20.021093 14419 raft_consensus.cc:3054] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.021225 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet f66defba523f4df996929f98c848f95a with delete_type TABLET_DATA_DELETED (replaced by tablet b9979224e3874a7982e1bada8395c921) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.021250 14812 tablet_bootstrap.cc:492] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.021327 14812 ts_tablet_manager.cc:1397] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.002s
W20250115 03:06:20.021816 14626 leader_election.cc:343] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.022107 14419 raft_consensus.cc:2463] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:20.022118 14812 raft_consensus.cc:357] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.022277 14820 ts_tablet_manager.cc:1918] T 0f3bf849316a4ebd841c3345ca17087e P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.022280 14812 raft_consensus.cc:383] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.022325 14812 raft_consensus.cc:738] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.022413 14820 log.cc:1198] T 0f3bf849316a4ebd841c3345ca17087e P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/0f3bf849316a4ebd841c3345ca17087e
I20250115 03:06:20.022486 14628 leader_election.cc:304] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:20.022462 14812 consensus_queue.cc:260] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.022735 16088 raft_consensus.cc:2798] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:20.022812 16088 raft_consensus.cc:695] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:20.022879 14812 ts_tablet_manager.cc:1428] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:20.022922 16088 consensus_queue.cc:237] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.023573 14820 ts_tablet_manager.cc:1939] T 0f3bf849316a4ebd841c3345ca17087e P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.027900 14812 tablet_bootstrap.cc:492] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.029247 14812 tablet_bootstrap.cc:654] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.029827 15970 tablet.cc:1721] T 8b0f0539530643f78207e528c39530af P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:20.030225 15617 tablet.cc:1721] T 8b0f0539530643f78207e528c39530af P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:20.030639 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 0f3bf849316a4ebd841c3345ca17087e (table default.Table_96c8d5f733034ee3a6dd8a61dabfd288 [id=ff4681cab8ce44ceb45ed76c70c1132e]) successfully deleted
W20250115 03:06:20.030887 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet f66defba523f4df996929f98c848f95a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f66defba523f4df996929f98c848f95a already in progress: creating tablet
I20250115 03:06:20.031018 16185 tablet.cc:1721] T 8b0f0539530643f78207e528c39530af P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:20.031957 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_9a8c2b74534f4bf2b5fc30dff185e54c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:20.032960 14809 tablet_bootstrap.cc:492] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.033042 14809 ts_tablet_manager.cc:1397] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.086s user 0.002s sys 0.000s
I20250115 03:06:20.033525 14809 raft_consensus.cc:357] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.033632 14809 raft_consensus.cc:383] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.033661 14809 raft_consensus.cc:738] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.033775 14809 consensus_queue.cc:260] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.034029 14809 ts_tablet_manager.cc:1428] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.034982 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_10784ed50a0948d7a64a54cffcda719e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:20.035111 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_802ff80c57304d6393caa12f80fb4929"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:20.037684 14811 tablet_bootstrap.cc:492] T 7d894471f7b94aec8a19df043906d7e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.037767 14811 ts_tablet_manager.cc:1397] T 7d894471f7b94aec8a19df043906d7e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.077s user 0.002s sys 0.000s
I20250115 03:06:20.038182 14811 raft_consensus.cc:357] T 7d894471f7b94aec8a19df043906d7e7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.038283 14811 raft_consensus.cc:383] T 7d894471f7b94aec8a19df043906d7e7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.038314 14811 raft_consensus.cc:738] T 7d894471f7b94aec8a19df043906d7e7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.038417 14811 consensus_queue.cc:260] T 7d894471f7b94aec8a19df043906d7e7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.038671 14811 ts_tablet_manager.cc:1428] T 7d894471f7b94aec8a19df043906d7e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.038769 14809 tablet_bootstrap.cc:492] T 0521973e23984355a77a321b4676b479 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.038841 14811 tablet_bootstrap.cc:492] T 1f2dc6928e2941ef9b5b5a7891376500 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.039729 14809 tablet_bootstrap.cc:654] T 0521973e23984355a77a321b4676b479 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.042677 14811 tablet_bootstrap.cc:654] T 1f2dc6928e2941ef9b5b5a7891376500 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.045763 14812 tablet_bootstrap.cc:492] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.046245 14811 tablet_bootstrap.cc:492] T 1f2dc6928e2941ef9b5b5a7891376500 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.046324 14811 ts_tablet_manager.cc:1397] T 1f2dc6928e2941ef9b5b5a7891376500 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20250115 03:06:20.046327 14169 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_b49f54dbd6184a01a6565d15d71e14f1 [id=fe357a2a64fc4531870b305eb2480df1] alter complete (version 3)
I20250115 03:06:20.046501 14812 ts_tablet_manager.cc:1397] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.002s
I20250115 03:06:20.046712 14811 raft_consensus.cc:357] T 1f2dc6928e2941ef9b5b5a7891376500 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.046808 14811 raft_consensus.cc:383] T 1f2dc6928e2941ef9b5b5a7891376500 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.046834 14811 raft_consensus.cc:738] T 1f2dc6928e2941ef9b5b5a7891376500 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.046937 14811 consensus_queue.cc:260] T 1f2dc6928e2941ef9b5b5a7891376500 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.047181 14811 ts_tablet_manager.cc:1428] T 1f2dc6928e2941ef9b5b5a7891376500 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.047319 14811 tablet_bootstrap.cc:492] T cff95c69fb9045cf9ec0ebb128c9ef6b P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.047523 14812 raft_consensus.cc:357] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.047716 14812 raft_consensus.cc:383] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.047780 14812 raft_consensus.cc:738] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.048079 14811 tablet_bootstrap.cc:654] T cff95c69fb9045cf9ec0ebb128c9ef6b P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.048003 14812 consensus_queue.cc:260] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.048362 14812 ts_tablet_manager.cc:1428] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:20.048524 14812 tablet_bootstrap.cc:492] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.049410 14812 tablet_bootstrap.cc:654] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:20.049786 14626 consensus_peers.cc:487] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20250115 03:06:20.052093 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9a8c2b74534f4bf2b5fc30dff185e54c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:20.052174 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_802ff80c57304d6393caa12f80fb4929 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.052198 14811 tablet_bootstrap.cc:492] T cff95c69fb9045cf9ec0ebb128c9ef6b P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.052271 14811 ts_tablet_manager.cc:1397] T cff95c69fb9045cf9ec0ebb128c9ef6b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20250115 03:06:20.052695 14811 raft_consensus.cc:357] T cff95c69fb9045cf9ec0ebb128c9ef6b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.052809 14811 raft_consensus.cc:383] T cff95c69fb9045cf9ec0ebb128c9ef6b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.052850 14811 raft_consensus.cc:738] T cff95c69fb9045cf9ec0ebb128c9ef6b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
W20250115 03:06:20.019181 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet f66defba523f4df996929f98c848f95a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f66defba523f4df996929f98c848f95a already in progress: creating tablet
I20250115 03:06:20.052980 14811 consensus_queue.cc:260] T cff95c69fb9045cf9ec0ebb128c9ef6b P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.053241 14811 ts_tablet_manager.cc:1428] T cff95c69fb9045cf9ec0ebb128c9ef6b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.053400 14811 tablet_bootstrap.cc:492] T d5655935aabd48a19d0ef03739e089a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.054234 14811 tablet_bootstrap.cc:654] T d5655935aabd48a19d0ef03739e089a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:20.054833 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_10784ed50a0948d7a64a54cffcda719e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.061906 14811 tablet_bootstrap.cc:492] T d5655935aabd48a19d0ef03739e089a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.061991 14811 ts_tablet_manager.cc:1397] T d5655935aabd48a19d0ef03739e089a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20250115 03:06:20.063642 14812 tablet_bootstrap.cc:492] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.063766 14812 ts_tablet_manager.cc:1397] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.002s
I20250115 03:06:20.062398 14811 raft_consensus.cc:357] T d5655935aabd48a19d0ef03739e089a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.064098 14811 raft_consensus.cc:383] T d5655935aabd48a19d0ef03739e089a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.064047 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ece95328be52409b94bde437fffffc25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:20.064148 14811 raft_consensus.cc:738] T d5655935aabd48a19d0ef03739e089a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
W20250115 03:06:20.064363 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ece95328be52409b94bde437fffffc25 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.064736 14812 raft_consensus.cc:357] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.064882 14812 raft_consensus.cc:383] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.065038 14809 tablet_bootstrap.cc:492] T 0521973e23984355a77a321b4676b479 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.065126 14809 ts_tablet_manager.cc:1397] T 0521973e23984355a77a321b4676b479 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20250115 03:06:20.065582 14809 raft_consensus.cc:357] T 0521973e23984355a77a321b4676b479 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.065681 14809 raft_consensus.cc:383] T 0521973e23984355a77a321b4676b479 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.065712 14809 raft_consensus.cc:738] T 0521973e23984355a77a321b4676b479 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.065814 14809 consensus_queue.cc:260] T 0521973e23984355a77a321b4676b479 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.066084 14809 ts_tablet_manager.cc:1428] T 0521973e23984355a77a321b4676b479 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.066228 14809 tablet_bootstrap.cc:492] T 66b748fc853f499fafc1f476c8f79b8b P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.067083 14809 tablet_bootstrap.cc:654] T 66b748fc853f499fafc1f476c8f79b8b P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.067708 14812 raft_consensus.cc:738] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.067919 14812 consensus_queue.cc:260] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.068293 16001 raft_consensus.cc:491] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.068368 16001 raft_consensus.cc:513] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.068464 14811 consensus_queue.cc:260] T d5655935aabd48a19d0ef03739e089a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.068686 16001 leader_election.cc:290] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:20.069294 14812 ts_tablet_manager.cc:1428] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20250115 03:06:20.069442 14812 tablet_bootstrap.cc:492] T 4102586f32d94a998cf2807434ee0ebf P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.070242 14811 ts_tablet_manager.cc:1428] T d5655935aabd48a19d0ef03739e089a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20250115 03:06:20.070401 14811 tablet_bootstrap.cc:492] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.070705 14812 tablet_bootstrap.cc:654] T 4102586f32d94a998cf2807434ee0ebf P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:20.071151 14628 leader_election.cc:343] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 623887ec17374f598f48054e17582f3f
I20250115 03:06:20.071277 14811 tablet_bootstrap.cc:654] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.072649 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "623887ec17374f598f48054e17582f3f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:20.078347 16001 consensus_queue.cc:1035] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:20.078557 14626 leader_election.cc:343] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.078629 14626 leader_election.cc:304] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:20.078969 16001 raft_consensus.cc:2743] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.079299 14812 tablet_bootstrap.cc:492] T 4102586f32d94a998cf2807434ee0ebf P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.079375 14812 ts_tablet_manager.cc:1397] T 4102586f32d94a998cf2807434ee0ebf P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20250115 03:06:20.079727 14812 raft_consensus.cc:357] T 4102586f32d94a998cf2807434ee0ebf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.079820 14812 raft_consensus.cc:383] T 4102586f32d94a998cf2807434ee0ebf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.079854 14812 raft_consensus.cc:738] T 4102586f32d94a998cf2807434ee0ebf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.079957 14812 consensus_queue.cc:260] T 4102586f32d94a998cf2807434ee0ebf P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.080202 14812 ts_tablet_manager.cc:1428] T 4102586f32d94a998cf2807434ee0ebf P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:20.080360 14812 tablet_bootstrap.cc:492] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.080827 14809 tablet_bootstrap.cc:492] T 66b748fc853f499fafc1f476c8f79b8b P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.080896 14809 ts_tablet_manager.cc:1397] T 66b748fc853f499fafc1f476c8f79b8b P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20250115 03:06:20.081264 14812 tablet_bootstrap.cc:654] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.081328 14809 raft_consensus.cc:357] T 66b748fc853f499fafc1f476c8f79b8b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.081415 14809 raft_consensus.cc:383] T 66b748fc853f499fafc1f476c8f79b8b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.081441 14809 raft_consensus.cc:738] T 66b748fc853f499fafc1f476c8f79b8b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.081563 14809 consensus_queue.cc:260] T 66b748fc853f499fafc1f476c8f79b8b P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.081835 14809 ts_tablet_manager.cc:1428] T 66b748fc853f499fafc1f476c8f79b8b P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.082034 14809 tablet_bootstrap.cc:492] T adbb0ae719f24dcc96709a83981e213d P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.082962 14809 tablet_bootstrap.cc:654] T adbb0ae719f24dcc96709a83981e213d P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.087280 14812 tablet_bootstrap.cc:492] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.087409 14812 ts_tablet_manager.cc:1397] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20250115 03:06:20.087868 14812 raft_consensus.cc:357] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.088011 14812 raft_consensus.cc:383] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.088086 14812 raft_consensus.cc:738] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.088255 14812 consensus_queue.cc:260] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.088621 14812 ts_tablet_manager.cc:1428] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.090688 14812 tablet_bootstrap.cc:492] T d2564a20857f45d9a0c7e3b74eb58a91 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.091567 14812 tablet_bootstrap.cc:654] T d2564a20857f45d9a0c7e3b74eb58a91 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.092252 14166 catalog_manager.cc:5526] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:20.092451 14166 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 15fbf8d5de904fc4b284af664f3bd60f (table default.Table_655e9d02aacf4b88ba521f0f57a5f522 [id=6ed01835736b40f8b4caeca7378770d8]). Expected version 1 got 0
I20250115 03:06:20.093778 14654 tablet_service.cc:1467] Processing CreateTablet for tablet ffc493e8444147018bdf50382dc95b63 (DEFAULT_TABLE table=default.Table_2868f144930b418199ddf15146d94c13 [id=036a621d8a314a6480abd793aad707eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.094024 14654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffc493e8444147018bdf50382dc95b63. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.094020 14396 tablet_service.cc:1467] Processing CreateTablet for tablet ffc493e8444147018bdf50382dc95b63 (DEFAULT_TABLE table=default.Table_2868f144930b418199ddf15146d94c13 [id=036a621d8a314a6480abd793aad707eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.094277 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffc493e8444147018bdf50382dc95b63. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.094308 14653 tablet_service.cc:1467] Processing CreateTablet for tablet fe0dd7895a6547e8a105492083f2ec22 (DEFAULT_TABLE table=default.Table_5a9c638d77a84fefb016a56854ac897f [id=ac3c3d9a44e248649310cc277565097d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.094509 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe0dd7895a6547e8a105492083f2ec22. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.096378 14399 tablet_service.cc:1467] Processing CreateTablet for tablet fe0dd7895a6547e8a105492083f2ec22 (DEFAULT_TABLE table=default.Table_5a9c638d77a84fefb016a56854ac897f [id=ac3c3d9a44e248649310cc277565097d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.096592 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe0dd7895a6547e8a105492083f2ec22. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.100090 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 5076e95bfbac445899cb8ae88b49969a (DEFAULT_TABLE table=default.Table_4ec1365f1a874eb49b19337126347443 [id=57db97740d53455ea9894ea4d707ab2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.100306 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5076e95bfbac445899cb8ae88b49969a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.100564 14390 tablet_service.cc:1467] Processing CreateTablet for tablet b06b9691451f4a88b07044fd626aa6d0 (DEFAULT_TABLE table=default.Table_b45108a844cb400cbc6220e82aaafb9e [id=e0cddff8b58344afbd8e4f4a1c3daae4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.100775 14391 tablet_service.cc:1467] Processing CreateTablet for tablet 9476eae94d664f5eb94865db2c95dca6 (DEFAULT_TABLE table=default.Table_75817b46a6414888988ff7373ea11b27 [id=dfa93dac2ee14cfd8fe486869349c288]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.100798 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 6004f4adca8e47ddb91f7e39c4cf9089 (DEFAULT_TABLE table=default.Table_64d33ab8807d44a996a1f0fcc2deb7e2 [id=62fd48d944c24d15962e1708c7b71b27]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.101016 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 118dd31cc927421ba837ca44766cc925 (DEFAULT_TABLE table=default.Table_2ec77dbcffce476aa7f1d98daa698e84 [id=e8f7ed89657b4465a78feb971b65acb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.101112 14397 tablet_service.cc:1467] Processing CreateTablet for tablet ec1a8d0326f84ca0a87851010c1e68b7 (DEFAULT_TABLE table=default.Table_ab72071c412848939b54fcc72d5974fa [id=aff8cb5016d84881917b303cb3c2e4d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.101156 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 34bd3886562742bc980c571ca76230fe (DEFAULT_TABLE table=default.Table_e303383ddf3845828de6837219313014 [id=654dfdc4477b476f80e6af5b066713c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.101112 14395 tablet_service.cc:1467] Processing CreateTablet for tablet 5c5e30d9b2ce4537b00b216ef1f2dd1c (DEFAULT_TABLE table=default.Table_cb51431eaddb48858a61d248b688bbe7 [id=5624f7f1c61140a798718c8c85da14ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.101636 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 9eb56ebdb1a24b8ea8fbc4c129fded95 (DEFAULT_TABLE table=default.Table_eca9fde960bd4324a474bbda7f6e3317 [id=f80db4b835b24d09bb76d1078e2244b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.101751 14654 tablet_service.cc:1467] Processing CreateTablet for tablet bbf95c74a6d449f7a77a685804fd2e5b (DEFAULT_TABLE table=default.Table_fa0e3b934db54f57b3b6565e1a6abb38 [id=9cbc68c14da444039d7272170a7c6c78]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.101840 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9eb56ebdb1a24b8ea8fbc4c129fded95. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.101959 14654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbf95c74a6d449f7a77a685804fd2e5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.102159 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 5076e95bfbac445899cb8ae88b49969a (DEFAULT_TABLE table=default.Table_4ec1365f1a874eb49b19337126347443 [id=57db97740d53455ea9894ea4d707ab2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.102301 14671 tablet_service.cc:1467] Processing CreateTablet for tablet 34bd3886562742bc980c571ca76230fe (DEFAULT_TABLE table=default.Table_e303383ddf3845828de6837219313014 [id=654dfdc4477b476f80e6af5b066713c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.102352 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5076e95bfbac445899cb8ae88b49969a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.102555 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34bd3886562742bc980c571ca76230fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.103015 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b06b9691451f4a88b07044fd626aa6d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.103616 14658 tablet_service.cc:1467] Processing CreateTablet for tablet 9476eae94d664f5eb94865db2c95dca6 (DEFAULT_TABLE table=default.Table_75817b46a6414888988ff7373ea11b27 [id=dfa93dac2ee14cfd8fe486869349c288]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.103857 14657 tablet_service.cc:1467] Processing CreateTablet for tablet d444135ee35c4f95b210736aaecc668c (DEFAULT_TABLE table=default.Table_85da758784964b0c9135d0abba13c4cb [id=0b32abc361734a40860ccbaf7fc3fde5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.104105 14656 tablet_service.cc:1467] Processing CreateTablet for tablet 118dd31cc927421ba837ca44766cc925 (DEFAULT_TABLE table=default.Table_2ec77dbcffce476aa7f1d98daa698e84 [id=e8f7ed89657b4465a78feb971b65acb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.104158 14655 tablet_service.cc:1467] Processing CreateTablet for tablet b06b9691451f4a88b07044fd626aa6d0 (DEFAULT_TABLE table=default.Table_b45108a844cb400cbc6220e82aaafb9e [id=e0cddff8b58344afbd8e4f4a1c3daae4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.104347 14659 tablet_service.cc:1467] Processing CreateTablet for tablet ec1a8d0326f84ca0a87851010c1e68b7 (DEFAULT_TABLE table=default.Table_ab72071c412848939b54fcc72d5974fa [id=aff8cb5016d84881917b303cb3c2e4d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.104391 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 6004f4adca8e47ddb91f7e39c4cf9089 (DEFAULT_TABLE table=default.Table_64d33ab8807d44a996a1f0fcc2deb7e2 [id=62fd48d944c24d15962e1708c7b71b27]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.101027 14392 tablet_service.cc:1467] Processing CreateTablet for tablet d444135ee35c4f95b210736aaecc668c (DEFAULT_TABLE table=default.Table_85da758784964b0c9135d0abba13c4cb [id=0b32abc361734a40860ccbaf7fc3fde5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.102166 14668 tablet_service.cc:1467] Processing CreateTablet for tablet 5c5e30d9b2ce4537b00b216ef1f2dd1c (DEFAULT_TABLE table=default.Table_cb51431eaddb48858a61d248b688bbe7 [id=5624f7f1c61140a798718c8c85da14ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.106362 14554 raft_consensus.cc:3054] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.108011 16185 tablet.cc:1721] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:20.108167 14392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d444135ee35c4f95b210736aaecc668c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.109083 14858 raft_consensus.cc:491] T d5655935aabd48a19d0ef03739e089a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.109155 14858 raft_consensus.cc:513] T d5655935aabd48a19d0ef03739e089a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.109428 14858 leader_election.cc:290] T d5655935aabd48a19d0ef03739e089a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:20.110734 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5655935aabd48a19d0ef03739e089a4" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:20.112373 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5655935aabd48a19d0ef03739e089a4" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:20.112689 14493 leader_election.cc:304] T d5655935aabd48a19d0ef03739e089a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:20.112942 14858 raft_consensus.cc:2743] T d5655935aabd48a19d0ef03739e089a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.113027 14391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9476eae94d664f5eb94865db2c95dca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.114251 14658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9476eae94d664f5eb94865db2c95dca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.114881 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 118dd31cc927421ba837ca44766cc925. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.116042 14657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d444135ee35c4f95b210736aaecc668c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.116659 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d894471f7b94aec8a19df043906d7e7 with delete_type TABLET_DATA_DELETED (replaced by tablet 194d943d173340e7a8fba69967d9b090) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.116808 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet cff95c69fb9045cf9ec0ebb128c9ef6b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.116997 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f2dc6928e2941ef9b5b5a7891376500 with delete_type TABLET_DATA_DELETED (replaced by tablet 7f9818d0493547358fd5654a01d4f456) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.118074 14532 tablet_service.cc:1467] Processing CreateTablet for tablet fe0dd7895a6547e8a105492083f2ec22 (DEFAULT_TABLE table=default.Table_5a9c638d77a84fefb016a56854ac897f [id=ac3c3d9a44e248649310cc277565097d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.118135 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 9eb56ebdb1a24b8ea8fbc4c129fded95 (DEFAULT_TABLE table=default.Table_eca9fde960bd4324a474bbda7f6e3317 [id=f80db4b835b24d09bb76d1078e2244b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.118285 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe0dd7895a6547e8a105492083f2ec22. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.118319 14535 tablet_service.cc:1467] Processing CreateTablet for tablet bbf95c74a6d449f7a77a685804fd2e5b (DEFAULT_TABLE table=default.Table_fa0e3b934db54f57b3b6565e1a6abb38 [id=9cbc68c14da444039d7272170a7c6c78]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.118539 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbf95c74a6d449f7a77a685804fd2e5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.119521 14518 tablet_service.cc:1467] Processing CreateTablet for tablet d444135ee35c4f95b210736aaecc668c (DEFAULT_TABLE table=default.Table_85da758784964b0c9135d0abba13c4cb [id=0b32abc361734a40860ccbaf7fc3fde5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.119712 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d444135ee35c4f95b210736aaecc668c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.120029 14820 tablet_replica.cc:331] T 7d894471f7b94aec8a19df043906d7e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.120148 14820 raft_consensus.cc:2238] T 7d894471f7b94aec8a19df043906d7e7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.120268 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 5076e95bfbac445899cb8ae88b49969a (DEFAULT_TABLE table=default.Table_4ec1365f1a874eb49b19337126347443 [id=57db97740d53455ea9894ea4d707ab2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.120306 14820 raft_consensus.cc:2267] T 7d894471f7b94aec8a19df043906d7e7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.120477 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5076e95bfbac445899cb8ae88b49969a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.121605 14527 tablet_service.cc:1467] Processing CreateTablet for tablet ec1a8d0326f84ca0a87851010c1e68b7 (DEFAULT_TABLE table=default.Table_ab72071c412848939b54fcc72d5974fa [id=aff8cb5016d84881917b303cb3c2e4d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.121795 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec1a8d0326f84ca0a87851010c1e68b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.122318 14536 tablet_service.cc:1467] Processing CreateTablet for tablet b06b9691451f4a88b07044fd626aa6d0 (DEFAULT_TABLE table=default.Table_b45108a844cb400cbc6220e82aaafb9e [id=e0cddff8b58344afbd8e4f4a1c3daae4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.122501 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b06b9691451f4a88b07044fd626aa6d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.123031 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 0c580cdf3a854df988df32a9294c83bc
I20250115 03:06:20.123669 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet c2afa048b3fd4362b50c8a6da306c25a
I20250115 03:06:20.123894 14171 catalog_manager.cc:6596] ReplaceTablet: tablet c2afa048b3fd4362b50c8a6da306c25a deleted and replaced by tablet 6501c82435154d6e8f2c12b7c82da296
I20250115 03:06:20.124071 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9eb56ebdb1a24b8ea8fbc4c129fded95. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.124477 14526 tablet_service.cc:1467] Processing CreateTablet for tablet 6004f4adca8e47ddb91f7e39c4cf9089 (DEFAULT_TABLE table=default.Table_64d33ab8807d44a996a1f0fcc2deb7e2 [id=62fd48d944c24d15962e1708c7b71b27]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.124660 14526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6004f4adca8e47ddb91f7e39c4cf9089. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.125191 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3db38338e5b04298954aded6d5482169" } modify_external_catalogs: true
I20250115 03:06:20.125281 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3db38338e5b04298954aded6d5482169" } modify_external_catalogs: true
I20250115 03:06:20.125679 14169 catalog_manager.cc:5526] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:20.125885 14169 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 6bcfde5fe3d4425e98edb4dc54096718 (table default.Table_a63d6344914343c4af56eef21f467c6f [id=f45053c781fc460ab55f592eed6fb946]). Expected version 1 got 0
I20250115 03:06:20.126147 14169 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet b56279bb16054869a97006d88574e381 (table default.Table_23165257844448ce93a97233b487ddc7 [id=080d0a4dfa174d3ba2979bb850d0604a]). Expected version 1 got 0
I20250115 03:06:20.126590 14534 tablet_service.cc:1467] Processing CreateTablet for tablet 9476eae94d664f5eb94865db2c95dca6 (DEFAULT_TABLE table=default.Table_75817b46a6414888988ff7373ea11b27 [id=dfa93dac2ee14cfd8fe486869349c288]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.126794 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9476eae94d664f5eb94865db2c95dca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.128445 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_bca65e8db1d94382a3528cf21a111c64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:20.128638 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 118dd31cc927421ba837ca44766cc925 (DEFAULT_TABLE table=default.Table_2ec77dbcffce476aa7f1d98daa698e84 [id=e8f7ed89657b4465a78feb971b65acb7]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:20.128751 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bca65e8db1d94382a3528cf21a111c64 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.128834 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 118dd31cc927421ba837ca44766cc925. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.129884 14656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 118dd31cc927421ba837ca44766cc925. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.131352 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6004f4adca8e47ddb91f7e39c4cf9089. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.131522 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 22cc0833bfde458ebe3985c05d29b8e6
I20250115 03:06:20.131716 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b06b9691451f4a88b07044fd626aa6d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.133252 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34bd3886562742bc980c571ca76230fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.134255 14659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec1a8d0326f84ca0a87851010c1e68b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.135129 14530 tablet_service.cc:1467] Processing CreateTablet for tablet ffc493e8444147018bdf50382dc95b63 (DEFAULT_TABLE table=default.Table_2868f144930b418199ddf15146d94c13 [id=036a621d8a314a6480abd793aad707eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.135317 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffc493e8444147018bdf50382dc95b63. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.136004 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6004f4adca8e47ddb91f7e39c4cf9089. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.137213 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec1a8d0326f84ca0a87851010c1e68b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.139065 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c5e30d9b2ce4537b00b216ef1f2dd1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.139245 14395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c5e30d9b2ce4537b00b216ef1f2dd1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.148272 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 34bd3886562742bc980c571ca76230fe (DEFAULT_TABLE table=default.Table_e303383ddf3845828de6837219313014 [id=654dfdc4477b476f80e6af5b066713c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.148502 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34bd3886562742bc980c571ca76230fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.148773 14552 raft_consensus.cc:3054] T 51b412277e634ea39bb6d2b90a43b875 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.149663 14811 tablet_bootstrap.cc:492] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.149752 14811 ts_tablet_manager.cc:1397] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.079s user 0.002s sys 0.000s
I20250115 03:06:20.150185 14811 raft_consensus.cc:357] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.150287 14811 raft_consensus.cc:383] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.150321 14811 raft_consensus.cc:738] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.150439 14811 consensus_queue.cc:260] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.150712 14811 ts_tablet_manager.cc:1428] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.150727 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 22cc0833bfde458ebe3985c05d29b8e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.150864 14823 tablet_replica.cc:331] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:20.150872 14811 tablet_bootstrap.cc:492] T cc67fb818b8146629d92f9ac1944dba6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.150985 14823 raft_consensus.cc:2238] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:20.151010 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c580cdf3a854df988df32a9294c83bc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.151153 14823 raft_consensus.cc:2267] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.151238 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 5c5e30d9b2ce4537b00b216ef1f2dd1c (DEFAULT_TABLE table=default.Table_cb51431eaddb48858a61d248b688bbe7 [id=5624f7f1c61140a798718c8c85da14ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.151448 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c5e30d9b2ce4537b00b216ef1f2dd1c. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:20.151424 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 0c580cdf3a854df988df32a9294c83bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c580cdf3a854df988df32a9294c83bc already in progress: creating tablet
I20250115 03:06:20.151548 14823 ts_tablet_manager.cc:1905] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.151715 14811 tablet_bootstrap.cc:654] T cc67fb818b8146629d92f9ac1944dba6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.152455 14809 tablet_bootstrap.cc:492] T adbb0ae719f24dcc96709a83981e213d P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.152529 14809 ts_tablet_manager.cc:1397] T adbb0ae719f24dcc96709a83981e213d P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.071s user 0.002s sys 0.000s
I20250115 03:06:20.152565 14811 tablet_bootstrap.cc:492] T cc67fb818b8146629d92f9ac1944dba6 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.152626 14811 ts_tablet_manager.cc:1397] T cc67fb818b8146629d92f9ac1944dba6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:20.153017 14809 raft_consensus.cc:357] T adbb0ae719f24dcc96709a83981e213d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.153046 14811 raft_consensus.cc:357] T cc67fb818b8146629d92f9ac1944dba6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.153111 14809 raft_consensus.cc:383] T adbb0ae719f24dcc96709a83981e213d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.153117 14811 raft_consensus.cc:383] T cc67fb818b8146629d92f9ac1944dba6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.153141 14809 raft_consensus.cc:738] T adbb0ae719f24dcc96709a83981e213d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.153146 14811 raft_consensus.cc:738] T cc67fb818b8146629d92f9ac1944dba6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.153247 14809 consensus_queue.cc:260] T adbb0ae719f24dcc96709a83981e213d P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.153265 14811 consensus_queue.cc:260] T cc67fb818b8146629d92f9ac1944dba6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.153311 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet c2afa048b3fd4362b50c8a6da306c25a with delete_type TABLET_DATA_DELETED (replaced by tablet 6501c82435154d6e8f2c12b7c82da296) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.153498 14809 ts_tablet_manager.cc:1428] T adbb0ae719f24dcc96709a83981e213d P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.153535 14811 ts_tablet_manager.cc:1428] T cc67fb818b8146629d92f9ac1944dba6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.153589 14823 ts_tablet_manager.cc:1918] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:20.153649 14823 log.cc:1198] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/22cc0833bfde458ebe3985c05d29b8e6
I20250115 03:06:20.153669 14811 tablet_bootstrap.cc:492] T 80aa570e2c154172ad6da29d268bede7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.153669 14809 tablet_bootstrap.cc:492] T c6fa98c6fb304dc49b8e0ced989cc936 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.153863 14823 ts_tablet_manager.cc:1939] T 22cc0833bfde458ebe3985c05d29b8e6 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:20.154529 14809 tablet_bootstrap.cc:654] T c6fa98c6fb304dc49b8e0ced989cc936 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.154616 14659 tablet_service.cc:1514] Processing DeleteTablet for tablet 4102586f32d94a998cf2807434ee0ebf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.154688 14553 raft_consensus.cc:3054] T 5cb34cb2006f4716b87bef67f3a6dd8e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.154754 14655 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c580cdf3a854df988df32a9294c83bc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.155036 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 623887ec17374f598f48054e17582f3f with delete_type TABLET_DATA_DELETED (replaced by tablet 12678da7ffee4a46b4690ac3e94cb19b) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.155141 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet c2afa048b3fd4362b50c8a6da306c25a with delete_type TABLET_DATA_DELETED (replaced by tablet 6501c82435154d6e8f2c12b7c82da296) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.155243 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet 22cc0833bfde458ebe3985c05d29b8e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.155725 14823 tablet_replica.cc:331] T 4102586f32d94a998cf2807434ee0ebf P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:20.155706 15390 raft_consensus.cc:491] T 0521973e23984355a77a321b4676b479 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.155786 15390 raft_consensus.cc:513] T 0521973e23984355a77a321b4676b479 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.155834 14823 raft_consensus.cc:2238] T 4102586f32d94a998cf2807434ee0ebf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.155911 14823 raft_consensus.cc:2267] T 4102586f32d94a998cf2807434ee0ebf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.156117 15390 leader_election.cc:290] T 0521973e23984355a77a321b4676b479 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:20.156250 14820 ts_tablet_manager.cc:1905] T 7d894471f7b94aec8a19df043906d7e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.156584 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 22cc0833bfde458ebe3985c05d29b8e6 (table default.Table_3db38338e5b04298954aded6d5482169 [id=f56a0de6e56441bfa620902f175feb4a]) successfully deleted
I20250115 03:06:20.156813 15426 tablet.cc:1721] T 5cb34cb2006f4716b87bef67f3a6dd8e P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:20.157142 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0521973e23984355a77a321b4676b479" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:20.157480 14358 leader_election.cc:343] T 0521973e23984355a77a321b4676b479 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.158169 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0521973e23984355a77a321b4676b479" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:20.158365 14556 raft_consensus.cc:3054] T 01433c76a398462bb6708a02997047fa P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:20.158509 14356 leader_election.cc:343] T 0521973e23984355a77a321b4676b479 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.158564 14356 leader_election.cc:304] T 0521973e23984355a77a321b4676b479 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:20.159672 15390 raft_consensus.cc:2743] T 0521973e23984355a77a321b4676b479 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.160192 14395 tablet_service.cc:1514] Processing DeleteTablet for tablet c2afa048b3fd4362b50c8a6da306c25a with delete_type TABLET_DATA_DELETED (replaced by tablet 6501c82435154d6e8f2c12b7c82da296) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.160333 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c580cdf3a854df988df32a9294c83bc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.160570 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 9eb56ebdb1a24b8ea8fbc4c129fded95 (DEFAULT_TABLE table=default.Table_eca9fde960bd4324a474bbda7f6e3317 [id=f80db4b835b24d09bb76d1078e2244b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.160804 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9eb56ebdb1a24b8ea8fbc4c129fded95. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:20.161634 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet c2afa048b3fd4362b50c8a6da306c25a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c2afa048b3fd4362b50c8a6da306c25a already in progress: creating tablet
I20250115 03:06:20.161751 14820 ts_tablet_manager.cc:1918] T 7d894471f7b94aec8a19df043906d7e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.161813 14820 log.cc:1198] T 7d894471f7b94aec8a19df043906d7e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/7d894471f7b94aec8a19df043906d7e7
I20250115 03:06:20.161916 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 22cc0833bfde458ebe3985c05d29b8e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.162084 14819 tablet_replica.cc:331] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:20.162209 14396 tablet_service.cc:1467] Processing CreateTablet for tablet bbf95c74a6d449f7a77a685804fd2e5b (DEFAULT_TABLE table=default.Table_fa0e3b934db54f57b3b6565e1a6abb38 [id=9cbc68c14da444039d7272170a7c6c78]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.162209 14819 raft_consensus.cc:2238] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.162423 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbf95c74a6d449f7a77a685804fd2e5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.162461 14819 raft_consensus.cc:2267] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.162793 14819 ts_tablet_manager.cc:1905] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.163726 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet cd6f136b92ba4fc7b956d52aa0c20f53 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:20.164798 14819 ts_tablet_manager.cc:1918] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:20.164857 14819 log.cc:1198] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/22cc0833bfde458ebe3985c05d29b8e6
I20250115 03:06:20.165499 14819 ts_tablet_manager.cc:1939] T 22cc0833bfde458ebe3985c05d29b8e6 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:20.166054 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 22cc0833bfde458ebe3985c05d29b8e6 (table default.Table_3db38338e5b04298954aded6d5482169 [id=f56a0de6e56441bfa620902f175feb4a]) successfully deleted
I20250115 03:06:20.168396 14823 ts_tablet_manager.cc:1905] T 4102586f32d94a998cf2807434ee0ebf P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.161996 14820 ts_tablet_manager.cc:1939] T 7d894471f7b94aec8a19df043906d7e7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.166392 14811 tablet_bootstrap.cc:654] T 80aa570e2c154172ad6da29d268bede7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.166622 14858 raft_consensus.cc:491] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.169469 14858 raft_consensus.cc:513] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.169597 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b9c09b6315345039cdea9eeee3f181a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.169775 14858 leader_election.cc:290] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:20.169744 16265 tablet.cc:1721] T 01433c76a398462bb6708a02997047fa P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:20.170224 14668 tablet_service.cc:1467] Processing CreateTablet for tablet 0aeb6e439d574790a1554f6ff79079c6 (DEFAULT_TABLE table=default.Table_ece95328be52409b94bde437fffffc25 [id=ccab6d271d0e4fa1a64e1dc14fe29821]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:20.170392 14493 leader_election.cc:343] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 3b9c09b6315345039cdea9eeee3f181a
I20250115 03:06:20.170432 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0aeb6e439d574790a1554f6ff79079c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.171476 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 66b748fc853f499fafc1f476c8f79b8b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.172008 14823 ts_tablet_manager.cc:1918] T 4102586f32d94a998cf2807434ee0ebf P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.172125 14823 log.cc:1198] T 4102586f32d94a998cf2807434ee0ebf P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/4102586f32d94a998cf2807434ee0ebf
I20250115 03:06:20.172175 14819 tablet_replica.cc:331] T 66b748fc853f499fafc1f476c8f79b8b P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:20.172238 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet adbb0ae719f24dcc96709a83981e213d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.172374 14819 raft_consensus.cc:2238] T 66b748fc853f499fafc1f476c8f79b8b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.172472 14819 raft_consensus.cc:2267] T 66b748fc853f499fafc1f476c8f79b8b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.172647 14823 ts_tablet_manager.cc:1939] T 4102586f32d94a998cf2807434ee0ebf P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:20.172859 14819 ts_tablet_manager.cc:1905] T 66b748fc853f499fafc1f476c8f79b8b P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.173306 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 7d894471f7b94aec8a19df043906d7e7 (table default.Table_1a2ec424c5cb40fbb3c93f327531e36f [id=50a51914c37b462aa45a71fb260c809b]) successfully deleted
I20250115 03:06:20.173393 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 0aeb6e439d574790a1554f6ff79079c6 (DEFAULT_TABLE table=default.Table_ece95328be52409b94bde437fffffc25 [id=ccab6d271d0e4fa1a64e1dc14fe29821]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.173494 14820 tablet_replica.cc:331] T cff95c69fb9045cf9ec0ebb128c9ef6b P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.173831 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0aeb6e439d574790a1554f6ff79079c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.173887 14820 raft_consensus.cc:2238] T cff95c69fb9045cf9ec0ebb128c9ef6b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.173969 14820 raft_consensus.cc:2267] T cff95c69fb9045cf9ec0ebb128c9ef6b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.174387 14820 ts_tablet_manager.cc:1905] T cff95c69fb9045cf9ec0ebb128c9ef6b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.174592 14819 ts_tablet_manager.cc:1918] T 66b748fc853f499fafc1f476c8f79b8b P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.174645 14819 log.cc:1198] T 66b748fc853f499fafc1f476c8f79b8b P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/66b748fc853f499fafc1f476c8f79b8b
I20250115 03:06:20.174818 14819 ts_tablet_manager.cc:1939] T 66b748fc853f499fafc1f476c8f79b8b P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
W20250115 03:06:20.175066 14493 leader_election.cc:343] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 3b9c09b6315345039cdea9eeee3f181a
I20250115 03:06:20.175132 14493 leader_election.cc:304] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:20.175328 14858 raft_consensus.cc:2743] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.175675 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 4102586f32d94a998cf2807434ee0ebf (table default.Table_a1c9e28bac9c491e948bc3fc9b173bad [id=2fb8c68c76304deebd009b01eb078616]) successfully deleted
I20250115 03:06:20.175786 14823 tablet_replica.cc:331] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
W20250115 03:06:20.175828 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 0c580cdf3a854df988df32a9294c83bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c580cdf3a854df988df32a9294c83bc already in progress: creating tablet
I20250115 03:06:20.175910 14823 raft_consensus.cc:2238] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.176034 14823 raft_consensus.cc:2267] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.176193 14820 ts_tablet_manager.cc:1918] T cff95c69fb9045cf9ec0ebb128c9ef6b P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.176548 14823 ts_tablet_manager.cc:1905] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.177799 14820 log.cc:1198] T cff95c69fb9045cf9ec0ebb128c9ef6b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/cff95c69fb9045cf9ec0ebb128c9ef6b
I20250115 03:06:20.178201 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet cc67fb818b8146629d92f9ac1944dba6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.178529 14823 ts_tablet_manager.cc:1918] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.178633 14823 log.cc:1198] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/623887ec17374f598f48054e17582f3f
I20250115 03:06:20.178854 14823 ts_tablet_manager.cc:1939] T 623887ec17374f598f48054e17582f3f P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:20.179369 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 623887ec17374f598f48054e17582f3f (table default.Table_0337ad43f9b646a48c61c66c15f1844a [id=7f5506ea0a5e465cba8de1ba2f5e75c6]) successfully deleted
W20250115 03:06:20.179783 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet c2afa048b3fd4362b50c8a6da306c25a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c2afa048b3fd4362b50c8a6da306c25a already in progress: creating tablet
I20250115 03:06:20.180004 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 66b748fc853f499fafc1f476c8f79b8b (table default.Table_7eacd8191bf44769b6371caa6d4772f1 [id=ea9e7141223d43de8962caa78ab14914]) successfully deleted
I20250115 03:06:20.180193 14819 tablet_replica.cc:331] T adbb0ae719f24dcc96709a83981e213d P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:20.180816 14811 tablet_bootstrap.cc:492] T 80aa570e2c154172ad6da29d268bede7 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.180912 14811 ts_tablet_manager.cc:1397] T 80aa570e2c154172ad6da29d268bede7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20250115 03:06:20.181327 14811 raft_consensus.cc:357] T 80aa570e2c154172ad6da29d268bede7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.181504 14820 ts_tablet_manager.cc:1939] T cff95c69fb9045cf9ec0ebb128c9ef6b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.181510 14811 raft_consensus.cc:383] T 80aa570e2c154172ad6da29d268bede7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.181659 14811 raft_consensus.cc:738] T 80aa570e2c154172ad6da29d268bede7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.181809 14811 consensus_queue.cc:260] T 80aa570e2c154172ad6da29d268bede7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.181862 14819 raft_consensus.cc:2238] T adbb0ae719f24dcc96709a83981e213d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.181974 14812 tablet_bootstrap.cc:492] T d2564a20857f45d9a0c7e3b74eb58a91 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.182094 14812 ts_tablet_manager.cc:1397] T d2564a20857f45d9a0c7e3b74eb58a91 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.091s user 0.001s sys 0.001s
I20250115 03:06:20.182132 14811 ts_tablet_manager.cc:1428] T 80aa570e2c154172ad6da29d268bede7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:20.181954 14819 raft_consensus.cc:2267] T adbb0ae719f24dcc96709a83981e213d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.182428 14811 tablet_bootstrap.cc:492] T 5737f10aa50e4059a814585019346872 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.182586 14812 raft_consensus.cc:357] T d2564a20857f45d9a0c7e3b74eb58a91 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.182740 14812 raft_consensus.cc:383] T d2564a20857f45d9a0c7e3b74eb58a91 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.182816 14812 raft_consensus.cc:738] T d2564a20857f45d9a0c7e3b74eb58a91 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.183010 14812 consensus_queue.cc:260] T d2564a20857f45d9a0c7e3b74eb58a91 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.183349 14811 tablet_bootstrap.cc:654] T 5737f10aa50e4059a814585019346872 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.183455 14819 ts_tablet_manager.cc:1905] T adbb0ae719f24dcc96709a83981e213d P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.185241 14819 ts_tablet_manager.cc:1918] T adbb0ae719f24dcc96709a83981e213d P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.185297 14819 log.cc:1198] T adbb0ae719f24dcc96709a83981e213d P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/adbb0ae719f24dcc96709a83981e213d
I20250115 03:06:20.185477 14819 ts_tablet_manager.cc:1939] T adbb0ae719f24dcc96709a83981e213d P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:20.185997 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet adbb0ae719f24dcc96709a83981e213d (table default.Table_e73230225df744a2a11f246bea89b8b0 [id=407331115fba4ca28b006a587584ea0d]) successfully deleted
I20250115 03:06:20.187206 14820 tablet_replica.cc:331] T 1f2dc6928e2941ef9b5b5a7891376500 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.187304 14820 raft_consensus.cc:2238] T 1f2dc6928e2941ef9b5b5a7891376500 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.187371 14820 raft_consensus.cc:2267] T 1f2dc6928e2941ef9b5b5a7891376500 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.189406 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet d2564a20857f45d9a0c7e3b74eb58a91 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.190222 14812 ts_tablet_manager.cc:1428] T d2564a20857f45d9a0c7e3b74eb58a91 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20250115 03:06:20.190562 14820 ts_tablet_manager.cc:1905] T 1f2dc6928e2941ef9b5b5a7891376500 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:20.190815 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet d2564a20857f45d9a0c7e3b74eb58a91 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d2564a20857f45d9a0c7e3b74eb58a91 already in progress: creating tablet
I20250115 03:06:20.191680 14812 tablet_bootstrap.cc:492] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.193416 14820 ts_tablet_manager.cc:1918] T 1f2dc6928e2941ef9b5b5a7891376500 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.193498 14812 tablet_bootstrap.cc:654] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.193909 14820 log.cc:1198] T 1f2dc6928e2941ef9b5b5a7891376500 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/1f2dc6928e2941ef9b5b5a7891376500
I20250115 03:06:20.194191 14820 ts_tablet_manager.cc:1939] T 1f2dc6928e2941ef9b5b5a7891376500 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.195356 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet cff95c69fb9045cf9ec0ebb128c9ef6b (table default.Table_1ed9a3074bb7436797358399890084b7 [id=8a02676ab3334b7c81ce22069b29551f]) successfully deleted
I20250115 03:06:20.195487 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet d2564a20857f45d9a0c7e3b74eb58a91 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.195794 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_c95d9b4b8beb4ebea3e620d35b56bd40" } modify_external_catalogs: true
I20250115 03:06:20.195878 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_fe5a8443c8e54cb6a0f436c90a2b5b91" } new_table_name: "default.Table_54bdb95d2f9244c2a2e3559a36b1f6e1" modify_external_catalogs: true
I20250115 03:06:20.196090 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_05cfde56d7dd49b994fe32031c716aeb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:20.196079 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_5072bde4d75d4184a223755f60397e83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.196429 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_05cfde56d7dd49b994fe32031c716aeb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:20.196484 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5072bde4d75d4184a223755f60397e83 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.196682 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ece95328be52409b94bde437fffffc25" } modify_external_catalogs: true
I20250115 03:06:20.196733 14820 tablet_replica.cc:331] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.196779 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ece95328be52409b94bde437fffffc25" } modify_external_catalogs: true
I20250115 03:06:20.196836 14820 raft_consensus.cc:2238] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.196954 14820 raft_consensus.cc:2267] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.194248 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 80aa570e2c154172ad6da29d268bede7 with delete_type TABLET_DATA_DELETED (replaced by tablet 815ef85ce5cf4ae0a56c5734837e03ba) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.197266 14820 ts_tablet_manager.cc:1905] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.197430 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_968db1bba3cf48dbba6f4456bcfeb41a" } new_table_name: "default.Table_a37ebbac9e4843e9a3376be17c3dec26" modify_external_catalogs: true
I20250115 03:06:20.197512 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_375bb4e2095c4675b734247fabacbec7" } new_table_name: "default.Table_4b1d421298a845129d33322313978924" modify_external_catalogs: true
I20250115 03:06:20.197616 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 1f2dc6928e2941ef9b5b5a7891376500 (table default.Table_652f8e06e005491381404a11759aefb1 [id=6fb6e311a0af4f51b190a880f84373d1]) successfully deleted
W20250115 03:06:20.197762 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet c2afa048b3fd4362b50c8a6da306c25a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c2afa048b3fd4362b50c8a6da306c25a already in progress: creating tablet
I20250115 03:06:20.197923 14823 tablet_replica.cc:331] T d2564a20857f45d9a0c7e3b74eb58a91 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:20.198025 14823 raft_consensus.cc:2238] T d2564a20857f45d9a0c7e3b74eb58a91 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.198104 14823 raft_consensus.cc:2267] T d2564a20857f45d9a0c7e3b74eb58a91 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.198246 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d6bfa64233ea4a3d87766891d5b37ca9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:20.198487 14823 ts_tablet_manager.cc:1905] T d2564a20857f45d9a0c7e3b74eb58a91 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:20.198551 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d6bfa64233ea4a3d87766891d5b37ca9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.199178 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_c3e03487af564b27a13203fb3ea3e1de" } modify_external_catalogs: true
I20250115 03:06:20.199267 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_c3e03487af564b27a13203fb3ea3e1de" } modify_external_catalogs: true
I20250115 03:06:20.200268 14885 raft_consensus.cc:491] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.200340 14885 raft_consensus.cc:513] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.200632 14885 leader_election.cc:290] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:20.201030 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_99e4573f330b4bf5b33dac953542aca5" } new_table_name: "default.Table_a838d05cbe294b9e8ec38d0c1a3f6221" modify_external_catalogs: true
I20250115 03:06:20.201534 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b4eb3b20bb4908884e5716e7b2a369" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:20.201896 14356 leader_election.cc:343] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.202311 14823 ts_tablet_manager.cc:1918] T d2564a20857f45d9a0c7e3b74eb58a91 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.202361 14823 log.cc:1198] T d2564a20857f45d9a0c7e3b74eb58a91 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/d2564a20857f45d9a0c7e3b74eb58a91
I20250115 03:06:20.202517 14823 ts_tablet_manager.cc:1939] T d2564a20857f45d9a0c7e3b74eb58a91 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:20.202625 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b4eb3b20bb4908884e5716e7b2a369" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:20.195894 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_c95d9b4b8beb4ebea3e620d35b56bd40" } modify_external_catalogs: true
W20250115 03:06:20.206355 14358 leader_election.cc:343] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.206426 14358 leader_election.cc:304] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:20.206597 15401 raft_consensus.cc:2743] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.210032 14820 ts_tablet_manager.cc:1918] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:20.210095 14820 log.cc:1198] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/22cc0833bfde458ebe3985c05d29b8e6
I20250115 03:06:20.210314 14820 ts_tablet_manager.cc:1939] T 22cc0833bfde458ebe3985c05d29b8e6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.210867 14820 tablet_replica.cc:331] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.210976 14820 raft_consensus.cc:2238] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.211059 14820 raft_consensus.cc:2267] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.211477 14820 ts_tablet_manager.cc:1905] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.213232 14820 ts_tablet_manager.cc:1918] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.213346 14820 log.cc:1198] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/3b9c09b6315345039cdea9eeee3f181a
I20250115 03:06:20.213568 14820 ts_tablet_manager.cc:1939] T 3b9c09b6315345039cdea9eeee3f181a P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.214049 14820 tablet_replica.cc:331] T cc67fb818b8146629d92f9ac1944dba6 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.214193 14820 raft_consensus.cc:2238] T cc67fb818b8146629d92f9ac1944dba6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.214318 14820 raft_consensus.cc:2267] T cc67fb818b8146629d92f9ac1944dba6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.215582 14811 tablet_bootstrap.cc:492] T 5737f10aa50e4059a814585019346872 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.215665 14811 ts_tablet_manager.cc:1397] T 5737f10aa50e4059a814585019346872 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.033s user 0.000s sys 0.002s
I20250115 03:06:20.216086 14811 raft_consensus.cc:357] T 5737f10aa50e4059a814585019346872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.216184 14811 raft_consensus.cc:383] T 5737f10aa50e4059a814585019346872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.216215 14811 raft_consensus.cc:738] T 5737f10aa50e4059a814585019346872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
W20250115 03:06:20.216334 14626 consensus_peers.cc:487] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:20.216321 14811 consensus_queue.cc:260] T 5737f10aa50e4059a814585019346872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.216572 14811 ts_tablet_manager.cc:1428] T 5737f10aa50e4059a814585019346872 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:20.216753 14811 tablet_bootstrap.cc:492] T e5a8293f43134a54b3f8dfb5fb99d6ea P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.217623 14811 tablet_bootstrap.cc:654] T e5a8293f43134a54b3f8dfb5fb99d6ea P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.218370 14537 tablet_service.cc:1467] Processing CreateTablet for tablet d5fa50804f5443d08c1a9723626049de (DEFAULT_TABLE table=default.Table_9a8c2b74534f4bf2b5fc30dff185e54c [id=ef166e3ea16d4bbbb6c35bb4f3993dd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.218720 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 22cc0833bfde458ebe3985c05d29b8e6 (table default.Table_3db38338e5b04298954aded6d5482169 [id=f56a0de6e56441bfa620902f175feb4a]) successfully deleted
I20250115 03:06:20.218961 14527 tablet_service.cc:1467] Processing CreateTablet for tablet eeb37ffddf484ced806c148940e8f00d (DEFAULT_TABLE table=default.Table_10784ed50a0948d7a64a54cffcda719e [id=00078131a06346fe84c286b20596d99f]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:20.218991 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 0c580cdf3a854df988df32a9294c83bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c580cdf3a854df988df32a9294c83bc already in progress: creating tablet
I20250115 03:06:20.219174 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 3b9c09b6315345039cdea9eeee3f181a (table default.Table_561cca4d810b4e73934fcee99a8994c7 [id=270d1081a9a04f09b76df0f45cfbbff6]) successfully deleted
I20250115 03:06:20.219362 14534 tablet_service.cc:1467] Processing CreateTablet for tablet ff57789676224715b5872ead0a8e68fd (DEFAULT_TABLE table=default.Table_3386c15b41014720beb4e38c88d64b08 [id=480f8200824d432e92247a90f43f0b77]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.219609 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 73cbcb3b31624b28bbac7bdaf063c7d9 (DEFAULT_TABLE table=default.Table_472a764a0bfe4cf7ad316cc6de021448 [id=1b12a13cca4a4b34be11ba0adb84954d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.219866 14530 tablet_service.cc:1467] Processing CreateTablet for tablet b9979224e3874a7982e1bada8395c921 (DEFAULT_TABLE table=default.Table_4b64655665164e00b508a19e84b9991f [id=c206de1bd1fa4e3db3d80167aca79165]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.220050 14820 ts_tablet_manager.cc:1905] T cc67fb818b8146629d92f9ac1944dba6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.218384 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 93b979bef1a543ba84092cb255acb7c2 (DEFAULT_TABLE table=default.Table_802ff80c57304d6393caa12f80fb4929 [id=ef28d0ab1e27446486c90392a13fa4f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.220845 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5fa50804f5443d08c1a9723626049de. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.222898 14820 ts_tablet_manager.cc:1918] T cc67fb818b8146629d92f9ac1944dba6 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.222973 14820 log.cc:1198] T cc67fb818b8146629d92f9ac1944dba6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/cc67fb818b8146629d92f9ac1944dba6
I20250115 03:06:20.223150 14820 ts_tablet_manager.cc:1939] T cc67fb818b8146629d92f9ac1944dba6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.223285 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 293a76d0d49e4a12ab951cbc7311d0f2
I20250115 03:06:20.223472 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 0aeb6e439d574790a1554f6ff79079c6
I20250115 03:06:20.223609 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_5e83ef9eef624e33af51363bb4ed5920"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:20.223825 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0f3709600e57401499c5004118dc8ef3" } new_table_name: "default.Table_5c360e43ddae405b9a2843fc20a9dfb9" modify_external_catalogs: true
W20250115 03:06:20.223914 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5e83ef9eef624e33af51363bb4ed5920 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.224047 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_09dd39785939469795df4097f8b9a6a4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.224319 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_09dd39785939469795df4097f8b9a6a4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.224674 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet f2af9569a0254f32b4b6ceab5bfbfa74 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:20.225070 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff57789676224715b5872ead0a8e68fd. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:20.225370 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dcc1c999a9d34006a9bd4ec2bec02bdd consensus state has no leader
I20250115 03:06:20.227113 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73cbcb3b31624b28bbac7bdaf063c7d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.227571 14858 raft_consensus.cc:491] T 907301ca08e0467f9f29c6dcb36bdd5e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.227636 14858 raft_consensus.cc:513] T 907301ca08e0467f9f29c6dcb36bdd5e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.227861 14820 tablet_replica.cc:331] T 80aa570e2c154172ad6da29d268bede7 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.227942 14858 leader_election.cc:290] T 907301ca08e0467f9f29c6dcb36bdd5e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:20.227968 14820 raft_consensus.cc:2238] T 80aa570e2c154172ad6da29d268bede7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.228057 14820 raft_consensus.cc:2267] T 80aa570e2c154172ad6da29d268bede7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.228403 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "907301ca08e0467f9f29c6dcb36bdd5e" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:20.229449 14526 tablet_service.cc:1514] Processing DeleteTablet for tablet 293a76d0d49e4a12ab951cbc7311d0f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.229727 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "907301ca08e0467f9f29c6dcb36bdd5e" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:20.229786 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet cc67fb818b8146629d92f9ac1944dba6 (table default.Table_3fab7f59cc324dcf811b852f62ba1f9e [id=0a2e80b35ce148ff85d040843c7a1a2d]) successfully deleted
I20250115 03:06:20.230031 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9979224e3874a7982e1bada8395c921. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.230288 14493 leader_election.cc:304] T 907301ca08e0467f9f29c6dcb36bdd5e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
W20250115 03:06:20.231844 14165 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e47012a789bd469f9691e5a4162ff512 consensus state has no leader
I20250115 03:06:20.231979 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93b979bef1a543ba84092cb255acb7c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.232055 14171 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet 6501c82435154d6e8f2c12b7c82da296
I20250115 03:06:20.234539 14853 raft_consensus.cc:2743] T 907301ca08e0467f9f29c6dcb36bdd5e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.234912 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeb37ffddf484ced806c148940e8f00d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.237136 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aeb6e439d574790a1554f6ff79079c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.237560 14820 ts_tablet_manager.cc:1905] T 80aa570e2c154172ad6da29d268bede7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.239457 14820 ts_tablet_manager.cc:1918] T 80aa570e2c154172ad6da29d268bede7 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.239526 14820 log.cc:1198] T 80aa570e2c154172ad6da29d268bede7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/80aa570e2c154172ad6da29d268bede7
I20250115 03:06:20.239706 14820 ts_tablet_manager.cc:1939] T 80aa570e2c154172ad6da29d268bede7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.241652 14809 tablet_bootstrap.cc:492] T c6fa98c6fb304dc49b8e0ced989cc936 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.241758 14809 ts_tablet_manager.cc:1397] T c6fa98c6fb304dc49b8e0ced989cc936 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.088s user 0.002s sys 0.000s
I20250115 03:06:20.242216 14809 raft_consensus.cc:357] T c6fa98c6fb304dc49b8e0ced989cc936 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.242311 14809 raft_consensus.cc:383] T c6fa98c6fb304dc49b8e0ced989cc936 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.242339 14809 raft_consensus.cc:738] T c6fa98c6fb304dc49b8e0ced989cc936 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.242450 14809 consensus_queue.cc:260] T c6fa98c6fb304dc49b8e0ced989cc936 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.242729 14809 ts_tablet_manager.cc:1428] T c6fa98c6fb304dc49b8e0ced989cc936 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.242904 14809 tablet_bootstrap.cc:492] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.243041 14812 tablet_bootstrap.cc:492] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.243062 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 93b979bef1a543ba84092cb255acb7c2 (DEFAULT_TABLE table=default.Table_802ff80c57304d6393caa12f80fb4929 [id=ef28d0ab1e27446486c90392a13fa4f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.243116 14812 ts_tablet_manager.cc:1397] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.051s user 0.001s sys 0.001s
I20250115 03:06:20.243299 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93b979bef1a543ba84092cb255acb7c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.243549 14812 raft_consensus.cc:357] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.243651 14812 raft_consensus.cc:383] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.243681 14812 raft_consensus.cc:738] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.243808 14809 tablet_bootstrap.cc:654] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.243755 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 93b979bef1a543ba84092cb255acb7c2 (DEFAULT_TABLE table=default.Table_802ff80c57304d6393caa12f80fb4929 [id=ef28d0ab1e27446486c90392a13fa4f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.243790 14812 consensus_queue.cc:260] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.243979 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93b979bef1a543ba84092cb255acb7c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.244072 14812 ts_tablet_manager.cc:1428] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.244251 14812 tablet_bootstrap.cc:492] T 2c69daff62144976ae9082c89cc58eca P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.245115 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 73cbcb3b31624b28bbac7bdaf063c7d9 (DEFAULT_TABLE table=default.Table_472a764a0bfe4cf7ad316cc6de021448 [id=1b12a13cca4a4b34be11ba0adb84954d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.245339 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73cbcb3b31624b28bbac7bdaf063c7d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.245385 14395 tablet_service.cc:1467] Processing CreateTablet for tablet ff57789676224715b5872ead0a8e68fd (DEFAULT_TABLE table=default.Table_3386c15b41014720beb4e38c88d64b08 [id=480f8200824d432e92247a90f43f0b77]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.245589 14395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff57789676224715b5872ead0a8e68fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.246206 14655 tablet_service.cc:1467] Processing CreateTablet for tablet ff57789676224715b5872ead0a8e68fd (DEFAULT_TABLE table=default.Table_3386c15b41014720beb4e38c88d64b08 [id=480f8200824d432e92247a90f43f0b77]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.246423 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff57789676224715b5872ead0a8e68fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.247504 14398 tablet_service.cc:1467] Processing CreateTablet for tablet eeb37ffddf484ced806c148940e8f00d (DEFAULT_TABLE table=default.Table_10784ed50a0948d7a64a54cffcda719e [id=00078131a06346fe84c286b20596d99f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.247702 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeb37ffddf484ced806c148940e8f00d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.248353 14659 tablet_service.cc:1467] Processing CreateTablet for tablet 73cbcb3b31624b28bbac7bdaf063c7d9 (DEFAULT_TABLE table=default.Table_472a764a0bfe4cf7ad316cc6de021448 [id=1b12a13cca4a4b34be11ba0adb84954d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.248451 14812 tablet_bootstrap.cc:654] T 2c69daff62144976ae9082c89cc58eca P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.248580 14659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73cbcb3b31624b28bbac7bdaf063c7d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.249615 14393 tablet_service.cc:1467] Processing CreateTablet for tablet b9979224e3874a7982e1bada8395c921 (DEFAULT_TABLE table=default.Table_4b64655665164e00b508a19e84b9991f [id=c206de1bd1fa4e3db3d80167aca79165]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.249833 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9979224e3874a7982e1bada8395c921. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.250435 14653 tablet_service.cc:1467] Processing CreateTablet for tablet eeb37ffddf484ced806c148940e8f00d (DEFAULT_TABLE table=default.Table_10784ed50a0948d7a64a54cffcda719e [id=00078131a06346fe84c286b20596d99f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.250695 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eeb37ffddf484ced806c148940e8f00d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.251678 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aeb6e439d574790a1554f6ff79079c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.251835 14391 tablet_service.cc:1514] Processing DeleteTablet for tablet 293a76d0d49e4a12ab951cbc7311d0f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.252537 14656 tablet_service.cc:1467] Processing CreateTablet for tablet b9979224e3874a7982e1bada8395c921 (DEFAULT_TABLE table=default.Table_4b64655665164e00b508a19e84b9991f [id=c206de1bd1fa4e3db3d80167aca79165]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.252745 14657 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aeb6e439d574790a1554f6ff79079c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.252861 14670 tablet_service.cc:1514] Processing DeleteTablet for tablet 293a76d0d49e4a12ab951cbc7311d0f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.242882 14399 tablet_service.cc:1467] Processing CreateTablet for tablet d5fa50804f5443d08c1a9723626049de (DEFAULT_TABLE table=default.Table_9a8c2b74534f4bf2b5fc30dff185e54c [id=ef166e3ea16d4bbbb6c35bb4f3993dd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.252913 14809 tablet_bootstrap.cc:492] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.253036 14809 ts_tablet_manager.cc:1397] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20250115 03:06:20.253078 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5fa50804f5443d08c1a9723626049de. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.253540 14809 raft_consensus.cc:357] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.253650 14809 raft_consensus.cc:383] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.253683 14809 raft_consensus.cc:738] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.253793 14809 consensus_queue.cc:260] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.254051 14809 ts_tablet_manager.cc:1428] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:20.254206 14809 tablet_bootstrap.cc:492] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.242736 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 0aeb6e439d574790a1554f6ff79079c6 (DEFAULT_TABLE table=default.Table_ece95328be52409b94bde437fffffc25 [id=ccab6d271d0e4fa1a64e1dc14fe29821]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.255164 14809 tablet_bootstrap.cc:654] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.255184 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0aeb6e439d574790a1554f6ff79079c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.255862 14668 tablet_service.cc:1467] Processing CreateTablet for tablet d5fa50804f5443d08c1a9723626049de (DEFAULT_TABLE table=default.Table_9a8c2b74534f4bf2b5fc30dff185e54c [id=ef166e3ea16d4bbbb6c35bb4f3993dd9]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:20.258335 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dda418ed197e4af2aa511d3f1851663e consensus state has no leader
I20250115 03:06:20.260576 14811 tablet_bootstrap.cc:492] T e5a8293f43134a54b3f8dfb5fb99d6ea P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.260659 14811 ts_tablet_manager.cc:1397] T e5a8293f43134a54b3f8dfb5fb99d6ea P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.044s user 0.001s sys 0.000s
I20250115 03:06:20.260787 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 80aa570e2c154172ad6da29d268bede7 (table default.Table_24cbe6b962b248a9b340506b63c41b20 [id=9866275e65ab4dfcbfee6bf209c3654e]) successfully deleted
W20250115 03:06:20.260916 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 293a76d0d49e4a12ab951cbc7311d0f2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 293a76d0d49e4a12ab951cbc7311d0f2 already in progress: creating tablet
W20250115 03:06:20.261066 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 0aeb6e439d574790a1554f6ff79079c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0aeb6e439d574790a1554f6ff79079c6 already in progress: creating tablet
I20250115 03:06:20.262135 14811 raft_consensus.cc:357] T e5a8293f43134a54b3f8dfb5fb99d6ea P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.262243 14811 raft_consensus.cc:383] T e5a8293f43134a54b3f8dfb5fb99d6ea P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.262276 14811 raft_consensus.cc:738] T e5a8293f43134a54b3f8dfb5fb99d6ea P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.262378 14811 consensus_queue.cc:260] T e5a8293f43134a54b3f8dfb5fb99d6ea P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.262568 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet f2af9569a0254f32b4b6ceab5bfbfa74
I20250115 03:06:20.262634 14811 ts_tablet_manager.cc:1428] T e5a8293f43134a54b3f8dfb5fb99d6ea P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:20.262796 14811 tablet_bootstrap.cc:492] T ca2093980c8e480e99ae0cd79606f318 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.262831 14163 catalog_manager.cc:6596] ReplaceTablet: tablet f2af9569a0254f32b4b6ceab5bfbfa74 deleted and replaced by tablet b1bb007cbce447cc8692ac3647d76403
I20250115 03:06:20.263020 15970 tablet.cc:1721] T d5655935aabd48a19d0ef03739e089a4 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:20.264312 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet f2af9569a0254f32b4b6ceab5bfbfa74 with delete_type TABLET_DATA_DELETED (replaced by tablet b1bb007cbce447cc8692ac3647d76403) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.264652 14656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9979224e3874a7982e1bada8395c921. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.265957 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5fa50804f5443d08c1a9723626049de. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.266669 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_7304b9eba417497a9cead8b57492fbf0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.267007 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7304b9eba417497a9cead8b57492fbf0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.267488 15617 tablet.cc:1721] T d5655935aabd48a19d0ef03739e089a4 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:20.272550 14811 tablet_bootstrap.cc:654] T ca2093980c8e480e99ae0cd79606f318 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.278175 15153 consensus_queue.cc:1035] T e5a8293f43134a54b3f8dfb5fb99d6ea P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:20.278638 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet f2af9569a0254f32b4b6ceab5bfbfa74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2af9569a0254f32b4b6ceab5bfbfa74 already in progress: creating tablet
I20250115 03:06:20.279011 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_93bd5dbdd2344875ba2d8d8a3d0a7591"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.279323 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93bd5dbdd2344875ba2d8d8a3d0a7591 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.280324 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_8b419392614441cab2ca1d2041e0d85a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.280620 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b419392614441cab2ca1d2041e0d85a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.281513 14811 tablet_bootstrap.cc:492] T ca2093980c8e480e99ae0cd79606f318 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.281600 14811 ts_tablet_manager.cc:1397] T ca2093980c8e480e99ae0cd79606f318 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20250115 03:06:20.282017 14811 raft_consensus.cc:357] T ca2093980c8e480e99ae0cd79606f318 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.282114 14811 raft_consensus.cc:383] T ca2093980c8e480e99ae0cd79606f318 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.282141 14811 raft_consensus.cc:738] T ca2093980c8e480e99ae0cd79606f318 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.282246 14811 consensus_queue.cc:260] T ca2093980c8e480e99ae0cd79606f318 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.282486 14811 ts_tablet_manager.cc:1428] T ca2093980c8e480e99ae0cd79606f318 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:20.282644 14811 tablet_bootstrap.cc:492] T 62ec9057d85447bdb6b86cdc1f9015ac P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.283552 14811 tablet_bootstrap.cc:654] T 62ec9057d85447bdb6b86cdc1f9015ac P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.284523 14811 tablet_bootstrap.cc:492] T 62ec9057d85447bdb6b86cdc1f9015ac P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.284596 14811 ts_tablet_manager.cc:1397] T 62ec9057d85447bdb6b86cdc1f9015ac P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
W20250115 03:06:20.284822 14158 catalog_manager.cc:4839] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 0aeb6e439d574790a1554f6ff79079c6 because the tablet was not found. No further retry: Not found: Tablet not found: 0aeb6e439d574790a1554f6ff79079c6
W20250115 03:06:20.284961 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 293a76d0d49e4a12ab951cbc7311d0f2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 293a76d0d49e4a12ab951cbc7311d0f2 already in progress: creating tablet
I20250115 03:06:20.285035 14811 raft_consensus.cc:357] T 62ec9057d85447bdb6b86cdc1f9015ac P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.285125 14811 raft_consensus.cc:383] T 62ec9057d85447bdb6b86cdc1f9015ac P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.285153 14811 raft_consensus.cc:738] T 62ec9057d85447bdb6b86cdc1f9015ac P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.285392 14811 consensus_queue.cc:260] T 62ec9057d85447bdb6b86cdc1f9015ac P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.286477 14811 ts_tablet_manager.cc:1428] T 62ec9057d85447bdb6b86cdc1f9015ac P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:20.286865 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet f2af9569a0254f32b4b6ceab5bfbfa74 with delete_type TABLET_DATA_DELETED (replaced by tablet b1bb007cbce447cc8692ac3647d76403) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.287232 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet f2af9569a0254f32b4b6ceab5bfbfa74 with delete_type TABLET_DATA_DELETED (replaced by tablet b1bb007cbce447cc8692ac3647d76403) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:20.289059 14626 consensus_peers.cc:487] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:20.293200 14811 tablet_bootstrap.cc:492] T c251960c7c9a44429102a294f35f925c P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.293239 14422 raft_consensus.cc:1270] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 5c0dae7aff4b44c699275b95ba639770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:20.294394 14811 tablet_bootstrap.cc:654] T c251960c7c9a44429102a294f35f925c P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.294461 15153 consensus_queue.cc:1035] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:20.295863 14811 tablet_bootstrap.cc:492] T c251960c7c9a44429102a294f35f925c P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.295948 14811 ts_tablet_manager.cc:1397] T c251960c7c9a44429102a294f35f925c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:20.296352 14811 raft_consensus.cc:357] T c251960c7c9a44429102a294f35f925c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.296448 14811 raft_consensus.cc:383] T c251960c7c9a44429102a294f35f925c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.296478 14811 raft_consensus.cc:738] T c251960c7c9a44429102a294f35f925c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.296576 14811 consensus_queue.cc:260] T c251960c7c9a44429102a294f35f925c P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.296828 14811 ts_tablet_manager.cc:1428] T c251960c7c9a44429102a294f35f925c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:20.297011 14811 tablet_bootstrap.cc:492] T da95daa40c5b4615a0219f6d8e3770f6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.297844 14811 tablet_bootstrap.cc:654] T da95daa40c5b4615a0219f6d8e3770f6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.299698 14811 tablet_bootstrap.cc:492] T da95daa40c5b4615a0219f6d8e3770f6 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.299782 14811 ts_tablet_manager.cc:1397] T da95daa40c5b4615a0219f6d8e3770f6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:20.300125 14162 master_service.cc:946] ReplaceTablet: received request to replace tablet 6004f4adca8e47ddb91f7e39c4cf9089 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:20.300165 14811 raft_consensus.cc:357] T da95daa40c5b4615a0219f6d8e3770f6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.300244 14811 raft_consensus.cc:383] T da95daa40c5b4615a0219f6d8e3770f6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.300271 14811 raft_consensus.cc:738] T da95daa40c5b4615a0219f6d8e3770f6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.300366 14811 consensus_queue.cc:260] T da95daa40c5b4615a0219f6d8e3770f6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.300609 14811 ts_tablet_manager.cc:1428] T da95daa40c5b4615a0219f6d8e3770f6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20250115 03:06:20.302662 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet f2af9569a0254f32b4b6ceab5bfbfa74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2af9569a0254f32b4b6ceab5bfbfa74 already in progress: creating tablet
I20250115 03:06:20.304462 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet c6fa98c6fb304dc49b8e0ced989cc936 (table default.Table_9f442bb6e9024025b7fa97a7ddd069f5 [id=8f805d745ece4ec8845a43798122dd4f]). Expected version 1 got 0
I20250115 03:06:20.308061 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 62ec9057d85447bdb6b86cdc1f9015ac with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.308211 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet ca2093980c8e480e99ae0cd79606f318 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.308473 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 839bdd3580024e849e9e1439ade8d3d9 (DEFAULT_TABLE table=default.Table_bca65e8db1d94382a3528cf21a111c64 [id=3353d1e096344f14ac749ef381173db7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.308768 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 839bdd3580024e849e9e1439ade8d3d9 (DEFAULT_TABLE table=default.Table_bca65e8db1d94382a3528cf21a111c64 [id=3353d1e096344f14ac749ef381173db7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.309027 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 839bdd3580024e849e9e1439ade8d3d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.310189 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 6004f4adca8e47ddb91f7e39c4cf9089
I20250115 03:06:20.310446 14162 catalog_manager.cc:6596] ReplaceTablet: tablet 6004f4adca8e47ddb91f7e39c4cf9089 deleted and replaced by tablet a198e74fd7db4247a673ccb834e7b150
I20250115 03:06:20.310631 14809 tablet_bootstrap.cc:492] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.310701 14809 ts_tablet_manager.cc:1397] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.057s user 0.000s sys 0.001s
I20250115 03:06:20.311087 14809 raft_consensus.cc:357] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.311174 14809 raft_consensus.cc:383] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.311203 14809 raft_consensus.cc:738] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.311314 14809 consensus_queue.cc:260] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.311373 14668 tablet_service.cc:1467] Processing CreateTablet for tablet 839bdd3580024e849e9e1439ade8d3d9 (DEFAULT_TABLE table=default.Table_bca65e8db1d94382a3528cf21a111c64 [id=3353d1e096344f14ac749ef381173db7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.311609 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 839bdd3580024e849e9e1439ade8d3d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.311614 14809 ts_tablet_manager.cc:1428] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.311764 14809 tablet_bootstrap.cc:492] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.312251 14163 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_a37ebbac9e4843e9a3376be17c3dec26 [id=ed8713b68f234c46bc2ef1dae0059a2f] alter complete (version 1)
I20250115 03:06:20.312639 14811 tablet_bootstrap.cc:492] T e2c6db404348466ea5211ffa69c8f01c P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.313519 14811 tablet_bootstrap.cc:654] T e2c6db404348466ea5211ffa69c8f01c P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.313895 14809 tablet_bootstrap.cc:654] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.314126 14553 raft_consensus.cc:3054] T 907301ca08e0467f9f29c6dcb36bdd5e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.314119 14656 tablet_service.cc:1514] Processing DeleteTablet for tablet 6004f4adca8e47ddb91f7e39c4cf9089 with delete_type TABLET_DATA_DELETED (replaced by tablet a198e74fd7db4247a673ccb834e7b150) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.314381 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet 6004f4adca8e47ddb91f7e39c4cf9089 with delete_type TABLET_DATA_DELETED (replaced by tablet a198e74fd7db4247a673ccb834e7b150) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.315394 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 6004f4adca8e47ddb91f7e39c4cf9089 with delete_type TABLET_DATA_DELETED (replaced by tablet a198e74fd7db4247a673ccb834e7b150) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:20.319335 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 6004f4adca8e47ddb91f7e39c4cf9089 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6004f4adca8e47ddb91f7e39c4cf9089 already in progress: creating tablet
I20250115 03:06:20.321117 14811 tablet_bootstrap.cc:492] T e2c6db404348466ea5211ffa69c8f01c P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.321198 14811 ts_tablet_manager.cc:1397] T e2c6db404348466ea5211ffa69c8f01c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20250115 03:06:20.321653 14811 raft_consensus.cc:357] T e2c6db404348466ea5211ffa69c8f01c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.321763 14811 raft_consensus.cc:383] T e2c6db404348466ea5211ffa69c8f01c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.321799 14811 raft_consensus.cc:738] T e2c6db404348466ea5211ffa69c8f01c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.321897 14811 consensus_queue.cc:260] T e2c6db404348466ea5211ffa69c8f01c P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.322147 14811 ts_tablet_manager.cc:1428] T e2c6db404348466ea5211ffa69c8f01c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.322425 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet c251960c7c9a44429102a294f35f925c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.322625 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet da95daa40c5b4615a0219f6d8e3770f6 with delete_type TABLET_DATA_DELETED (replaced by tablet 678b4453675b42ab98832a20f1b0271b) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.322829 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e2ad21250b7046048d799b38c19d91ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.323434 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e2ad21250b7046048d799b38c19d91ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.325340 14820 tablet_replica.cc:331] T 62ec9057d85447bdb6b86cdc1f9015ac P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.325481 14820 raft_consensus.cc:2238] T 62ec9057d85447bdb6b86cdc1f9015ac P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.325476 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 839bdd3580024e849e9e1439ade8d3d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.325594 14820 raft_consensus.cc:2267] T 62ec9057d85447bdb6b86cdc1f9015ac P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.326610 14820 ts_tablet_manager.cc:1905] T 62ec9057d85447bdb6b86cdc1f9015ac P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.327759 14811 tablet_bootstrap.cc:492] T 623887ec17374f598f48054e17582f3f P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.328699 14820 ts_tablet_manager.cc:1918] T 62ec9057d85447bdb6b86cdc1f9015ac P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.328788 14820 log.cc:1198] T 62ec9057d85447bdb6b86cdc1f9015ac P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/62ec9057d85447bdb6b86cdc1f9015ac
I20250115 03:06:20.329654 14820 ts_tablet_manager.cc:1939] T 62ec9057d85447bdb6b86cdc1f9015ac P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.329947 14820 tablet_replica.cc:331] T ca2093980c8e480e99ae0cd79606f318 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.329952 14809 tablet_bootstrap.cc:492] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.330020 14809 ts_tablet_manager.cc:1397] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.000s
I20250115 03:06:20.330046 14820 raft_consensus.cc:2238] T ca2093980c8e480e99ae0cd79606f318 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.330073 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 62ec9057d85447bdb6b86cdc1f9015ac (table default.Table_5a68d6c8c53b4746a7123860185fe0cc [id=84c18cf488cc4c28aaf93fa72334acb7]) successfully deleted
I20250115 03:06:20.330130 14820 raft_consensus.cc:2267] T ca2093980c8e480e99ae0cd79606f318 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.330569 14820 ts_tablet_manager.cc:1905] T ca2093980c8e480e99ae0cd79606f318 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.328723 14811 tablet_bootstrap.cc:654] T 623887ec17374f598f48054e17582f3f P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.331915 14809 raft_consensus.cc:357] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.332077 14809 raft_consensus.cc:383] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.332150 14809 raft_consensus.cc:738] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.332625 14809 consensus_queue.cc:260] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.332890 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 3bc1f0f72a3b4c79847c0a91fe98ccd1 (table default.Table_b165e5d12ca845b9984c99dde5e2e06e [id=dfe0ca35843c4fc6b975dd1ab87e9a99]). Expected version 1 got 0
I20250115 03:06:20.333014 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 6bcfde5fe3d4425e98edb4dc54096718 (table default.Table_a63d6344914343c4af56eef21f467c6f [id=f45053c781fc460ab55f592eed6fb946]). Expected version 1 got 0
I20250115 03:06:20.333683 14820 ts_tablet_manager.cc:1918] T ca2093980c8e480e99ae0cd79606f318 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.333808 14820 log.cc:1198] T ca2093980c8e480e99ae0cd79606f318 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/ca2093980c8e480e99ae0cd79606f318
I20250115 03:06:20.333948 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet e2c6db404348466ea5211ffa69c8f01c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:15 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.334218 14820 ts_tablet_manager.cc:1939] T ca2093980c8e480e99ae0cd79606f318 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.334782 14820 tablet_replica.cc:331] T c251960c7c9a44429102a294f35f925c P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.334928 14820 raft_consensus.cc:2238] T c251960c7c9a44429102a294f35f925c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.335031 14820 raft_consensus.cc:2267] T c251960c7c9a44429102a294f35f925c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.335175 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet ca2093980c8e480e99ae0cd79606f318 (table default.Table_0aecd3e601c04f0497cefafda025e6e1 [id=bde9696f4c534fe58bb1de4fd7e00201]) successfully deleted
W20250115 03:06:20.335337 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 6004f4adca8e47ddb91f7e39c4cf9089 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6004f4adca8e47ddb91f7e39c4cf9089 already in progress: creating tablet
I20250115 03:06:20.335685 14820 ts_tablet_manager.cc:1905] T c251960c7c9a44429102a294f35f925c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.336640 14162 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 676ec8b9dc8241d4ae40488d3ce8f61e (table default.Table_0b6cd33c30014981b78899a738418cbf [id=63edd09bfaaa4d56b4314ce760c87d3d]). Expected version 1 got 0
I20250115 03:06:20.338869 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet d2564a20857f45d9a0c7e3b74eb58a91 (table default.Table_7c8f638a9380432c844dc355d43df492 [id=6a9388b294f24f358a5d5f34568e8a13]) successfully deleted
I20250115 03:06:20.339023 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_9f442bb6e9024025b7fa97a7ddd069f5" } modify_external_catalogs: true
I20250115 03:06:20.339145 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_9f442bb6e9024025b7fa97a7ddd069f5" } modify_external_catalogs: true
W20250115 03:06:20.339056 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 0aeb6e439d574790a1554f6ff79079c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0aeb6e439d574790a1554f6ff79079c6 already in progress: creating tablet
W20250115 03:06:20.339802 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 293a76d0d49e4a12ab951cbc7311d0f2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 293a76d0d49e4a12ab951cbc7311d0f2 already in progress: creating tablet
I20250115 03:06:20.340087 14809 ts_tablet_manager.cc:1428] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
I20250115 03:06:20.341203 14809 tablet_bootstrap.cc:492] T b8ec24d2937d44e79b839276bf615fa0 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.343220 14809 tablet_bootstrap.cc:654] T b8ec24d2937d44e79b839276bf615fa0 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.345698 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_397a360e045b4b5fabed16c0feff8112" } modify_external_catalogs: true
I20250115 03:06:20.345973 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_397a360e045b4b5fabed16c0feff8112" } modify_external_catalogs: true
I20250115 03:06:20.347090 14168 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 676ec8b9dc8241d4ae40488d3ce8f61e (table default.Table_0b6cd33c30014981b78899a738418cbf [id=63edd09bfaaa4d56b4314ce760c87d3d]). Expected version 1 got 0
I20250115 03:06:20.347551 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet c6fa98c6fb304dc49b8e0ced989cc936
I20250115 03:06:20.351770 14820 ts_tablet_manager.cc:1918] T c251960c7c9a44429102a294f35f925c P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.351851 14820 log.cc:1198] T c251960c7c9a44429102a294f35f925c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/c251960c7c9a44429102a294f35f925c
I20250115 03:06:20.352057 14820 ts_tablet_manager.cc:1939] T c251960c7c9a44429102a294f35f925c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.352355 14820 tablet_replica.cc:331] T da95daa40c5b4615a0219f6d8e3770f6 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.352487 14820 raft_consensus.cc:2238] T da95daa40c5b4615a0219f6d8e3770f6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.352581 14820 raft_consensus.cc:2267] T da95daa40c5b4615a0219f6d8e3770f6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.353232 14820 ts_tablet_manager.cc:1905] T da95daa40c5b4615a0219f6d8e3770f6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.354116 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet c251960c7c9a44429102a294f35f925c (table default.Table_90aedfcdb1274e1497cf36620b980768 [id=9e4f5de45eb24beabef550d15dcad8bf]) successfully deleted
I20250115 03:06:20.354168 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet c6fa98c6fb304dc49b8e0ced989cc936 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.355330 14820 ts_tablet_manager.cc:1918] T da95daa40c5b4615a0219f6d8e3770f6 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.355437 14820 log.cc:1198] T da95daa40c5b4615a0219f6d8e3770f6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/da95daa40c5b4615a0219f6d8e3770f6
I20250115 03:06:20.355656 14820 ts_tablet_manager.cc:1939] T da95daa40c5b4615a0219f6d8e3770f6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.355966 14820 tablet_replica.cc:331] T e2c6db404348466ea5211ffa69c8f01c P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.356057 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet c6fa98c6fb304dc49b8e0ced989cc936 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.356223 14819 tablet_replica.cc:331] T c6fa98c6fb304dc49b8e0ced989cc936 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:20.356240 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet c6fa98c6fb304dc49b8e0ced989cc936 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.356338 14819 raft_consensus.cc:2238] T c6fa98c6fb304dc49b8e0ced989cc936 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.356433 14819 raft_consensus.cc:2267] T c6fa98c6fb304dc49b8e0ced989cc936 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.357127 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet da95daa40c5b4615a0219f6d8e3770f6 (table default.Table_4bde77af131940a099eabda16fa2ccb2 [id=4f952e57c86746f6bb1df72e13589832]) successfully deleted
I20250115 03:06:20.357218 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet b9979224e3874a7982e1bada8395c921 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:20.357396 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet ef7cd35eb1614a29a9197e9c03185f7d
I20250115 03:06:20.357925 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet ef7cd35eb1614a29a9197e9c03185f7d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.358368 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet ef7cd35eb1614a29a9197e9c03185f7d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.358820 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet ef7cd35eb1614a29a9197e9c03185f7d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.356127 14820 raft_consensus.cc:2238] T e2c6db404348466ea5211ffa69c8f01c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.359630 14820 raft_consensus.cc:2267] T e2c6db404348466ea5211ffa69c8f01c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.361020 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet b9979224e3874a7982e1bada8395c921
I20250115 03:06:20.361128 14819 ts_tablet_manager.cc:1905] T c6fa98c6fb304dc49b8e0ced989cc936 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.361279 14163 catalog_manager.cc:6596] ReplaceTablet: tablet b9979224e3874a7982e1bada8395c921 deleted and replaced by tablet f5480dab59fb4399870a4c9fec977b3e
I20250115 03:06:20.361754 14820 ts_tablet_manager.cc:1905] T e2c6db404348466ea5211ffa69c8f01c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.360453 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_a6989a57431e4523afea72558cea88c2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.362367 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a6989a57431e4523afea72558cea88c2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.363415 14819 ts_tablet_manager.cc:1918] T c6fa98c6fb304dc49b8e0ced989cc936 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.363495 14819 log.cc:1198] T c6fa98c6fb304dc49b8e0ced989cc936 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/c6fa98c6fb304dc49b8e0ced989cc936
I20250115 03:06:20.363525 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet b9979224e3874a7982e1bada8395c921 with delete_type TABLET_DATA_DELETED (replaced by tablet f5480dab59fb4399870a4c9fec977b3e) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.363682 14819 ts_tablet_manager.cc:1939] T c6fa98c6fb304dc49b8e0ced989cc936 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
W20250115 03:06:20.363709 14626 consensus_peers.cc:487] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:20.363838 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet b9979224e3874a7982e1bada8395c921 with delete_type TABLET_DATA_DELETED (replaced by tablet f5480dab59fb4399870a4c9fec977b3e) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.364168 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet b9979224e3874a7982e1bada8395c921 with delete_type TABLET_DATA_DELETED (replaced by tablet f5480dab59fb4399870a4c9fec977b3e) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.364893 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet c6fa98c6fb304dc49b8e0ced989cc936 (table default.Table_9f442bb6e9024025b7fa97a7ddd069f5 [id=8f805d745ece4ec8845a43798122dd4f]) successfully deleted
I20250115 03:06:20.366647 14820 ts_tablet_manager.cc:1918] T e2c6db404348466ea5211ffa69c8f01c P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.366709 14820 log.cc:1198] T e2c6db404348466ea5211ffa69c8f01c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/e2c6db404348466ea5211ffa69c8f01c
I20250115 03:06:20.366904 14820 ts_tablet_manager.cc:1939] T e2c6db404348466ea5211ffa69c8f01c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.367605 15390 raft_consensus.cc:491] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.367678 15390 raft_consensus.cc:513] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.367988 15390 leader_election.cc:290] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:20.368332 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9febec144bb343a8a21817e6d3655ad7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:20.368588 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9febec144bb343a8a21817e6d3655ad7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:20.368669 14358 leader_election.cc:343] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:20.368886 14356 leader_election.cc:343] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.368963 14356 leader_election.cc:304] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:20.369128 15390 raft_consensus.cc:2743] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.369951 14809 tablet_bootstrap.cc:492] T b8ec24d2937d44e79b839276bf615fa0 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.370045 14809 ts_tablet_manager.cc:1397] T b8ec24d2937d44e79b839276bf615fa0 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20250115 03:06:20.370525 14809 raft_consensus.cc:357] T b8ec24d2937d44e79b839276bf615fa0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.370635 14809 raft_consensus.cc:383] T b8ec24d2937d44e79b839276bf615fa0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.370692 14809 raft_consensus.cc:738] T b8ec24d2937d44e79b839276bf615fa0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.370849 14809 consensus_queue.cc:260] T b8ec24d2937d44e79b839276bf615fa0 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.371165 14809 ts_tablet_manager.cc:1428] T b8ec24d2937d44e79b839276bf615fa0 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.371551 14809 tablet_bootstrap.cc:492] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.372507 14809 tablet_bootstrap.cc:654] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.373692 14809 tablet_bootstrap.cc:492] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.373838 14809 ts_tablet_manager.cc:1397] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:20.374357 14809 raft_consensus.cc:357] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.374508 14809 raft_consensus.cc:383] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.374583 14809 raft_consensus.cc:738] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.374781 14809 consensus_queue.cc:260] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.375170 14809 ts_tablet_manager.cc:1428] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.375384 14809 tablet_bootstrap.cc:492] T 85cceb5a3f234d85937ad0bb75348991 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.375844 14811 tablet_bootstrap.cc:492] T 623887ec17374f598f48054e17582f3f P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.375927 14811 ts_tablet_manager.cc:1397] T 623887ec17374f598f48054e17582f3f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.048s user 0.002s sys 0.000s
I20250115 03:06:20.376368 14811 raft_consensus.cc:357] T 623887ec17374f598f48054e17582f3f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.376461 14811 raft_consensus.cc:383] T 623887ec17374f598f48054e17582f3f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.376493 14811 raft_consensus.cc:738] T 623887ec17374f598f48054e17582f3f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.376607 14811 consensus_queue.cc:260] T 623887ec17374f598f48054e17582f3f P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.376624 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet e2c6db404348466ea5211ffa69c8f01c (table default.Table_ef19dd8e16aa4bf9a97897ebc1c8eef5 [id=38bfb0375c334d16b97c7251d6ddd76b]) successfully deleted
I20250115 03:06:20.376889 14811 ts_tablet_manager.cc:1428] T 623887ec17374f598f48054e17582f3f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20250115 03:06:20.377208 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet c6fa98c6fb304dc49b8e0ced989cc936 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c6fa98c6fb304dc49b8e0ced989cc936 already in progress: creating tablet
I20250115 03:06:20.377831 14811 tablet_bootstrap.cc:492] T 312dcdda8cf042648db9fdf6228062d9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.378743 14811 tablet_bootstrap.cc:654] T 312dcdda8cf042648db9fdf6228062d9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.378867 16292 consensus_queue.cc:1035] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:20.379595 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet ef7cd35eb1614a29a9197e9c03185f7d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ef7cd35eb1614a29a9197e9c03185f7d already in progress: creating tablet
W20250115 03:06:20.379809 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet b9979224e3874a7982e1bada8395c921 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b9979224e3874a7982e1bada8395c921 already in progress: creating tablet
I20250115 03:06:20.380167 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet b8ec24d2937d44e79b839276bf615fa0 with delete_type TABLET_DATA_DELETED (replaced by tablet 4febfae2f2794669b5997a1fa7ef7412) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.380529 14811 tablet_bootstrap.cc:492] T 312dcdda8cf042648db9fdf6228062d9 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.380646 14811 ts_tablet_manager.cc:1397] T 312dcdda8cf042648db9fdf6228062d9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:20.380859 14819 tablet_replica.cc:331] T b8ec24d2937d44e79b839276bf615fa0 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:20.380995 14819 raft_consensus.cc:2238] T b8ec24d2937d44e79b839276bf615fa0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.381083 14819 raft_consensus.cc:2267] T b8ec24d2937d44e79b839276bf615fa0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.381177 14811 raft_consensus.cc:357] T 312dcdda8cf042648db9fdf6228062d9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.381366 14811 raft_consensus.cc:383] T 312dcdda8cf042648db9fdf6228062d9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.381439 14811 raft_consensus.cc:738] T 312dcdda8cf042648db9fdf6228062d9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.381474 14819 ts_tablet_manager.cc:1905] T b8ec24d2937d44e79b839276bf615fa0 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.381618 14811 consensus_queue.cc:260] T 312dcdda8cf042648db9fdf6228062d9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.381927 14858 raft_consensus.cc:491] T 5737f10aa50e4059a814585019346872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.381992 14858 raft_consensus.cc:513] T 5737f10aa50e4059a814585019346872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.382297 14858 leader_election.cc:290] T 5737f10aa50e4059a814585019346872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:20.382647 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5737f10aa50e4059a814585019346872" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:20.382742 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5737f10aa50e4059a814585019346872" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:20.383070 14493 leader_election.cc:304] T 5737f10aa50e4059a814585019346872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:20.383204 14858 raft_consensus.cc:2743] T 5737f10aa50e4059a814585019346872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.383100 14811 ts_tablet_manager.cc:1428] T 312dcdda8cf042648db9fdf6228062d9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:20.383618 14819 ts_tablet_manager.cc:1918] T b8ec24d2937d44e79b839276bf615fa0 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.383680 14819 log.cc:1198] T b8ec24d2937d44e79b839276bf615fa0 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/b8ec24d2937d44e79b839276bf615fa0
I20250115 03:06:20.383886 14819 ts_tablet_manager.cc:1939] T b8ec24d2937d44e79b839276bf615fa0 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
W20250115 03:06:20.384996 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet ef7cd35eb1614a29a9197e9c03185f7d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ef7cd35eb1614a29a9197e9c03185f7d already in progress: creating tablet
I20250115 03:06:20.385442 14809 tablet_bootstrap.cc:654] T 85cceb5a3f234d85937ad0bb75348991 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.385865 14811 tablet_bootstrap.cc:492] T 15fbf8d5de904fc4b284af664f3bd60f P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.386799 14811 tablet_bootstrap.cc:654] T 15fbf8d5de904fc4b284af664f3bd60f P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:20.387843 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet b9979224e3874a7982e1bada8395c921 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b9979224e3874a7982e1bada8395c921 already in progress: creating tablet
I20250115 03:06:20.388181 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 623887ec17374f598f48054e17582f3f with delete_type TABLET_DATA_DELETED (replaced by tablet 12678da7ffee4a46b4690ac3e94cb19b) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.389334 14820 tablet_replica.cc:331] T 623887ec17374f598f48054e17582f3f P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.389451 14820 raft_consensus.cc:2238] T 623887ec17374f598f48054e17582f3f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.389542 14820 raft_consensus.cc:2267] T 623887ec17374f598f48054e17582f3f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.389973 14820 ts_tablet_manager.cc:1905] T 623887ec17374f598f48054e17582f3f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.392234 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 312dcdda8cf042648db9fdf6228062d9 with delete_type TABLET_DATA_DELETED (replaced by tablet 7b86e66428504217af08f667f660d304) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.393426 15617 tablet.cc:1721] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:20.393662 14812 tablet_bootstrap.cc:492] T 2c69daff62144976ae9082c89cc58eca P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.393604 16292 raft_consensus.cc:491] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.393731 14812 ts_tablet_manager.cc:1397] T 2c69daff62144976ae9082c89cc58eca P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.150s user 0.002s sys 0.000s
I20250115 03:06:20.394194 15970 tablet.cc:1721] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:20.394294 14812 raft_consensus.cc:357] T 2c69daff62144976ae9082c89cc58eca P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.394398 14812 raft_consensus.cc:383] T 2c69daff62144976ae9082c89cc58eca P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.394426 14812 raft_consensus.cc:738] T 2c69daff62144976ae9082c89cc58eca P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.394543 14812 consensus_queue.cc:260] T 2c69daff62144976ae9082c89cc58eca P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.394799 14812 ts_tablet_manager.cc:1428] T 2c69daff62144976ae9082c89cc58eca P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.394963 14812 tablet_bootstrap.cc:492] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.395859 14812 tablet_bootstrap.cc:654] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.393723 16292 raft_consensus.cc:513] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.397600 16292 leader_election.cc:290] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:20.398367 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd161e290914d9d8671c10becfeaee1" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:20.398707 14626 leader_election.cc:343] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.399103 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd161e290914d9d8671c10becfeaee1" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:20.399219 14421 raft_consensus.cc:2463] T fbd161e290914d9d8671c10becfeaee1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:20.399791 14628 leader_election.cc:304] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:20.399947 14553 raft_consensus.cc:3054] T 5737f10aa50e4059a814585019346872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.400300 16292 raft_consensus.cc:2798] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:20.400408 16292 raft_consensus.cc:491] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:20.402679 14168 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_4b1d421298a845129d33322313978924 [id=b298dce7ab664327893633f821ed4d89] alter complete (version 2)
I20250115 03:06:20.401175 16088 consensus_queue.cc:1035] T 5737f10aa50e4059a814585019346872 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:20.402907 14858 raft_consensus.cc:491] T e5a8293f43134a54b3f8dfb5fb99d6ea P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.402978 14858 raft_consensus.cc:513] T e5a8293f43134a54b3f8dfb5fb99d6ea P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.403275 14858 leader_election.cc:290] T e5a8293f43134a54b3f8dfb5fb99d6ea P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:20.403726 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5a8293f43134a54b3f8dfb5fb99d6ea" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:20.403869 16292 raft_consensus.cc:3054] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.404104 14820 ts_tablet_manager.cc:1918] T 623887ec17374f598f48054e17582f3f P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.404214 14820 log.cc:1198] T 623887ec17374f598f48054e17582f3f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/623887ec17374f598f48054e17582f3f
I20250115 03:06:20.404449 14820 ts_tablet_manager.cc:1939] T 623887ec17374f598f48054e17582f3f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.405660 14820 tablet_replica.cc:331] T 312dcdda8cf042648db9fdf6228062d9 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.405823 14820 raft_consensus.cc:2238] T 312dcdda8cf042648db9fdf6228062d9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.405807 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5a8293f43134a54b3f8dfb5fb99d6ea" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:20.405964 14820 raft_consensus.cc:2267] T 312dcdda8cf042648db9fdf6228062d9 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.406128 14493 leader_election.cc:304] T e5a8293f43134a54b3f8dfb5fb99d6ea P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:20.406322 14858 raft_consensus.cc:2743] T e5a8293f43134a54b3f8dfb5fb99d6ea P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.406986 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 623887ec17374f598f48054e17582f3f (table default.Table_0337ad43f9b646a48c61c66c15f1844a [id=7f5506ea0a5e465cba8de1ba2f5e75c6]) successfully deleted
I20250115 03:06:20.408612 14809 tablet_bootstrap.cc:492] T 85cceb5a3f234d85937ad0bb75348991 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.408727 14809 ts_tablet_manager.cc:1397] T 85cceb5a3f234d85937ad0bb75348991 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
I20250115 03:06:20.409248 14809 raft_consensus.cc:357] T 85cceb5a3f234d85937ad0bb75348991 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.409399 14809 raft_consensus.cc:383] T 85cceb5a3f234d85937ad0bb75348991 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.409473 14809 raft_consensus.cc:738] T 85cceb5a3f234d85937ad0bb75348991 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.409633 14809 consensus_queue.cc:260] T 85cceb5a3f234d85937ad0bb75348991 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.409963 14809 ts_tablet_manager.cc:1428] T 85cceb5a3f234d85937ad0bb75348991 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.410166 14809 tablet_bootstrap.cc:492] T 80ede4e9696d491aa385b375fcb5cf3b P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.411088 14809 tablet_bootstrap.cc:654] T 80ede4e9696d491aa385b375fcb5cf3b P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.414757 16292 raft_consensus.cc:513] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.415182 16292 leader_election.cc:290] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:20.415421 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c69daff62144976ae9082c89cc58eca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.415411 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd161e290914d9d8671c10becfeaee1" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:20.415505 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbd161e290914d9d8671c10becfeaee1" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:20.415589 14421 raft_consensus.cc:3054] T fbd161e290914d9d8671c10becfeaee1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.415978 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet b8ec24d2937d44e79b839276bf615fa0 (table default.Table_83fe43c0616b4351b2b4457195378f0c [id=7bec6149db914c939bc6fd50e47097b2]) successfully deleted
I20250115 03:06:20.415948 14527 tablet_service.cc:1467] Processing CreateTablet for tablet e179d1fd2d614a0e97112256babaf2a3 (DEFAULT_TABLE table=default.Table_93bd5dbdd2344875ba2d8d8a3d0a7591 [id=8dee15cef7b940bc90ad0ec0b37aa2e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.416180 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e179d1fd2d614a0e97112256babaf2a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.416457 14421 raft_consensus.cc:2463] T fbd161e290914d9d8671c10becfeaee1 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:20.416718 14628 leader_election.cc:304] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:20.418399 14397 tablet_service.cc:1467] Processing CreateTablet for tablet e179d1fd2d614a0e97112256babaf2a3 (DEFAULT_TABLE table=default.Table_93bd5dbdd2344875ba2d8d8a3d0a7591 [id=8dee15cef7b940bc90ad0ec0b37aa2e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.418709 14656 tablet_service.cc:1467] Processing CreateTablet for tablet 46c9ee4fb83d4822a4fd50f65975c6f5 (DEFAULT_TABLE table=default.Table_8b419392614441cab2ca1d2041e0d85a [id=2e8d56a2d141407ab96e4b8ec380054c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.418951 14656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46c9ee4fb83d4822a4fd50f65975c6f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.418968 14820 ts_tablet_manager.cc:1905] T 312dcdda8cf042648db9fdf6228062d9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.419234 16292 raft_consensus.cc:2798] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:20.419368 14670 tablet_service.cc:1467] Processing CreateTablet for tablet ed7bdf12b64b4c9e9dc3fb43330a0936 (DEFAULT_TABLE table=default.Table_5072bde4d75d4184a223755f60397e83 [id=66ff7d4342e54548adf8319023ae243b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.419590 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed7bdf12b64b4c9e9dc3fb43330a0936. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.419572 16292 raft_consensus.cc:695] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:20.420378 16292 consensus_queue.cc:237] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.420840 14820 ts_tablet_manager.cc:1918] T 312dcdda8cf042648db9fdf6228062d9 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.420893 14820 log.cc:1198] T 312dcdda8cf042648db9fdf6228062d9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/312dcdda8cf042648db9fdf6228062d9
I20250115 03:06:20.421036 14534 tablet_service.cc:1467] Processing CreateTablet for tablet 46c9ee4fb83d4822a4fd50f65975c6f5 (DEFAULT_TABLE table=default.Table_8b419392614441cab2ca1d2041e0d85a [id=2e8d56a2d141407ab96e4b8ec380054c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.421109 14820 ts_tablet_manager.cc:1939] T 312dcdda8cf042648db9fdf6228062d9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.421247 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46c9ee4fb83d4822a4fd50f65975c6f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.423156 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 46c9ee4fb83d4822a4fd50f65975c6f5 (DEFAULT_TABLE table=default.Table_8b419392614441cab2ca1d2041e0d85a [id=2e8d56a2d141407ab96e4b8ec380054c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.423404 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 56339d69e99d44bba448522ba41ab50e (DEFAULT_TABLE table=default.Table_7304b9eba417497a9cead8b57492fbf0 [id=74723539e8c9483cb65191a2ba41d945]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.423604 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56339d69e99d44bba448522ba41ab50e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.423890 15401 raft_consensus.cc:491] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.423962 15401 raft_consensus.cc:513] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.424273 15401 leader_election.cc:290] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:20.424535 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc1f0f72a3b4c79847c0a91fe98ccd1" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:20.424836 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc1f0f72a3b4c79847c0a91fe98ccd1" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:20.425107 14356 leader_election.cc:343] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:20.425330 14358 leader_election.cc:343] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.425392 14358 leader_election.cc:304] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:20.425563 15401 raft_consensus.cc:2743] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.425581 14671 tablet_service.cc:1467] Processing CreateTablet for tablet 56339d69e99d44bba448522ba41ab50e (DEFAULT_TABLE table=default.Table_7304b9eba417497a9cead8b57492fbf0 [id=74723539e8c9483cb65191a2ba41d945]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.425763 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 312dcdda8cf042648db9fdf6228062d9 (table default.Table_341a3cd1c8c94ff4b05fc21960a4e9b4 [id=c096ce170874405ea740b9ea829f886e]) successfully deleted
I20250115 03:06:20.425798 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56339d69e99d44bba448522ba41ab50e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.427664 14521 tablet_service.cc:1467] Processing CreateTablet for tablet d9ddca2936db4adb92c8ccb34693e24c (DEFAULT_TABLE table=default.Table_09dd39785939469795df4097f8b9a6a4 [id=9f1c1f1ed8e548d396a63d984cda85b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.427860 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9ddca2936db4adb92c8ccb34693e24c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.429780 14530 tablet_service.cc:1467] Processing CreateTablet for tablet ed7bdf12b64b4c9e9dc3fb43330a0936 (DEFAULT_TABLE table=default.Table_5072bde4d75d4184a223755f60397e83 [id=66ff7d4342e54548adf8319023ae243b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.429982 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed7bdf12b64b4c9e9dc3fb43330a0936. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.429944 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_30324f936db349899fc29c4213990c43" } modify_external_catalogs: true
I20250115 03:06:20.430047 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_30324f936db349899fc29c4213990c43" } modify_external_catalogs: true
I20250115 03:06:20.430032 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_def871c9b5dc4a4b8fdc40fe12ccaf7a" } new_table_name: "default.Table_ef9433c3090f4deea9bf840e58e460e5" modify_external_catalogs: true
I20250115 03:06:20.430559 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_49a6ee8663924497be95c6721d2e5887"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:20.430691 14168 master_service.cc:946] ReplaceTablet: received request to replace tablet 0687887b68b848d78f19f5674f269fb3 from {username='slave'} at 127.0.0.1:60432
W20250115 03:06:20.430886 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_49a6ee8663924497be95c6721d2e5887 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.431977 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 56339d69e99d44bba448522ba41ab50e (DEFAULT_TABLE table=default.Table_7304b9eba417497a9cead8b57492fbf0 [id=74723539e8c9483cb65191a2ba41d945]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.432085 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_1a09d6243c154136aaa074ea2bfd7f6d" } modify_external_catalogs: true
I20250115 03:06:20.432165 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_1a09d6243c154136aaa074ea2bfd7f6d" } modify_external_catalogs: true
I20250115 03:06:20.432235 14653 tablet_service.cc:1467] Processing CreateTablet for tablet d9ddca2936db4adb92c8ccb34693e24c (DEFAULT_TABLE table=default.Table_09dd39785939469795df4097f8b9a6a4 [id=9f1c1f1ed8e548d396a63d984cda85b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.432441 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9ddca2936db4adb92c8ccb34693e24c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.432394 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_8f491acf518347ee9b94a67183cfde02"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.432727 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f491acf518347ee9b94a67183cfde02 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.433233 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8ec408f3ef9f4344824b35f643021751" } modify_external_catalogs: true
I20250115 03:06:20.433326 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8ec408f3ef9f4344824b35f643021751" } modify_external_catalogs: true
I20250115 03:06:20.433426 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_10784ed50a0948d7a64a54cffcda719e" } new_table_name: "default.Table_82a3667e336f44c0a7ae2d9e06270e4b" modify_external_catalogs: true
I20250115 03:06:20.434459 14390 tablet_service.cc:1467] Processing CreateTablet for tablet d9ddca2936db4adb92c8ccb34693e24c (DEFAULT_TABLE table=default.Table_09dd39785939469795df4097f8b9a6a4 [id=9f1c1f1ed8e548d396a63d984cda85b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.435154 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_eef34cd0359d410e849cf181b1d9dcf2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.435300 14626 leader_election.cc:343] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:20.435417 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eef34cd0359d410e849cf181b1d9dcf2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:20.436105 14157 catalog_manager.cc:4670] Async tablet task AlterTable RPC for tablet dda418ed197e4af2aa511d3f1851663e (table default.Table_5c360e43ddae405b9a2843fc20a9dfb9 [id=cbab87c958a346199acdad65ab75be3c], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet dda418ed197e4af2aa511d3f1851663e consensus state has no leader
I20250115 03:06:20.436564 14391 tablet_service.cc:1467] Processing CreateTablet for tablet ed7bdf12b64b4c9e9dc3fb43330a0936 (DEFAULT_TABLE table=default.Table_5072bde4d75d4184a223755f60397e83 [id=66ff7d4342e54548adf8319023ae243b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.436811 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 60fd0318f85842818e0b60d5fef60400 (DEFAULT_TABLE table=default.Table_5e83ef9eef624e33af51363bb4ed5920 [id=b95644ee21df46b480df133f1b1aafc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.437003 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 8d5ea34c28e940c98b4ac8fee80a600b
I20250115 03:06:20.437073 14393 tablet_service.cc:1467] Processing CreateTablet for tablet b1bb007cbce447cc8692ac3647d76403 (DEFAULT_TABLE table=default.Table_cf9c2dc4f74c4c7497a9b49023b79010 [id=6842ac1dc70e4a30a9e875cb0be7507b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.438081 14167 catalog_manager.cc:5526] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:20.438336 14668 tablet_service.cc:1467] Processing CreateTablet for tablet e179d1fd2d614a0e97112256babaf2a3 (DEFAULT_TABLE table=default.Table_93bd5dbdd2344875ba2d8d8a3d0a7591 [id=8dee15cef7b940bc90ad0ec0b37aa2e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.438565 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e179d1fd2d614a0e97112256babaf2a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.438657 14670 tablet_service.cc:1467] Processing CreateTablet for tablet b2d42233d2234f7da262753c3187be43 (DEFAULT_TABLE table=default.Table_05cfde56d7dd49b994fe32031c716aeb [id=a3bfbc632b29496d82f803c1162e5551]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.438892 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2d42233d2234f7da262753c3187be43. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.439311 14823 tablet_replica.cc:331] T 2c69daff62144976ae9082c89cc58eca P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:20.439438 14823 raft_consensus.cc:2238] T 2c69daff62144976ae9082c89cc58eca P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.439621 14823 raft_consensus.cc:2267] T 2c69daff62144976ae9082c89cc58eca P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.440595 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e179d1fd2d614a0e97112256babaf2a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.442730 16292 raft_consensus.cc:491] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.442863 16292 raft_consensus.cc:513] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.443442 16292 leader_election.cc:290] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:20.443499 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56279bb16054869a97006d88574e381" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:20.443548 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ba92f093e14445bebf601200eebd2b59" } new_table_name: "default.Table_c0e04439c1b240b9a233912aeb3e71c1" modify_external_catalogs: true
I20250115 03:06:20.443617 14421 raft_consensus.cc:2463] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:20.443894 14628 leader_election.cc:304] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
W20250115 03:06:20.444643 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet eeb37ffddf484ced806c148940e8f00d consensus state has no leader
I20250115 03:06:20.443712 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2653ad06e063469cbfda9ba1bada2951" } new_table_name: "default.Table_31c0e931dbe844369b67344077333efa" modify_external_catalogs: true
I20250115 03:06:20.446138 14671 tablet_service.cc:1467] Processing CreateTablet for tablet b1bb007cbce447cc8692ac3647d76403 (DEFAULT_TABLE table=default.Table_cf9c2dc4f74c4c7497a9b49023b79010 [id=6842ac1dc70e4a30a9e875cb0be7507b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.446244 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56279bb16054869a97006d88574e381" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:20.446347 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1bb007cbce447cc8692ac3647d76403. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.446537 15153 raft_consensus.cc:2798] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:20.446583 15153 raft_consensus.cc:491] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:20.446616 15153 raft_consensus.cc:3054] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.447381 15153 raft_consensus.cc:513] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.447708 15153 leader_election.cc:290] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:20.447791 15153 raft_consensus.cc:491] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.447845 15153 raft_consensus.cc:513] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.448010 16265 tablet.cc:1721] T 0580a8f7b2984162b9275d74334b1905 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20250115 03:06:20.448123 15153 leader_election.cc:290] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:20.448362 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56279bb16054869a97006d88574e381" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:20.448616 14626 leader_election.cc:343] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.448993 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada45621f8864caab9189039fae0fae2" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:20.449005 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada45621f8864caab9189039fae0fae2" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:20.449097 14419 raft_consensus.cc:2463] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
W20250115 03:06:20.449334 14626 leader_election.cc:343] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.449339 14628 leader_election.cc:304] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:20.449745 14521 tablet_service.cc:1467] Processing CreateTablet for tablet b2d42233d2234f7da262753c3187be43 (DEFAULT_TABLE table=default.Table_05cfde56d7dd49b994fe32031c716aeb [id=a3bfbc632b29496d82f803c1162e5551]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:20.449879 14626 leader_election.cc:343] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.449923 15153 raft_consensus.cc:2798] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:20.449960 15153 raft_consensus.cc:491] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:20.449992 15153 raft_consensus.cc:3054] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.450172 14534 tablet_service.cc:1467] Processing CreateTablet for tablet 978e86e4ed524512b3cc69c262783b4f (DEFAULT_TABLE table=default.Table_d6bfa64233ea4a3d87766891d5b37ca9 [id=06bdc7dd95c9434898209d732a4be56d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.450397 14537 tablet_service.cc:1467] Processing CreateTablet for tablet b1bb007cbce447cc8692ac3647d76403 (DEFAULT_TABLE table=default.Table_cf9c2dc4f74c4c7497a9b49023b79010 [id=6842ac1dc70e4a30a9e875cb0be7507b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.450531 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d5ea34c28e940c98b4ac8fee80a600b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.450762 15153 raft_consensus.cc:513] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.450790 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 60fd0318f85842818e0b60d5fef60400 (DEFAULT_TABLE table=default.Table_5e83ef9eef624e33af51363bb4ed5920 [id=b95644ee21df46b480df133f1b1aafc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.450801 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56279bb16054869a97006d88574e381" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:20.450902 14421 raft_consensus.cc:3054] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.450976 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60fd0318f85842818e0b60d5fef60400. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.451061 15153 leader_election.cc:290] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:20.451717 14421 raft_consensus.cc:2463] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:20.451984 14628 leader_election.cc:304] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:20.452204 15153 raft_consensus.cc:2798] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:20.452333 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada45621f8864caab9189039fae0fae2" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:20.452427 14419 raft_consensus.cc:3054] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.452432 15153 raft_consensus.cc:695] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:20.452558 15153 consensus_queue.cc:237] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.454118 14419 raft_consensus.cc:2463] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:20.454382 14628 leader_election.cc:304] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:20.454484 15970 tablet.cc:1721] T 0580a8f7b2984162b9275d74334b1905 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20250115 03:06:20.454620 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2d42233d2234f7da262753c3187be43. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.455206 15617 tablet.cc:1721] T 0580a8f7b2984162b9275d74334b1905 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 5
I20250115 03:06:20.455483 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ada45621f8864caab9189039fae0fae2" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:20.456255 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1202288f2442477badeb35667dd08dab consensus state has no leader
W20250115 03:06:20.456612 14163 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 820d5af6daa74cadb53d2a5229854a60 consensus state has no leader
W20250115 03:06:20.457087 14626 leader_election.cc:343] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.457496 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 60fd0318f85842818e0b60d5fef60400 (DEFAULT_TABLE table=default.Table_5e83ef9eef624e33af51363bb4ed5920 [id=b95644ee21df46b480df133f1b1aafc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.458030 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60fd0318f85842818e0b60d5fef60400. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.458073 15153 raft_consensus.cc:2798] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:20.458199 15153 raft_consensus.cc:695] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:20.458667 15153 consensus_queue.cc:237] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.459185 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_9c222b05fd6142faa8e3623bd1e54482"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.459508 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c222b05fd6142faa8e3623bd1e54482 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.460080 14391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed7bdf12b64b4c9e9dc3fb43330a0936. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.460219 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2c71ee9d4f4d414997b2c7424816b1a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.460517 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2c71ee9d4f4d414997b2c7424816b1a8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.461426 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9ddca2936db4adb92c8ccb34693e24c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.461656 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46c9ee4fb83d4822a4fd50f65975c6f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.461823 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1bb007cbce447cc8692ac3647d76403. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.461982 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56339d69e99d44bba448522ba41ab50e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.462390 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6e2e63a19a64494884fc71e4b3364978" } new_table_name: "default.Table_ccd2815995fc485d827b79ae818fedae" modify_external_catalogs: true
W20250115 03:06:20.463193 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 8d5ea34c28e940c98b4ac8fee80a600b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d5ea34c28e940c98b4ac8fee80a600b already in progress: creating tablet
I20250115 03:06:20.468052 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60fd0318f85842818e0b60d5fef60400. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.469080 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1bb007cbce447cc8692ac3647d76403. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.470108 15401 raft_consensus.cc:491] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.470183 15401 raft_consensus.cc:513] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.470516 15401 leader_election.cc:290] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:20.470767 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1016cf85491d4ceda132ee57325b48a4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:20.471101 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 978e86e4ed524512b3cc69c262783b4f. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:20.471089 14356 leader_election.cc:343] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.472051 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1016cf85491d4ceda132ee57325b48a4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:20.472795 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 802a007b8d154198b291aaf9530161b9 consensus state has no leader
W20250115 03:06:20.472950 14358 leader_election.cc:343] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.473022 14358 leader_election.cc:304] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:20.473160 15401 raft_consensus.cc:2743] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.473564 14165 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_ef9433c3090f4deea9bf840e58e460e5 [id=0790b08615854d3688f940aba20b6520] alter complete (version 5)
I20250115 03:06:20.476353 14397 tablet_service.cc:1467] Processing CreateTablet for tablet b2d42233d2234f7da262753c3187be43 (DEFAULT_TABLE table=default.Table_05cfde56d7dd49b994fe32031c716aeb [id=a3bfbc632b29496d82f803c1162e5551]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.476579 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2d42233d2234f7da262753c3187be43. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:20.477123 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet f2af9569a0254f32b4b6ceab5bfbfa74 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2af9569a0254f32b4b6ceab5bfbfa74 already in progress: creating tablet
W20250115 03:06:20.477236 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 6004f4adca8e47ddb91f7e39c4cf9089 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6004f4adca8e47ddb91f7e39c4cf9089 already in progress: creating tablet
W20250115 03:06:20.477320 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet c6fa98c6fb304dc49b8e0ced989cc936 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c6fa98c6fb304dc49b8e0ced989cc936 already in progress: creating tablet
W20250115 03:06:20.477532 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet ef7cd35eb1614a29a9197e9c03185f7d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ef7cd35eb1614a29a9197e9c03185f7d already in progress: creating tablet
W20250115 03:06:20.477643 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet b9979224e3874a7982e1bada8395c921 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b9979224e3874a7982e1bada8395c921 already in progress: creating tablet
I20250115 03:06:20.478695 14391 tablet_service.cc:1467] Processing CreateTablet for tablet 978e86e4ed524512b3cc69c262783b4f (DEFAULT_TABLE table=default.Table_d6bfa64233ea4a3d87766891d5b37ca9 [id=06bdc7dd95c9434898209d732a4be56d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.478924 14391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 978e86e4ed524512b3cc69c262783b4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.480104 14163 catalog_manager.cc:5526] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:20.480285 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet b56279bb16054869a97006d88574e381 (table default.Table_23165257844448ce93a97233b487ddc7 [id=080d0a4dfa174d3ba2979bb850d0604a]). Expected version 1 got 0
I20250115 03:06:20.480904 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 978e86e4ed524512b3cc69c262783b4f (DEFAULT_TABLE table=default.Table_d6bfa64233ea4a3d87766891d5b37ca9 [id=06bdc7dd95c9434898209d732a4be56d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.481230 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 978e86e4ed524512b3cc69c262783b4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.481316 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 0687887b68b848d78f19f5674f269fb3
I20250115 03:06:20.481535 14168 catalog_manager.cc:6596] ReplaceTablet: tablet 0687887b68b848d78f19f5674f269fb3 deleted and replaced by tablet 23cd1aa33e2344d28cb33372fcd41768
I20250115 03:06:20.482848 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_87645c60f5cc46fb9149c4fa9bf1a32e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.483139 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_87645c60f5cc46fb9149c4fa9bf1a32e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.483811 14534 tablet_service.cc:1467] Processing CreateTablet for tablet a198e74fd7db4247a673ccb834e7b150 (DEFAULT_TABLE table=default.Table_64d33ab8807d44a996a1f0fcc2deb7e2 [id=62fd48d944c24d15962e1708c7b71b27]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.484033 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a198e74fd7db4247a673ccb834e7b150. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.484409 14823 ts_tablet_manager.cc:1905] T 2c69daff62144976ae9082c89cc58eca P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.486150 14537 tablet_service.cc:1467] Processing CreateTablet for tablet f5480dab59fb4399870a4c9fec977b3e (DEFAULT_TABLE table=default.Table_4b64655665164e00b508a19e84b9991f [id=c206de1bd1fa4e3db3d80167aca79165]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.486363 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5480dab59fb4399870a4c9fec977b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.485267 15970 tablet.cc:1721] T 6bcfde5fe3d4425e98edb4dc54096718 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:20.489182 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 8c385a6ba38745f08232436dbe52375f (DEFAULT_TABLE table=default.Table_e2ad21250b7046048d799b38c19d91ee [id=65e2c73c98464729a70e5df78f94703b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.489392 15617 tablet.cc:1721] T 6bcfde5fe3d4425e98edb4dc54096718 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:20.489506 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c385a6ba38745f08232436dbe52375f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.490995 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 0687887b68b848d78f19f5674f269fb3 with delete_type TABLET_DATA_DELETED (replaced by tablet 23cd1aa33e2344d28cb33372fcd41768) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:20.491389 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 0687887b68b848d78f19f5674f269fb3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0687887b68b848d78f19f5674f269fb3 already in progress: creating tablet
I20250115 03:06:20.492216 14823 ts_tablet_manager.cc:1918] T 2c69daff62144976ae9082c89cc58eca P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.494916 16265 tablet.cc:1721] T dcdb5282e41c43dbb235ae78465d19f9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:20.495635 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 2a9144a303eb4db08840b2f2749c52d6
I20250115 03:06:20.496107 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_45d93b4ddb90448a82b421099d4eb4e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.496415 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_45d93b4ddb90448a82b421099d4eb4e5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.497702 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a9144a303eb4db08840b2f2749c52d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.500388 14823 log.cc:1198] T 2c69daff62144976ae9082c89cc58eca P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/2c69daff62144976ae9082c89cc58eca
I20250115 03:06:20.500665 14823 ts_tablet_manager.cc:1939] T 2c69daff62144976ae9082c89cc58eca P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:20.500665 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f92458688aee4eaf9c942f6778309a2b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.500970 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f92458688aee4eaf9c942f6778309a2b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:20.501678 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 2a9144a303eb4db08840b2f2749c52d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2a9144a303eb4db08840b2f2749c52d6 already in progress: creating tablet
I20250115 03:06:20.505282 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 75b7e033e3ad4e2381ed0c2919e55dd9
I20250115 03:06:20.505738 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet 75b7e033e3ad4e2381ed0c2919e55dd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:20.506647 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 75b7e033e3ad4e2381ed0c2919e55dd9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75b7e033e3ad4e2381ed0c2919e55dd9 already in progress: creating tablet
I20250115 03:06:20.509142 14391 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d5ea34c28e940c98b4ac8fee80a600b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.509958 16292 raft_consensus.cc:491] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.510038 16292 raft_consensus.cc:513] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.510426 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d5ea34c28e940c98b4ac8fee80a600b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:20.511590 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 8d5ea34c28e940c98b4ac8fee80a600b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d5ea34c28e940c98b4ac8fee80a600b already in progress: creating tablet
I20250115 03:06:20.512207 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 8c385a6ba38745f08232436dbe52375f (DEFAULT_TABLE table=default.Table_e2ad21250b7046048d799b38c19d91ee [id=65e2c73c98464729a70e5df78f94703b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.512362 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21524aef80f74a6c899ea66b9b022389" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:20.512444 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c385a6ba38745f08232436dbe52375f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.512485 14422 raft_consensus.cc:2463] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:20.512805 14628 leader_election.cc:304] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:20.512980 15153 raft_consensus.cc:2798] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:20.513032 15153 raft_consensus.cc:491] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:20.513077 15153 raft_consensus.cc:3054] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.513913 15153 raft_consensus.cc:513] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.514052 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 0687887b68b848d78f19f5674f269fb3 with delete_type TABLET_DATA_DELETED (replaced by tablet 23cd1aa33e2344d28cb33372fcd41768) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.514231 16292 leader_election.cc:290] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:20.514302 14390 tablet_service.cc:1467] Processing CreateTablet for tablet a198e74fd7db4247a673ccb834e7b150 (DEFAULT_TABLE table=default.Table_64d33ab8807d44a996a1f0fcc2deb7e2 [id=62fd48d944c24d15962e1708c7b71b27]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.514511 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a198e74fd7db4247a673ccb834e7b150. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.514643 15153 leader_election.cc:290] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:20.514990 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21524aef80f74a6c899ea66b9b022389" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:20.515272 14626 leader_election.cc:343] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.515270 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21524aef80f74a6c899ea66b9b022389" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:20.515365 14422 raft_consensus.cc:3054] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.516271 14422 raft_consensus.cc:2463] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:20.516574 14812 tablet_bootstrap.cc:492] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.513958 14671 tablet_service.cc:1514] Processing DeleteTablet for tablet 0687887b68b848d78f19f5674f269fb3 with delete_type TABLET_DATA_DELETED (replaced by tablet 23cd1aa33e2344d28cb33372fcd41768) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.516666 14812 ts_tablet_manager.cc:1397] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.122s user 0.001s sys 0.000s
I20250115 03:06:20.517077 14391 tablet_service.cc:1467] Processing CreateTablet for tablet 8c385a6ba38745f08232436dbe52375f (DEFAULT_TABLE table=default.Table_e2ad21250b7046048d799b38c19d91ee [id=65e2c73c98464729a70e5df78f94703b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.517194 14812 raft_consensus.cc:357] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.517282 14812 raft_consensus.cc:383] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.517292 14391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c385a6ba38745f08232436dbe52375f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.517311 14812 raft_consensus.cc:738] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.517416 14812 consensus_queue.cc:260] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.517719 14812 ts_tablet_manager.cc:1428] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.517882 14812 tablet_bootstrap.cc:492] T adbb0ae719f24dcc96709a83981e213d P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.518667 14812 tablet_bootstrap.cc:654] T adbb0ae719f24dcc96709a83981e213d P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.519275 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21524aef80f74a6c899ea66b9b022389" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:20.521272 14628 leader_election.cc:304] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:20.521729 16001 raft_consensus.cc:2798] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:20.521811 16001 raft_consensus.cc:695] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:20.521919 16001 consensus_queue.cc:237] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.522782 15401 raft_consensus.cc:491] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.522857 15401 raft_consensus.cc:513] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.523156 15401 leader_election.cc:290] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:20.523463 14626 leader_election.cc:343] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.524654 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b5710268643446b91d04e2f4c7f999e" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:20.525148 14356 leader_election.cc:343] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.525975 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b5710268643446b91d04e2f4c7f999e" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:20.526306 14358 leader_election.cc:343] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.526372 14358 leader_election.cc:304] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:20.526604 15401 raft_consensus.cc:2743] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.527798 14171 catalog_manager.cc:5526] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:20.532120 14812 tablet_bootstrap.cc:492] T adbb0ae719f24dcc96709a83981e213d P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.532194 14812 ts_tablet_manager.cc:1397] T adbb0ae719f24dcc96709a83981e213d P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20250115 03:06:20.532586 14812 raft_consensus.cc:357] T adbb0ae719f24dcc96709a83981e213d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.532673 14812 raft_consensus.cc:383] T adbb0ae719f24dcc96709a83981e213d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.532699 14812 raft_consensus.cc:738] T adbb0ae719f24dcc96709a83981e213d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.532831 14812 consensus_queue.cc:260] T adbb0ae719f24dcc96709a83981e213d P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.533100 14812 ts_tablet_manager.cc:1428] T adbb0ae719f24dcc96709a83981e213d P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.533236 14812 tablet_bootstrap.cc:492] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.533203 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 0e94c152a7b440eb87d13397010711a0 (DEFAULT_TABLE table=default.Table_9c222b05fd6142faa8e3623bd1e54482 [id=d398d066db4d4414a4fd5c9a525a0d30]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.533442 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e94c152a7b440eb87d13397010711a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.533493 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2770f36eb4a14c4d890cba28a5853565"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.533772 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2770f36eb4a14c4d890cba28a5853565 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.534566 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 0ba157d911a145a2a665eed1ec4fc08b (DEFAULT_TABLE table=default.Table_8f491acf518347ee9b94a67183cfde02 [id=e0946f0e55bc4367ba654cac0c43d65b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.534785 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ba157d911a145a2a665eed1ec4fc08b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.534922 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet 2c69daff62144976ae9082c89cc58eca from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:20.535535 14526 tablet_service.cc:1467] Processing CreateTablet for tablet 617e65692e864978894eb3989849c9cb (DEFAULT_TABLE table=default.Table_eef34cd0359d410e849cf181b1d9dcf2 [id=33cad8b9060f42c492d22042dcbb2542]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.535740 14526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 617e65692e864978894eb3989849c9cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.536504 14812 tablet_bootstrap.cc:654] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.536677 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 8ec7bef5519f40419f0ec4bfc7750797 (DEFAULT_TABLE table=default.Table_a6989a57431e4523afea72558cea88c2 [id=346a0df8fc404e31849cb2258288c453]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.536881 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ec7bef5519f40419f0ec4bfc7750797. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.537101 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 16f3790cbec44f15ba1ccf84568c2296 (DEFAULT_TABLE table=default.Table_2c71ee9d4f4d414997b2c7424816b1a8 [id=7c6f7f881b6f42d0ac9a7511f304cb7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.537431 14534 tablet_service.cc:1467] Processing CreateTablet for tablet 23cd1aa33e2344d28cb33372fcd41768 (DEFAULT_TABLE table=default.Table_34987b0302b64565abe50420f8c86a38 [id=07a15fb8d76e43c89f7e05eccc1dccba]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.537793 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 0d500bdccf2748738a85c00dd441319a (DEFAULT_TABLE table=default.Table_49a6ee8663924497be95c6721d2e5887 [id=cb1db5179d1644d7ba7a56002cbb47ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.538086 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d500bdccf2748738a85c00dd441319a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.538241 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16f3790cbec44f15ba1ccf84568c2296. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.540220 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23cd1aa33e2344d28cb33372fcd41768. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.544176 14812 tablet_bootstrap.cc:492] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.544306 14812 ts_tablet_manager.cc:1397] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20250115 03:06:20.544795 14812 raft_consensus.cc:357] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.544958 14812 raft_consensus.cc:383] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.545034 14812 raft_consensus.cc:738] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.545214 14812 consensus_queue.cc:260] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.546343 14812 ts_tablet_manager.cc:1428] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:20.546540 14812 tablet_bootstrap.cc:492] T 5589b497b0e24740ad49d1392f5a737b P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.546908 14171 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_a63d6344914343c4af56eef21f467c6f [id=f45053c781fc460ab55f592eed6fb946] alter complete (version 1)
I20250115 03:06:20.547596 14812 tablet_bootstrap.cc:654] T 5589b497b0e24740ad49d1392f5a737b P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.548543 14812 tablet_bootstrap.cc:492] T 5589b497b0e24740ad49d1392f5a737b P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.548617 14812 ts_tablet_manager.cc:1397] T 5589b497b0e24740ad49d1392f5a737b P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20250115 03:06:20.549044 14812 raft_consensus.cc:357] T 5589b497b0e24740ad49d1392f5a737b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.549139 14812 raft_consensus.cc:383] T 5589b497b0e24740ad49d1392f5a737b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.549181 14812 raft_consensus.cc:738] T 5589b497b0e24740ad49d1392f5a737b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.549290 14812 consensus_queue.cc:260] T 5589b497b0e24740ad49d1392f5a737b P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.549540 14812 ts_tablet_manager.cc:1428] T 5589b497b0e24740ad49d1392f5a737b P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.549695 14812 tablet_bootstrap.cc:492] T bfc34dc76ff94b8eaeb5c2822e6498da P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.550534 14812 tablet_bootstrap.cc:654] T bfc34dc76ff94b8eaeb5c2822e6498da P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.551788 14812 tablet_bootstrap.cc:492] T bfc34dc76ff94b8eaeb5c2822e6498da P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.551867 14812 ts_tablet_manager.cc:1397] T bfc34dc76ff94b8eaeb5c2822e6498da P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:20.552245 14812 raft_consensus.cc:357] T bfc34dc76ff94b8eaeb5c2822e6498da P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.552340 14812 raft_consensus.cc:383] T bfc34dc76ff94b8eaeb5c2822e6498da P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.552420 14812 raft_consensus.cc:738] T bfc34dc76ff94b8eaeb5c2822e6498da P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.552546 14812 consensus_queue.cc:260] T bfc34dc76ff94b8eaeb5c2822e6498da P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.554221 14171 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 9febec144bb343a8a21817e6d3655ad7 (table default.Table_57fbe77148284b4a9666630570f69321 [id=c9ed0b9c92ef40efa5c51cfdd8389325]). Expected version 2 got 0
I20250115 03:06:20.554373 14171 catalog_manager.cc:5526] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:20.555083 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d58c19b62a194b46858ee8955ebf3f82" } modify_external_catalogs: true
I20250115 03:06:20.555186 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d58c19b62a194b46858ee8955ebf3f82" } modify_external_catalogs: true
I20250115 03:06:20.555260 14534 tablet_service.cc:1467] Processing CreateTablet for tablet beb2fd90c41f4841bfa2147bdcd3e017 (DEFAULT_TABLE table=default.Table_f92458688aee4eaf9c942f6778309a2b [id=8aa253a82c15415a81338d171983b575]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.555472 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet beb2fd90c41f4841bfa2147bdcd3e017. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.557468 14534 tablet_service.cc:1467] Processing CreateTablet for tablet c6e78aa3a68345189d746d79dc2e3976 (DEFAULT_TABLE table=default.Table_87645c60f5cc46fb9149c4fa9bf1a32e [id=43ddf3c8ac02432d9fd4bb4775d72789]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.557667 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6e78aa3a68345189d746d79dc2e3976. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.557809 14811 tablet_bootstrap.cc:492] T 15fbf8d5de904fc4b284af664f3bd60f P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.557885 14811 ts_tablet_manager.cc:1397] T 15fbf8d5de904fc4b284af664f3bd60f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.172s user 0.002s sys 0.000s
I20250115 03:06:20.558019 14812 ts_tablet_manager.cc:1428] T bfc34dc76ff94b8eaeb5c2822e6498da P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:20.558174 14812 tablet_bootstrap.cc:492] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.558300 14811 raft_consensus.cc:357] T 15fbf8d5de904fc4b284af664f3bd60f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.558389 14811 raft_consensus.cc:383] T 15fbf8d5de904fc4b284af664f3bd60f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.558419 14811 raft_consensus.cc:738] T 15fbf8d5de904fc4b284af664f3bd60f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.558540 14811 consensus_queue.cc:260] T 15fbf8d5de904fc4b284af664f3bd60f P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.558787 14811 ts_tablet_manager.cc:1428] T 15fbf8d5de904fc4b284af664f3bd60f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.558950 14811 tablet_bootstrap.cc:492] T 6bcfde5fe3d4425e98edb4dc54096718 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.559012 14812 tablet_bootstrap.cc:654] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.559440 14527 tablet_service.cc:1467] Processing CreateTablet for tablet e6e675d648474366897367c1a6bc7579 (DEFAULT_TABLE table=default.Table_45d93b4ddb90448a82b421099d4eb4e5 [id=09231ba470f54d2da96b595ce71a9e73]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.559626 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6e675d648474366897367c1a6bc7579. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.559645 14811 tablet_bootstrap.cc:654] T 6bcfde5fe3d4425e98edb4dc54096718 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.562646 14811 tablet_bootstrap.cc:492] T 6bcfde5fe3d4425e98edb4dc54096718 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.562723 14811 ts_tablet_manager.cc:1397] T 6bcfde5fe3d4425e98edb4dc54096718 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:20.563060 14811 raft_consensus.cc:357] T 6bcfde5fe3d4425e98edb4dc54096718 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.563144 14811 raft_consensus.cc:383] T 6bcfde5fe3d4425e98edb4dc54096718 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.563172 14811 raft_consensus.cc:738] T 6bcfde5fe3d4425e98edb4dc54096718 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.563271 14811 consensus_queue.cc:260] T 6bcfde5fe3d4425e98edb4dc54096718 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.563509 14811 ts_tablet_manager.cc:1428] T 6bcfde5fe3d4425e98edb4dc54096718 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:20.563654 14811 tablet_bootstrap.cc:492] T b56279bb16054869a97006d88574e381 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.564291 14811 tablet_bootstrap.cc:654] T b56279bb16054869a97006d88574e381 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.564663 14812 tablet_bootstrap.cc:492] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.564745 14812 ts_tablet_manager.cc:1397] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:20.564720 14163 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 15fbf8d5de904fc4b284af664f3bd60f (table default.Table_655e9d02aacf4b88ba521f0f57a5f522 [id=6ed01835736b40f8b4caeca7378770d8]). Expected version 1 got 0
I20250115 03:06:20.565203 14812 raft_consensus.cc:357] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.565326 14812 raft_consensus.cc:383] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.565385 14812 raft_consensus.cc:738] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.565518 14812 consensus_queue.cc:260] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.565820 14812 ts_tablet_manager.cc:1428] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.566105 14812 tablet_bootstrap.cc:492] T 0521973e23984355a77a321b4676b479 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.566490 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_91812981cd64473f84541f170fb473bc" } modify_external_catalogs: true
I20250115 03:06:20.566591 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_91812981cd64473f84541f170fb473bc" } modify_external_catalogs: true
I20250115 03:06:20.567111 14812 tablet_bootstrap.cc:654] T 0521973e23984355a77a321b4676b479 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.568843 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet b3334f58b90b4947af52150546ccaa28
I20250115 03:06:20.571612 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet e356bbb02b9649b9b30a3cbadcf59a28
I20250115 03:06:20.576177 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet e356bbb02b9649b9b30a3cbadcf59a28 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.576303 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet b3334f58b90b4947af52150546ccaa28 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:20.576720 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet e356bbb02b9649b9b30a3cbadcf59a28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e356bbb02b9649b9b30a3cbadcf59a28 already in progress: creating tablet
W20250115 03:06:20.576885 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet b3334f58b90b4947af52150546ccaa28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b3334f58b90b4947af52150546ccaa28 already in progress: creating tablet
I20250115 03:06:20.578114 14165 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 6bcfde5fe3d4425e98edb4dc54096718 (table default.Table_a63d6344914343c4af56eef21f467c6f [id=f45053c781fc460ab55f592eed6fb946]). Expected version 1 got 0
I20250115 03:06:20.580773 15153 raft_consensus.cc:491] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.580852 15153 raft_consensus.cc:513] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.581193 15153 leader_election.cc:290] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
W20250115 03:06:20.581719 14628 leader_election.cc:343] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: c7e9a1d1ada941b68688e0ed6742aa99
I20250115 03:06:20.582032 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 62b185f27b5d4002b04dbb9d4ab566ca (DEFAULT_TABLE table=default.Table_2770f36eb4a14c4d890cba28a5853565 [id=60e131ee4dd44f92970b4e64f2a52a9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.582257 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62b185f27b5d4002b04dbb9d4ab566ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.582552 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7e9a1d1ada941b68688e0ed6742aa99" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:20.582849 14626 leader_election.cc:343] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.582919 14626 leader_election.cc:304] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:20.583102 15153 raft_consensus.cc:2743] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.586081 14670 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a9144a303eb4db08840b2f2749c52d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.586493 14653 tablet_service.cc:1467] Processing CreateTablet for tablet a198e74fd7db4247a673ccb834e7b150 (DEFAULT_TABLE table=default.Table_64d33ab8807d44a996a1f0fcc2deb7e2 [id=62fd48d944c24d15962e1708c7b71b27]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.586596 14809 tablet_bootstrap.cc:492] T 80ede4e9696d491aa385b375fcb5cf3b P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.586687 14809 ts_tablet_manager.cc:1397] T 80ede4e9696d491aa385b375fcb5cf3b P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.177s user 0.002s sys 0.000s
I20250115 03:06:20.586725 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a198e74fd7db4247a673ccb834e7b150. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.587141 14809 raft_consensus.cc:357] T 80ede4e9696d491aa385b375fcb5cf3b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.587232 14809 raft_consensus.cc:383] T 80ede4e9696d491aa385b375fcb5cf3b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.587257 14809 raft_consensus.cc:738] T 80ede4e9696d491aa385b375fcb5cf3b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.587358 14809 consensus_queue.cc:260] T 80ede4e9696d491aa385b375fcb5cf3b P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.587633 14809 ts_tablet_manager.cc:1428] T 80ede4e9696d491aa385b375fcb5cf3b P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.587965 14809 tablet_bootstrap.cc:492] T 7b86e66428504217af08f667f660d304 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.588891 14809 tablet_bootstrap.cc:654] T 7b86e66428504217af08f667f660d304 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.591040 15153 consensus_queue.cc:1035] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:20.591149 14811 tablet_bootstrap.cc:492] T b56279bb16054869a97006d88574e381 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.591218 14811 ts_tablet_manager.cc:1397] T b56279bb16054869a97006d88574e381 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.028s user 0.001s sys 0.000s
I20250115 03:06:20.592105 14811 raft_consensus.cc:357] T b56279bb16054869a97006d88574e381 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.592202 14811 raft_consensus.cc:383] T b56279bb16054869a97006d88574e381 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.592233 14811 raft_consensus.cc:738] T b56279bb16054869a97006d88574e381 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.592324 14811 consensus_queue.cc:260] T b56279bb16054869a97006d88574e381 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.593297 14391 tablet_service.cc:1467] Processing CreateTablet for tablet f5480dab59fb4399870a4c9fec977b3e (DEFAULT_TABLE table=default.Table_4b64655665164e00b508a19e84b9991f [id=c206de1bd1fa4e3db3d80167aca79165]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.593533 14391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5480dab59fb4399870a4c9fec977b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.594776 14811 ts_tablet_manager.cc:1428] T b56279bb16054869a97006d88574e381 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:20.594918 14811 tablet_bootstrap.cc:492] T 98913be4c8e74d7185957b0c4ad6abce P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.595465 14811 tablet_bootstrap.cc:654] T 98913be4c8e74d7185957b0c4ad6abce P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.595499 14671 tablet_service.cc:1467] Processing CreateTablet for tablet f5480dab59fb4399870a4c9fec977b3e (DEFAULT_TABLE table=default.Table_4b64655665164e00b508a19e84b9991f [id=c206de1bd1fa4e3db3d80167aca79165]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.595715 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5480dab59fb4399870a4c9fec977b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.596266 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 2c69daff62144976ae9082c89cc58eca (table default.Table_24b2f1f4bbb14d2fa7cba35e089f6707 [id=9c79f5b643664aebbc02a5d815485f61]) successfully deleted
I20250115 03:06:20.597738 14390 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a9144a303eb4db08840b2f2749c52d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:20.597880 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 8d5ea34c28e940c98b4ac8fee80a600b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8d5ea34c28e940c98b4ac8fee80a600b already in progress: creating tablet
I20250115 03:06:20.597920 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 75b7e033e3ad4e2381ed0c2919e55dd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:20.597997 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 0687887b68b848d78f19f5674f269fb3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0687887b68b848d78f19f5674f269fb3 already in progress: creating tablet
I20250115 03:06:20.598021 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 85cceb5a3f234d85937ad0bb75348991 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:20.598203 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 2a9144a303eb4db08840b2f2749c52d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2a9144a303eb4db08840b2f2749c52d6 already in progress: creating tablet
I20250115 03:06:20.598742 14671 tablet_service.cc:1467] Processing CreateTablet for tablet 0ba157d911a145a2a665eed1ec4fc08b (DEFAULT_TABLE table=default.Table_8f491acf518347ee9b94a67183cfde02 [id=e0946f0e55bc4367ba654cac0c43d65b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.598949 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ba157d911a145a2a665eed1ec4fc08b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.600553 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 75b7e033e3ad4e2381ed0c2919e55dd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.602205 14819 tablet_replica.cc:331] T 85cceb5a3f234d85937ad0bb75348991 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:20.602324 14819 raft_consensus.cc:2238] T 85cceb5a3f234d85937ad0bb75348991 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.602397 14819 raft_consensus.cc:2267] T 85cceb5a3f234d85937ad0bb75348991 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.602784 14819 ts_tablet_manager.cc:1905] T 85cceb5a3f234d85937ad0bb75348991 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.604676 14819 ts_tablet_manager.cc:1918] T 85cceb5a3f234d85937ad0bb75348991 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.604729 14819 log.cc:1198] T 85cceb5a3f234d85937ad0bb75348991 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/85cceb5a3f234d85937ad0bb75348991
I20250115 03:06:20.604900 14819 ts_tablet_manager.cc:1939] T 85cceb5a3f234d85937ad0bb75348991 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:20.605173 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet 4102586f32d94a998cf2807434ee0ebf from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:20.605278 14391 tablet_service.cc:1467] Processing CreateTablet for tablet 0ba157d911a145a2a665eed1ec4fc08b (DEFAULT_TABLE table=default.Table_8f491acf518347ee9b94a67183cfde02 [id=e0946f0e55bc4367ba654cac0c43d65b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.605490 14391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ba157d911a145a2a665eed1ec4fc08b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.606439 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 8ec7bef5519f40419f0ec4bfc7750797 (DEFAULT_TABLE table=default.Table_a6989a57431e4523afea72558cea88c2 [id=346a0df8fc404e31849cb2258288c453]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.606662 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ec7bef5519f40419f0ec4bfc7750797. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.607228 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 16f3790cbec44f15ba1ccf84568c2296 (DEFAULT_TABLE table=default.Table_2c71ee9d4f4d414997b2c7424816b1a8 [id=7c6f7f881b6f42d0ac9a7511f304cb7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.607436 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16f3790cbec44f15ba1ccf84568c2296. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.607646 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 23cd1aa33e2344d28cb33372fcd41768 (DEFAULT_TABLE table=default.Table_34987b0302b64565abe50420f8c86a38 [id=07a15fb8d76e43c89f7e05eccc1dccba]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.607867 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23cd1aa33e2344d28cb33372fcd41768. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.609097 14811 tablet_bootstrap.cc:492] T 98913be4c8e74d7185957b0c4ad6abce P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.609169 14811 ts_tablet_manager.cc:1397] T 98913be4c8e74d7185957b0c4ad6abce P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20250115 03:06:20.609467 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 8ec7bef5519f40419f0ec4bfc7750797 (DEFAULT_TABLE table=default.Table_a6989a57431e4523afea72558cea88c2 [id=346a0df8fc404e31849cb2258288c453]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.609663 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ec7bef5519f40419f0ec4bfc7750797. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.611181 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 15fbf8d5de904fc4b284af664f3bd60f (table default.Table_655e9d02aacf4b88ba521f0f57a5f522 [id=6ed01835736b40f8b4caeca7378770d8]). Expected version 1 got 0
I20250115 03:06:20.611641 14390 tablet_service.cc:1467] Processing CreateTablet for tablet 23cd1aa33e2344d28cb33372fcd41768 (DEFAULT_TABLE table=default.Table_34987b0302b64565abe50420f8c86a38 [id=07a15fb8d76e43c89f7e05eccc1dccba]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.611657 14811 raft_consensus.cc:357] T 98913be4c8e74d7185957b0c4ad6abce P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.611753 14811 raft_consensus.cc:383] T 98913be4c8e74d7185957b0c4ad6abce P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.611781 14811 raft_consensus.cc:738] T 98913be4c8e74d7185957b0c4ad6abce P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.611856 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23cd1aa33e2344d28cb33372fcd41768. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.612075 14811 consensus_queue.cc:260] T 98913be4c8e74d7185957b0c4ad6abce P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.614493 14811 ts_tablet_manager.cc:1428] T 98913be4c8e74d7185957b0c4ad6abce P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:20.614967 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 0e94c152a7b440eb87d13397010711a0 (DEFAULT_TABLE table=default.Table_9c222b05fd6142faa8e3623bd1e54482 [id=d398d066db4d4414a4fd5c9a525a0d30]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.615161 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e94c152a7b440eb87d13397010711a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.615537 14390 tablet_service.cc:1467] Processing CreateTablet for tablet 0e94c152a7b440eb87d13397010711a0 (DEFAULT_TABLE table=default.Table_9c222b05fd6142faa8e3623bd1e54482 [id=d398d066db4d4414a4fd5c9a525a0d30]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.615761 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e94c152a7b440eb87d13397010711a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.616546 14396 tablet_service.cc:1467] Processing CreateTablet for tablet beb2fd90c41f4841bfa2147bdcd3e017 (DEFAULT_TABLE table=default.Table_f92458688aee4eaf9c942f6778309a2b [id=8aa253a82c15415a81338d171983b575]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.616710 14811 tablet_bootstrap.cc:492] T fbd161e290914d9d8671c10becfeaee1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.616744 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet beb2fd90c41f4841bfa2147bdcd3e017. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.616756 14657 tablet_service.cc:1467] Processing CreateTablet for tablet c6e78aa3a68345189d746d79dc2e3976 (DEFAULT_TABLE table=default.Table_87645c60f5cc46fb9149c4fa9bf1a32e [id=43ddf3c8ac02432d9fd4bb4775d72789]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.616966 14657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6e78aa3a68345189d746d79dc2e3976. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.617647 14811 tablet_bootstrap.cc:654] T fbd161e290914d9d8671c10becfeaee1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.618450 14811 tablet_bootstrap.cc:492] T fbd161e290914d9d8671c10becfeaee1 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.618521 14811 ts_tablet_manager.cc:1397] T fbd161e290914d9d8671c10becfeaee1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:20.618697 14668 tablet_service.cc:1467] Processing CreateTablet for tablet 0d500bdccf2748738a85c00dd441319a (DEFAULT_TABLE table=default.Table_49a6ee8663924497be95c6721d2e5887 [id=cb1db5179d1644d7ba7a56002cbb47ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.618912 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d500bdccf2748738a85c00dd441319a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.618865 14811 raft_consensus.cc:357] T fbd161e290914d9d8671c10becfeaee1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.618944 14811 raft_consensus.cc:383] T fbd161e290914d9d8671c10becfeaee1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.618969 14811 raft_consensus.cc:738] T fbd161e290914d9d8671c10becfeaee1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.619061 14811 consensus_queue.cc:260] T fbd161e290914d9d8671c10becfeaee1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.619151 14809 tablet_bootstrap.cc:492] T 7b86e66428504217af08f667f660d304 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.619220 14809 ts_tablet_manager.cc:1397] T 7b86e66428504217af08f667f660d304 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.031s user 0.001s sys 0.000s
I20250115 03:06:20.619272 14811 ts_tablet_manager.cc:1428] T fbd161e290914d9d8671c10becfeaee1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.619402 14811 tablet_bootstrap.cc:492] T d2564a20857f45d9a0c7e3b74eb58a91 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.619602 14809 raft_consensus.cc:357] T 7b86e66428504217af08f667f660d304 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.619690 14809 raft_consensus.cc:383] T 7b86e66428504217af08f667f660d304 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.619720 14809 raft_consensus.cc:738] T 7b86e66428504217af08f667f660d304 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.619840 14809 consensus_queue.cc:260] T 7b86e66428504217af08f667f660d304 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.620091 14809 ts_tablet_manager.cc:1428] T 7b86e66428504217af08f667f660d304 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.620199 14811 tablet_bootstrap.cc:654] T d2564a20857f45d9a0c7e3b74eb58a91 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.620240 14809 tablet_bootstrap.cc:492] T 0f20bb66f36647e0811d0865b8f414f7 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.620790 14656 tablet_service.cc:1467] Processing CreateTablet for tablet beb2fd90c41f4841bfa2147bdcd3e017 (DEFAULT_TABLE table=default.Table_f92458688aee4eaf9c942f6778309a2b [id=8aa253a82c15415a81338d171983b575]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.621008 14656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet beb2fd90c41f4841bfa2147bdcd3e017. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.621100 14809 tablet_bootstrap.cc:654] T 0f20bb66f36647e0811d0865b8f414f7 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.622490 16001 raft_consensus.cc:491] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.622570 16001 raft_consensus.cc:513] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.622875 16001 leader_election.cc:290] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:20.623147 14811 tablet_bootstrap.cc:492] T d2564a20857f45d9a0c7e3b74eb58a91 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.623209 14811 ts_tablet_manager.cc:1397] T d2564a20857f45d9a0c7e3b74eb58a91 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:20.623703 14811 raft_consensus.cc:357] T d2564a20857f45d9a0c7e3b74eb58a91 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.623781 14811 raft_consensus.cc:383] T d2564a20857f45d9a0c7e3b74eb58a91 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.623807 14811 raft_consensus.cc:738] T d2564a20857f45d9a0c7e3b74eb58a91 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.623903 14811 consensus_queue.cc:260] T d2564a20857f45d9a0c7e3b74eb58a91 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.624115 14811 ts_tablet_manager.cc:1428] T d2564a20857f45d9a0c7e3b74eb58a91 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.624245 14811 tablet_bootstrap.cc:492] T 21524aef80f74a6c899ea66b9b022389 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.624498 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 617e65692e864978894eb3989849c9cb (DEFAULT_TABLE table=default.Table_eef34cd0359d410e849cf181b1d9dcf2 [id=33cad8b9060f42c492d22042dcbb2542]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.624692 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 617e65692e864978894eb3989849c9cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.625074 14811 tablet_bootstrap.cc:654] T 21524aef80f74a6c899ea66b9b022389 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.625567 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b4eb3b20bb4908884e5716e7b2a369" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:20.625675 14422 raft_consensus.cc:2463] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:20.625761 16001 raft_consensus.cc:491] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.625831 16001 raft_consensus.cc:513] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.626138 16001 leader_election.cc:290] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:20.626644 14628 leader_election.cc:304] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:20.626660 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 0d500bdccf2748738a85c00dd441319a (DEFAULT_TABLE table=default.Table_49a6ee8663924497be95c6721d2e5887 [id=cb1db5179d1644d7ba7a56002cbb47ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.626798 16001 raft_consensus.cc:2798] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:20.626840 16001 raft_consensus.cc:491] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:20.626864 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d500bdccf2748738a85c00dd441319a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.626873 16001 raft_consensus.cc:3054] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.627650 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9febec144bb343a8a21817e6d3655ad7" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:20.627743 14422 raft_consensus.cc:2463] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:20.627756 16001 raft_consensus.cc:513] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.628051 16001 leader_election.cc:290] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:20.628645 14811 tablet_bootstrap.cc:492] T 21524aef80f74a6c899ea66b9b022389 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.628670 14628 leader_election.cc:304] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:20.628719 14811 ts_tablet_manager.cc:1397] T 21524aef80f74a6c899ea66b9b022389 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20250115 03:06:20.628815 16001 raft_consensus.cc:2798] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:20.628882 16001 raft_consensus.cc:491] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:20.628917 16001 raft_consensus.cc:3054] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.629096 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b4eb3b20bb4908884e5716e7b2a369" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:20.629184 14422 raft_consensus.cc:3054] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.629699 16001 raft_consensus.cc:513] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.629914 14671 tablet_service.cc:1467] Processing CreateTablet for tablet 16f3790cbec44f15ba1ccf84568c2296 (DEFAULT_TABLE table=default.Table_2c71ee9d4f4d414997b2c7424816b1a8 [id=7c6f7f881b6f42d0ac9a7511f304cb7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.629977 14422 raft_consensus.cc:2463] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:20.630115 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16f3790cbec44f15ba1ccf84568c2296. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.630338 14811 raft_consensus.cc:357] T 21524aef80f74a6c899ea66b9b022389 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.630430 14811 raft_consensus.cc:383] T 21524aef80f74a6c899ea66b9b022389 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.630456 14811 raft_consensus.cc:738] T 21524aef80f74a6c899ea66b9b022389 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.630520 14628 leader_election.cc:304] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:20.630558 14811 consensus_queue.cc:260] T 21524aef80f74a6c899ea66b9b022389 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.630659 16292 raft_consensus.cc:2798] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:20.630724 16292 raft_consensus.cc:695] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:20.630777 14811 ts_tablet_manager.cc:1428] T 21524aef80f74a6c899ea66b9b022389 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:20.630841 16292 consensus_queue.cc:237] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.630908 14811 tablet_bootstrap.cc:492] T 4102586f32d94a998cf2807434ee0ebf P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.631518 16001 leader_election.cc:290] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:20.631690 14811 tablet_bootstrap.cc:654] T 4102586f32d94a998cf2807434ee0ebf P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.631963 14390 tablet_service.cc:1467] Processing CreateTablet for tablet c6e78aa3a68345189d746d79dc2e3976 (DEFAULT_TABLE table=default.Table_87645c60f5cc46fb9149c4fa9bf1a32e [id=43ddf3c8ac02432d9fd4bb4775d72789]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.632161 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6e78aa3a68345189d746d79dc2e3976. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.632678 14657 tablet_service.cc:1514] Processing DeleteTablet for tablet adbb0ae719f24dcc96709a83981e213d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.632963 14656 tablet_service.cc:1467] Processing CreateTablet for tablet e6e675d648474366897367c1a6bc7579 (DEFAULT_TABLE table=default.Table_45d93b4ddb90448a82b421099d4eb4e5 [id=09231ba470f54d2da96b595ce71a9e73]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.633106 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet c7e9a1d1ada941b68688e0ed6742aa99 with delete_type TABLET_DATA_DELETED (replaced by tablet 238e68300f304266aaed94bc3b76a01d) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.633230 14657 tablet_service.cc:1514] Processing DeleteTablet for tablet b3334f58b90b4947af52150546ccaa28 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.634315 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet b3334f58b90b4947af52150546ccaa28 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.634363 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9febec144bb343a8a21817e6d3655ad7" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:20.634479 14422 raft_consensus.cc:3054] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.634582 14657 tablet_service.cc:1514] Processing DeleteTablet for tablet e356bbb02b9649b9b30a3cbadcf59a28 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.634882 14811 tablet_bootstrap.cc:492] T 4102586f32d94a998cf2807434ee0ebf P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.634944 14811 ts_tablet_manager.cc:1397] T 4102586f32d94a998cf2807434ee0ebf P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:20.635285 14422 raft_consensus.cc:2463] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:20.635278 14811 raft_consensus.cc:357] T 4102586f32d94a998cf2807434ee0ebf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.635356 14811 raft_consensus.cc:383] T 4102586f32d94a998cf2807434ee0ebf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.635383 14811 raft_consensus.cc:738] T 4102586f32d94a998cf2807434ee0ebf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.635484 14811 consensus_queue.cc:260] T 4102586f32d94a998cf2807434ee0ebf P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.635680 14391 tablet_service.cc:1467] Processing CreateTablet for tablet e6e675d648474366897367c1a6bc7579 (DEFAULT_TABLE table=default.Table_45d93b4ddb90448a82b421099d4eb4e5 [id=09231ba470f54d2da96b595ce71a9e73]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.635725 14811 ts_tablet_manager.cc:1428] T 4102586f32d94a998cf2807434ee0ebf P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.635862 14811 tablet_bootstrap.cc:492] T c7e9a1d1ada941b68688e0ed6742aa99 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.635874 14391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6e675d648474366897367c1a6bc7579. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.636642 14811 tablet_bootstrap.cc:654] T c7e9a1d1ada941b68688e0ed6742aa99 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.637303 14657 tablet_service.cc:1514] Processing DeleteTablet for tablet bfc34dc76ff94b8eaeb5c2822e6498da with delete_type TABLET_DATA_DELETED (replaced by tablet f2ca06c402cd46dc8311c8a47da92846) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.637610 14628 leader_election.cc:304] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:20.637642 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 617e65692e864978894eb3989849c9cb (DEFAULT_TABLE table=default.Table_eef34cd0359d410e849cf181b1d9dcf2 [id=33cad8b9060f42c492d22042dcbb2542]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.637755 16001 raft_consensus.cc:2798] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:20.637869 16001 raft_consensus.cc:695] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:20.637972 16001 consensus_queue.cc:237] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.640236 14390 tablet_service.cc:1514] Processing DeleteTablet for tablet e356bbb02b9649b9b30a3cbadcf59a28 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.641028 14553 raft_consensus.cc:3054] T d5655935aabd48a19d0ef03739e089a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.642642 16265 tablet.cc:1721] T d5655935aabd48a19d0ef03739e089a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:20.643976 14165 catalog_manager.cc:5526] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:20.644490 14163 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet b56279bb16054869a97006d88574e381 (table default.Table_23165257844448ce93a97233b487ddc7 [id=080d0a4dfa174d3ba2979bb850d0604a]). Expected version 1 got 0
I20250115 03:06:20.644896 14657 tablet_service.cc:1514] Processing DeleteTablet for tablet 5589b497b0e24740ad49d1392f5a737b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.647048 14656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6e675d648474366897367c1a6bc7579. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.649119 14823 tablet_replica.cc:331] T adbb0ae719f24dcc96709a83981e213d P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:20.649247 14823 raft_consensus.cc:2238] T adbb0ae719f24dcc96709a83981e213d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.649344 14823 raft_consensus.cc:2267] T adbb0ae719f24dcc96709a83981e213d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.649598 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_59631f20f5e241509755c4b0e207f4dc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.649937 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_59631f20f5e241509755c4b0e207f4dc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.649989 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_736f17451e344c1e9b3c1e0797cb2a6d" } new_table_name: "default.Table_691c78b3563f49c9a7957471bc430389" modify_external_catalogs: true
I20250115 03:06:20.650346 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b4eb3b20bb4908884e5716e7b2a369" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:20.650589 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b4eb3b20bb4908884e5716e7b2a369" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:20.650650 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9febec144bb343a8a21817e6d3655ad7" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:20.650748 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a1547d1d923f4f4da05aaeb519f4cdf0" } new_table_name: "default.Table_802f667f1ad246688cbdcffcb3001021" modify_external_catalogs: true
W20250115 03:06:20.650972 14626 leader_election.cc:343] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:20.651126 14626 leader_election.cc:343] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:20.651238 14626 leader_election.cc:343] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.651453 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_54eb2aa18ec047739357df4000c84570"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.651753 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_54eb2aa18ec047739357df4000c84570 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.653642 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_bca65e8db1d94382a3528cf21a111c64" } modify_external_catalogs: true
I20250115 03:06:20.653731 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_bca65e8db1d94382a3528cf21a111c64" } modify_external_catalogs: true
I20250115 03:06:20.654177 14169 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet b56279bb16054869a97006d88574e381 (table default.Table_23165257844448ce93a97233b487ddc7 [id=080d0a4dfa174d3ba2979bb850d0604a]). Expected version 1 got 0
I20250115 03:06:20.655000 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_320deab8a90b44af8922fcc44f378bbf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:20.650043 14167 master_service.cc:946] ReplaceTablet: received request to replace tablet 4535ac88f78f48da83d9d2f951afdfa6 from {username='slave'} at 127.0.0.1:60432
W20250115 03:06:20.655259 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_320deab8a90b44af8922fcc44f378bbf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.655766 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9febec144bb343a8a21817e6d3655ad7" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:20.656069 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet d2564a20857f45d9a0c7e3b74eb58a91 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.656201 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet 4102586f32d94a998cf2807434ee0ebf with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:20.656229 14626 leader_election.cc:343] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.656291 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 98913be4c8e74d7185957b0c4ad6abce with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.656387 14820 tablet_replica.cc:331] T d2564a20857f45d9a0c7e3b74eb58a91 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.656488 14820 raft_consensus.cc:2238] T d2564a20857f45d9a0c7e3b74eb58a91 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.656569 14820 raft_consensus.cc:2267] T d2564a20857f45d9a0c7e3b74eb58a91 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.657055 14820 ts_tablet_manager.cc:1905] T d2564a20857f45d9a0c7e3b74eb58a91 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.657117 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 617e65692e864978894eb3989849c9cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.657182 14823 ts_tablet_manager.cc:1905] T adbb0ae719f24dcc96709a83981e213d P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.659198 14823 ts_tablet_manager.cc:1918] T adbb0ae719f24dcc96709a83981e213d P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.659198 14820 ts_tablet_manager.cc:1918] T d2564a20857f45d9a0c7e3b74eb58a91 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.659257 14823 log.cc:1198] T adbb0ae719f24dcc96709a83981e213d P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/adbb0ae719f24dcc96709a83981e213d
I20250115 03:06:20.659261 14820 log.cc:1198] T d2564a20857f45d9a0c7e3b74eb58a91 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/d2564a20857f45d9a0c7e3b74eb58a91
I20250115 03:06:20.659456 14820 ts_tablet_manager.cc:1939] T d2564a20857f45d9a0c7e3b74eb58a91 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.659458 14823 ts_tablet_manager.cc:1939] T adbb0ae719f24dcc96709a83981e213d P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:20.660444 14823 tablet_replica.cc:331] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:20.660581 14823 raft_consensus.cc:2238] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.660669 14823 raft_consensus.cc:2267] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.660701 14165 catalog_manager.cc:5526] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:20.660897 14165 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 9febec144bb343a8a21817e6d3655ad7 (table default.Table_57fbe77148284b4a9666630570f69321 [id=c9ed0b9c92ef40efa5c51cfdd8389325]). Expected version 2 got 0
I20250115 03:06:20.660977 14820 tablet_replica.cc:331] T 4102586f32d94a998cf2807434ee0ebf P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.661687 14820 raft_consensus.cc:2238] T 4102586f32d94a998cf2807434ee0ebf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.661787 14820 raft_consensus.cc:2267] T 4102586f32d94a998cf2807434ee0ebf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:20.661942 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f2ca06c402cd46dc8311c8a47da92846 consensus state has no leader
I20250115 03:06:20.662196 14823 ts_tablet_manager.cc:1905] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:20.662324 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 75854bea6dec40ac9f205fec0ae66f39 consensus state has no leader
I20250115 03:06:20.663885 14823 ts_tablet_manager.cc:1918] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.663997 14823 log.cc:1198] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/c7e9a1d1ada941b68688e0ed6742aa99
I20250115 03:06:20.664233 14823 ts_tablet_manager.cc:1939] T c7e9a1d1ada941b68688e0ed6742aa99 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:20.664594 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 4535ac88f78f48da83d9d2f951afdfa6
I20250115 03:06:20.664860 14167 catalog_manager.cc:6596] ReplaceTablet: tablet 4535ac88f78f48da83d9d2f951afdfa6 deleted and replaced by tablet 3a3964469dbd460c8416828043f862c4
I20250115 03:06:20.665093 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 839bdd3580024e849e9e1439ade8d3d9
I20250115 03:06:20.665232 14823 tablet_replica.cc:331] T bfc34dc76ff94b8eaeb5c2822e6498da P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:20.665354 14823 raft_consensus.cc:2238] T bfc34dc76ff94b8eaeb5c2822e6498da P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.665496 14823 raft_consensus.cc:2267] T bfc34dc76ff94b8eaeb5c2822e6498da P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.665993 14823 ts_tablet_manager.cc:1905] T bfc34dc76ff94b8eaeb5c2822e6498da P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.666005 14820 ts_tablet_manager.cc:1905] T 4102586f32d94a998cf2807434ee0ebf P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.667940 14820 ts_tablet_manager.cc:1918] T 4102586f32d94a998cf2807434ee0ebf P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.668001 14820 log.cc:1198] T 4102586f32d94a998cf2807434ee0ebf P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/4102586f32d94a998cf2807434ee0ebf
I20250115 03:06:20.668210 14820 ts_tablet_manager.cc:1939] T 4102586f32d94a998cf2807434ee0ebf P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.668494 14820 tablet_replica.cc:331] T 98913be4c8e74d7185957b0c4ad6abce P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.668608 14820 raft_consensus.cc:2238] T 98913be4c8e74d7185957b0c4ad6abce P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.668690 14820 raft_consensus.cc:2267] T 98913be4c8e74d7185957b0c4ad6abce P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.669045 16189 raft_consensus.cc:491] T 21524aef80f74a6c899ea66b9b022389 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.669126 16189 raft_consensus.cc:513] T 21524aef80f74a6c899ea66b9b022389 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.669476 16189 leader_election.cc:290] T 21524aef80f74a6c899ea66b9b022389 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:20.669790 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21524aef80f74a6c899ea66b9b022389" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:20.670187 14820 ts_tablet_manager.cc:1905] T 98913be4c8e74d7185957b0c4ad6abce P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.670337 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21524aef80f74a6c899ea66b9b022389" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:20.670480 14422 raft_consensus.cc:2388] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in current term 1: Already voted for candidate 5c0dae7aff4b44c699275b95ba639770 in this term.
I20250115 03:06:20.670837 14493 leader_election.cc:304] T 21524aef80f74a6c899ea66b9b022389 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:20.670997 16189 raft_consensus.cc:3054] T 21524aef80f74a6c899ea66b9b022389 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.671797 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet d2564a20857f45d9a0c7e3b74eb58a91 (table default.Table_7c8f638a9380432c844dc355d43df492 [id=6a9388b294f24f358a5d5f34568e8a13]) successfully deleted
I20250115 03:06:20.671902 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 4102586f32d94a998cf2807434ee0ebf (table default.Table_a1c9e28bac9c491e948bc3fc9b173bad [id=2fb8c68c76304deebd009b01eb078616]) successfully deleted
I20250115 03:06:20.671896 16189 raft_consensus.cc:2743] T 21524aef80f74a6c899ea66b9b022389 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.672142 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_63bec1dd11364c7fa3f990397751b825"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.672408 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_63bec1dd11364c7fa3f990397751b825 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.673671 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 4535ac88f78f48da83d9d2f951afdfa6 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a3964469dbd460c8416828043f862c4) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.673813 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet 839bdd3580024e849e9e1439ade8d3d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.674149 14823 ts_tablet_manager.cc:1918] T bfc34dc76ff94b8eaeb5c2822e6498da P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.674208 14823 log.cc:1198] T bfc34dc76ff94b8eaeb5c2822e6498da P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/bfc34dc76ff94b8eaeb5c2822e6498da
I20250115 03:06:20.674389 14823 ts_tablet_manager.cc:1939] T bfc34dc76ff94b8eaeb5c2822e6498da P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:20.674896 14823 tablet_replica.cc:331] T 5589b497b0e24740ad49d1392f5a737b P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:20.675005 14823 raft_consensus.cc:2238] T 5589b497b0e24740ad49d1392f5a737b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.675087 14823 raft_consensus.cc:2267] T 5589b497b0e24740ad49d1392f5a737b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.677033 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ab60aad92f7f48fb8519f90f108f57ad" } new_table_name: "default.Table_e7b6ace2c7c04dbfa9e87c6a4697a243" modify_external_catalogs: true
I20250115 03:06:20.677114 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ba948e2cc56b4023a05595735e5a1434" } new_table_name: "default.Table_c906e8e84e6a4543aa7ed60a65e7a042" modify_external_catalogs: true
I20250115 03:06:20.677166 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_46a85c9731254247bbef481a29502e48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.677456 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_46a85c9731254247bbef481a29502e48 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.678108 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_40f50e0fd48242a59523563318d43e7a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.678386 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_40f50e0fd48242a59523563318d43e7a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.678992 14171 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_c920b2b2aef445488633aa45f8868aa8" } new_table_name: "default.Table_bfb8ef470a8a43e399bf8541e2171dc4" modify_external_catalogs: true
I20250115 03:06:20.680761 14823 ts_tablet_manager.cc:1905] T 5589b497b0e24740ad49d1392f5a737b P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.682020 14820 ts_tablet_manager.cc:1918] T 98913be4c8e74d7185957b0c4ad6abce P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.682096 14820 log.cc:1198] T 98913be4c8e74d7185957b0c4ad6abce P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/98913be4c8e74d7185957b0c4ad6abce
I20250115 03:06:20.682297 14820 ts_tablet_manager.cc:1939] T 98913be4c8e74d7185957b0c4ad6abce P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.682910 14823 ts_tablet_manager.cc:1918] T 5589b497b0e24740ad49d1392f5a737b P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.683027 14823 log.cc:1198] T 5589b497b0e24740ad49d1392f5a737b P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/5589b497b0e24740ad49d1392f5a737b
I20250115 03:06:20.683265 14823 ts_tablet_manager.cc:1939] T 5589b497b0e24740ad49d1392f5a737b P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:20.684340 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 98913be4c8e74d7185957b0c4ad6abce (table default.Table_71fadacb26bd4e4a8ddcd48e403350bb [id=529f6a8366ca4afd8e51cc396a8bb7a1]) successfully deleted
W20250115 03:06:20.685194 14165 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ec0f3424ebcb442597a94ccdbb25eeb6 consensus state has no leader
I20250115 03:06:20.685463 14811 tablet_bootstrap.cc:492] T c7e9a1d1ada941b68688e0ed6742aa99 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.685557 14811 ts_tablet_manager.cc:1397] T c7e9a1d1ada941b68688e0ed6742aa99 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.050s user 0.001s sys 0.000s
I20250115 03:06:20.686000 14811 raft_consensus.cc:357] T c7e9a1d1ada941b68688e0ed6742aa99 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.686105 14811 raft_consensus.cc:383] T c7e9a1d1ada941b68688e0ed6742aa99 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.686138 14811 raft_consensus.cc:738] T c7e9a1d1ada941b68688e0ed6742aa99 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.686250 14811 consensus_queue.cc:260] T c7e9a1d1ada941b68688e0ed6742aa99 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.686522 14811 ts_tablet_manager.cc:1428] T c7e9a1d1ada941b68688e0ed6742aa99 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.686810 14811 tablet_bootstrap.cc:492] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.687646 14811 tablet_bootstrap.cc:654] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:20.688783 14171 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet af712cc03750455087ecd381cc97d464 consensus state has no leader
W20250115 03:06:20.690410 14162 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 836541bcc82d4644b4c5eb2f3f25c030 consensus state has no leader
I20250115 03:06:20.693301 14390 tablet_service.cc:1467] Processing CreateTablet for tablet 62b185f27b5d4002b04dbb9d4ab566ca (DEFAULT_TABLE table=default.Table_2770f36eb4a14c4d890cba28a5853565 [id=60e131ee4dd44f92970b4e64f2a52a9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.693564 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62b185f27b5d4002b04dbb9d4ab566ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.695010 14167 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 15fbf8d5de904fc4b284af664f3bd60f (table default.Table_655e9d02aacf4b88ba521f0f57a5f522 [id=6ed01835736b40f8b4caeca7378770d8]). Expected version 1 got 0
I20250115 03:06:20.697294 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet c7e9a1d1ada941b68688e0ed6742aa99 with delete_type TABLET_DATA_DELETED (replaced by tablet 238e68300f304266aaed94bc3b76a01d) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.697713 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_59c02322b76b4ad18a2c6b0ac7438110" } modify_external_catalogs: true
I20250115 03:06:20.697817 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_59c02322b76b4ad18a2c6b0ac7438110" } modify_external_catalogs: true
W20250115 03:06:20.698176 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 0687887b68b848d78f19f5674f269fb3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0687887b68b848d78f19f5674f269fb3 already in progress: creating tablet
I20250115 03:06:20.702543 15401 raft_consensus.cc:491] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.702626 15401 raft_consensus.cc:513] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.702966 15401 leader_election.cc:290] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:20.703259 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676ec8b9dc8241d4ae40488d3ce8f61e" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:20.703676 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676ec8b9dc8241d4ae40488d3ce8f61e" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:20.704077 14356 leader_election.cc:343] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:20.704357 14358 leader_election.cc:343] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.704418 14358 leader_election.cc:304] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:20.704748 15401 raft_consensus.cc:2743] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.706758 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 62b185f27b5d4002b04dbb9d4ab566ca (DEFAULT_TABLE table=default.Table_2770f36eb4a14c4d890cba28a5853565 [id=60e131ee4dd44f92970b4e64f2a52a9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.707088 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62b185f27b5d4002b04dbb9d4ab566ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.708009 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2770f36eb4a14c4d890cba28a5853565" } modify_external_catalogs: true
I20250115 03:06:20.708031 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 21408c08158a4cd8b77bbff89e7839e7 (DEFAULT_TABLE table=default.Table_59631f20f5e241509755c4b0e207f4dc [id=5306fcebb37542b7a5dd17b1b44b7e52]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.708142 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2770f36eb4a14c4d890cba28a5853565" } modify_external_catalogs: true
I20250115 03:06:20.708294 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21408c08158a4cd8b77bbff89e7839e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.710233 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 11d577daf54c4a9d8f9667f178b7af1e (DEFAULT_TABLE table=default.Table_54eb2aa18ec047739357df4000c84570 [id=e48cd80f25dc4150ac7a8aa5b8867363]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.710426 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11d577daf54c4a9d8f9667f178b7af1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.712473 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b0334b90634c4543b8ced12e02fa7114"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:20.712642 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e2b439c7d5064b22a1e7d2033f74217b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.712756 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b0334b90634c4543b8ced12e02fa7114 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:20.712966 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e2b439c7d5064b22a1e7d2033f74217b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.714186 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_bd02870cfaf64d6187493faecb6ff806"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.714455 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd02870cfaf64d6187493faecb6ff806 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:20.715955 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 4535ac88f78f48da83d9d2f951afdfa6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4535ac88f78f48da83d9d2f951afdfa6 already in progress: creating tablet
W20250115 03:06:20.716115 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 839bdd3580024e849e9e1439ade8d3d9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 839bdd3580024e849e9e1439ade8d3d9 already in progress: creating tablet
I20250115 03:06:20.716161 14820 tablet_replica.cc:331] T c7e9a1d1ada941b68688e0ed6742aa99 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.716324 14820 raft_consensus.cc:2238] T c7e9a1d1ada941b68688e0ed6742aa99 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.716434 14820 raft_consensus.cc:2267] T c7e9a1d1ada941b68688e0ed6742aa99 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:20.716755 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 2a9144a303eb4db08840b2f2749c52d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2a9144a303eb4db08840b2f2749c52d6 already in progress: creating tablet
W20250115 03:06:20.716878 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 75b7e033e3ad4e2381ed0c2919e55dd9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75b7e033e3ad4e2381ed0c2919e55dd9 already in progress: creating tablet
I20250115 03:06:20.716990 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 85cceb5a3f234d85937ad0bb75348991 (table default.Table_eec974cf89de4f45930dbf47fa754f2a [id=36660de3890e411e974c41b7f8f85946]) successfully deleted
W20250115 03:06:20.717510 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet b3334f58b90b4947af52150546ccaa28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b3334f58b90b4947af52150546ccaa28 already in progress: creating tablet
W20250115 03:06:20.717689 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet e356bbb02b9649b9b30a3cbadcf59a28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e356bbb02b9649b9b30a3cbadcf59a28 already in progress: creating tablet
I20250115 03:06:20.718431 14390 tablet_service.cc:1514] Processing DeleteTablet for tablet 4535ac88f78f48da83d9d2f951afdfa6 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a3964469dbd460c8416828043f862c4) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.718535 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet d8f54e9ccd04475f826ee851b0f079c0
I20250115 03:06:20.718575 14391 tablet_service.cc:1514] Processing DeleteTablet for tablet 839bdd3580024e849e9e1439ade8d3d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.718705 14656 tablet_service.cc:1514] Processing DeleteTablet for tablet 839bdd3580024e849e9e1439ade8d3d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.718896 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 62b185f27b5d4002b04dbb9d4ab566ca
I20250115 03:06:20.719223 14656 tablet_service.cc:1467] Processing CreateTablet for tablet 21408c08158a4cd8b77bbff89e7839e7 (DEFAULT_TABLE table=default.Table_59631f20f5e241509755c4b0e207f4dc [id=5306fcebb37542b7a5dd17b1b44b7e52]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.719439 14656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21408c08158a4cd8b77bbff89e7839e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.719800 14657 tablet_service.cc:1467] Processing CreateTablet for tablet 11d577daf54c4a9d8f9667f178b7af1e (DEFAULT_TABLE table=default.Table_54eb2aa18ec047739357df4000c84570 [id=e48cd80f25dc4150ac7a8aa5b8867363]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.719990 14657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11d577daf54c4a9d8f9667f178b7af1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.721383 14391 tablet_service.cc:1467] Processing CreateTablet for tablet 21408c08158a4cd8b77bbff89e7839e7 (DEFAULT_TABLE table=default.Table_59631f20f5e241509755c4b0e207f4dc [id=5306fcebb37542b7a5dd17b1b44b7e52]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.721602 14391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21408c08158a4cd8b77bbff89e7839e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.722517 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 4535ac88f78f48da83d9d2f951afdfa6 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a3964469dbd460c8416828043f862c4) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.723098 14820 ts_tablet_manager.cc:1905] T c7e9a1d1ada941b68688e0ed6742aa99 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.723500 14390 tablet_service.cc:1514] Processing DeleteTablet for tablet 80ede4e9696d491aa385b375fcb5cf3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.723651 14819 tablet_replica.cc:331] T 80ede4e9696d491aa385b375fcb5cf3b P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:20.723685 14809 tablet_bootstrap.cc:492] T 0f20bb66f36647e0811d0865b8f414f7 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.723758 14809 ts_tablet_manager.cc:1397] T 0f20bb66f36647e0811d0865b8f414f7 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.104s user 0.001s sys 0.000s
I20250115 03:06:20.723781 14819 raft_consensus.cc:2238] T 80ede4e9696d491aa385b375fcb5cf3b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.723860 14819 raft_consensus.cc:2267] T 80ede4e9696d491aa385b375fcb5cf3b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.724212 14809 raft_consensus.cc:357] T 0f20bb66f36647e0811d0865b8f414f7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.724256 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet d8f54e9ccd04475f826ee851b0f079c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.724314 14809 raft_consensus.cc:383] T 0f20bb66f36647e0811d0865b8f414f7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.724347 14809 raft_consensus.cc:738] T 0f20bb66f36647e0811d0865b8f414f7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.724458 14809 consensus_queue.cc:260] T 0f20bb66f36647e0811d0865b8f414f7 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.724656 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet 62b185f27b5d4002b04dbb9d4ab566ca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.724742 14809 ts_tablet_manager.cc:1428] T 0f20bb66f36647e0811d0865b8f414f7 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.724920 14809 tablet_bootstrap.cc:492] T 7043616bd6b34299862336924a6f16bf P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.725946 14809 tablet_bootstrap.cc:654] T 7043616bd6b34299862336924a6f16bf P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.729552 14809 tablet_bootstrap.cc:492] T 7043616bd6b34299862336924a6f16bf P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.729689 14809 ts_tablet_manager.cc:1397] T 7043616bd6b34299862336924a6f16bf P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20250115 03:06:20.730558 14809 raft_consensus.cc:357] T 7043616bd6b34299862336924a6f16bf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.730707 14809 raft_consensus.cc:383] T 7043616bd6b34299862336924a6f16bf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.732154 14809 raft_consensus.cc:738] T 7043616bd6b34299862336924a6f16bf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.732575 14809 consensus_queue.cc:260] T 7043616bd6b34299862336924a6f16bf P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.734848 14809 ts_tablet_manager.cc:1428] T 7043616bd6b34299862336924a6f16bf P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:20.735625 14809 tablet_bootstrap.cc:492] T 998a515113d04034928c04dc5798a698 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.737259 14809 tablet_bootstrap.cc:654] T 998a515113d04034928c04dc5798a698 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.738056 14819 ts_tablet_manager.cc:1905] T 80ede4e9696d491aa385b375fcb5cf3b P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.740200 14819 ts_tablet_manager.cc:1918] T 80ede4e9696d491aa385b375fcb5cf3b P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.740259 14819 log.cc:1198] T 80ede4e9696d491aa385b375fcb5cf3b P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/80ede4e9696d491aa385b375fcb5cf3b
I20250115 03:06:20.740406 14809 tablet_bootstrap.cc:492] T 998a515113d04034928c04dc5798a698 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.740444 14819 ts_tablet_manager.cc:1939] T 80ede4e9696d491aa385b375fcb5cf3b P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:20.740468 14809 ts_tablet_manager.cc:1397] T 998a515113d04034928c04dc5798a698 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20250115 03:06:20.740841 14809 raft_consensus.cc:357] T 998a515113d04034928c04dc5798a698 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.740947 14809 raft_consensus.cc:383] T 998a515113d04034928c04dc5798a698 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.740976 14809 raft_consensus.cc:738] T 998a515113d04034928c04dc5798a698 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.741081 14809 consensus_queue.cc:260] T 998a515113d04034928c04dc5798a698 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.741322 14809 ts_tablet_manager.cc:1428] T 998a515113d04034928c04dc5798a698 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.741580 14809 tablet_bootstrap.cc:492] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.745648 14811 tablet_bootstrap.cc:492] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.745759 14811 ts_tablet_manager.cc:1397] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.059s user 0.002s sys 0.000s
I20250115 03:06:20.746114 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_5e774c70bcb0426fb48e52c32ce75a4c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:20.746189 14811 raft_consensus.cc:357] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.746291 14811 raft_consensus.cc:383] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.746325 14811 raft_consensus.cc:738] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
W20250115 03:06:20.746420 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5e774c70bcb0426fb48e52c32ce75a4c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.746426 14811 consensus_queue.cc:260] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.746687 14811 ts_tablet_manager.cc:1428] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.746863 14809 tablet_bootstrap.cc:654] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.747051 14811 tablet_bootstrap.cc:492] T 5589b497b0e24740ad49d1392f5a737b P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.752625 16189 raft_consensus.cc:491] T 6bcfde5fe3d4425e98edb4dc54096718 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.752703 16189 raft_consensus.cc:513] T 6bcfde5fe3d4425e98edb4dc54096718 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.753126 16189 leader_election.cc:290] T 6bcfde5fe3d4425e98edb4dc54096718 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:20.754514 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bcfde5fe3d4425e98edb4dc54096718" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:20.755226 14809 tablet_bootstrap.cc:492] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.755304 14809 ts_tablet_manager.cc:1397] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20250115 03:06:20.755287 14534 tablet_service.cc:1467] Processing CreateTablet for tablet 4ed67a64c93948a2be9d282965966bf6 (DEFAULT_TABLE table=default.Table_40f50e0fd48242a59523563318d43e7a [id=1293a326960a42048ecbd0b9314cf4e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.755640 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 87bcad70ce6b46eda63015a9f1dab1e7 (DEFAULT_TABLE table=default.Table_63bec1dd11364c7fa3f990397751b825 [id=a9a0fb3700d84688808fb0123be3fc66]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.755709 14809 raft_consensus.cc:357] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.755788 14809 raft_consensus.cc:383] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.755818 14809 raft_consensus.cc:738] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.755919 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bcfde5fe3d4425e98edb4dc54096718" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:20.755921 14809 consensus_queue.cc:260] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.756250 14493 leader_election.cc:304] T 6bcfde5fe3d4425e98edb4dc54096718 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:20.756369 14809 ts_tablet_manager.cc:1428] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.756482 16189 raft_consensus.cc:2743] T 6bcfde5fe3d4425e98edb4dc54096718 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.757499 14809 tablet_bootstrap.cc:492] T 0792e52d171446b48a7c9d1f03d37a79 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.758420 14809 tablet_bootstrap.cc:654] T 0792e52d171446b48a7c9d1f03d37a79 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.760747 14390 tablet_service.cc:1467] Processing CreateTablet for tablet 11d577daf54c4a9d8f9667f178b7af1e (DEFAULT_TABLE table=default.Table_54eb2aa18ec047739357df4000c84570 [id=e48cd80f25dc4150ac7a8aa5b8867363]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.761006 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11d577daf54c4a9d8f9667f178b7af1e. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:20.762284 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 75b7e033e3ad4e2381ed0c2919e55dd9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 75b7e033e3ad4e2381ed0c2919e55dd9 already in progress: creating tablet
I20250115 03:06:20.762768 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet adbb0ae719f24dcc96709a83981e213d (table default.Table_e73230225df744a2a11f246bea89b8b0 [id=407331115fba4ca28b006a587584ea0d]) successfully deleted
I20250115 03:06:20.764804 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 440c8e4d6d474e20ace7f1d55add76da (DEFAULT_TABLE table=default.Table_46a85c9731254247bbef481a29502e48 [id=d8c11ea4f2dc4c299e4a755429d2703f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.764916 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 3a3964469dbd460c8416828043f862c4 (DEFAULT_TABLE table=default.Table_019ba03032a745be836deba6e091013e [id=1f8b8805819040a9935767a2f7b49fdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.765278 14526 tablet_service.cc:1467] Processing CreateTablet for tablet c8ba7e498bcf4359a1a588c1daea3fb0 (DEFAULT_TABLE table=default.Table_320deab8a90b44af8922fcc44f378bbf [id=4ec5823613be45c783a075a2b6e73c5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.762871 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet c7e9a1d1ada941b68688e0ed6742aa99 (table default.Table_941290b0882541e185286447fca362b9 [id=a2689bcea23843c9bb9e1f08e335e58f]) successfully deleted
W20250115 03:06:20.765925 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet b3334f58b90b4947af52150546ccaa28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b3334f58b90b4947af52150546ccaa28 already in progress: creating tablet
W20250115 03:06:20.766041 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet e356bbb02b9649b9b30a3cbadcf59a28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e356bbb02b9649b9b30a3cbadcf59a28 already in progress: creating tablet
I20250115 03:06:20.766134 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet bfc34dc76ff94b8eaeb5c2822e6498da (table default.Table_802f667f1ad246688cbdcffcb3001021 [id=e929ffaba567498c815393836b264a12]) successfully deleted
I20250115 03:06:20.766222 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 5589b497b0e24740ad49d1392f5a737b (table default.Table_8daa0abab4d1464488bec21bb7ad375b [id=37d965d8a5eb408ea3f33313bd22c152]) successfully deleted
W20250115 03:06:20.766353 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 839bdd3580024e849e9e1439ade8d3d9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 839bdd3580024e849e9e1439ade8d3d9 already in progress: creating tablet
W20250115 03:06:20.766547 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 4535ac88f78f48da83d9d2f951afdfa6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4535ac88f78f48da83d9d2f951afdfa6 already in progress: creating tablet
I20250115 03:06:20.766927 14390 tablet_service.cc:1514] Processing DeleteTablet for tablet d8f54e9ccd04475f826ee851b0f079c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.767469 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 440c8e4d6d474e20ace7f1d55add76da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.767706 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ed67a64c93948a2be9d282965966bf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.767875 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a3964469dbd460c8416828043f862c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.767992 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87bcad70ce6b46eda63015a9f1dab1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.768114 14526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8ba7e498bcf4359a1a588c1daea3fb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.768219 14809 tablet_bootstrap.cc:492] T 0792e52d171446b48a7c9d1f03d37a79 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.768291 14809 ts_tablet_manager.cc:1397] T 0792e52d171446b48a7c9d1f03d37a79 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20250115 03:06:20.768699 14809 raft_consensus.cc:357] T 0792e52d171446b48a7c9d1f03d37a79 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.768787 14809 raft_consensus.cc:383] T 0792e52d171446b48a7c9d1f03d37a79 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.768818 14809 raft_consensus.cc:738] T 0792e52d171446b48a7c9d1f03d37a79 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.768939 14809 consensus_queue.cc:260] T 0792e52d171446b48a7c9d1f03d37a79 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.769201 14809 ts_tablet_manager.cc:1428] T 0792e52d171446b48a7c9d1f03d37a79 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.769363 14809 tablet_bootstrap.cc:492] T 815ef85ce5cf4ae0a56c5734837e03ba P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.770344 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_02c01532217e4a8d9092ecb9bed7c891" } modify_external_catalogs: true
I20250115 03:06:20.770442 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_02c01532217e4a8d9092ecb9bed7c891" } modify_external_catalogs: true
I20250115 03:06:20.773927 14811 tablet_bootstrap.cc:654] T 5589b497b0e24740ad49d1392f5a737b P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.774963 14820 ts_tablet_manager.cc:1918] T c7e9a1d1ada941b68688e0ed6742aa99 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.775030 14820 log.cc:1198] T c7e9a1d1ada941b68688e0ed6742aa99 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/c7e9a1d1ada941b68688e0ed6742aa99
I20250115 03:06:20.775221 14820 ts_tablet_manager.cc:1939] T c7e9a1d1ada941b68688e0ed6742aa99 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.776732 14809 tablet_bootstrap.cc:654] T 815ef85ce5cf4ae0a56c5734837e03ba P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.778291 14809 tablet_bootstrap.cc:492] T 815ef85ce5cf4ae0a56c5734837e03ba P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.778359 14809 ts_tablet_manager.cc:1397] T 815ef85ce5cf4ae0a56c5734837e03ba P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20250115 03:06:20.778735 14809 raft_consensus.cc:357] T 815ef85ce5cf4ae0a56c5734837e03ba P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.778821 14809 raft_consensus.cc:383] T 815ef85ce5cf4ae0a56c5734837e03ba P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.778843 14390 tablet_service.cc:1514] Processing DeleteTablet for tablet 62b185f27b5d4002b04dbb9d4ab566ca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.779095 14391 tablet_service.cc:1467] Processing CreateTablet for tablet 4ed67a64c93948a2be9d282965966bf6 (DEFAULT_TABLE table=default.Table_40f50e0fd48242a59523563318d43e7a [id=1293a326960a42048ecbd0b9314cf4e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.779287 14391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ed67a64c93948a2be9d282965966bf6. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:20.780764 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 4535ac88f78f48da83d9d2f951afdfa6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4535ac88f78f48da83d9d2f951afdfa6 already in progress: creating tablet
W20250115 03:06:20.780898 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 839bdd3580024e849e9e1439ade8d3d9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 839bdd3580024e849e9e1439ade8d3d9 already in progress: creating tablet
I20250115 03:06:20.781072 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 80ede4e9696d491aa385b375fcb5cf3b (table default.Table_75cd7b01fc534652975eb593912b88bc [id=111fc67094164d3f99965c246ae12b24]) successfully deleted
I20250115 03:06:20.781138 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 87bcad70ce6b46eda63015a9f1dab1e7 (DEFAULT_TABLE table=default.Table_63bec1dd11364c7fa3f990397751b825 [id=a9a0fb3700d84688808fb0123be3fc66]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.781370 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 440c8e4d6d474e20ace7f1d55add76da (DEFAULT_TABLE table=default.Table_46a85c9731254247bbef481a29502e48 [id=d8c11ea4f2dc4c299e4a755429d2703f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.781564 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 3a3964469dbd460c8416828043f862c4 (DEFAULT_TABLE table=default.Table_019ba03032a745be836deba6e091013e [id=1f8b8805819040a9935767a2f7b49fdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.781737 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 15fbf8d5de904fc4b284af664f3bd60f (table default.Table_655e9d02aacf4b88ba521f0f57a5f522 [id=6ed01835736b40f8b4caeca7378770d8]). Expected version 1 got 0
I20250115 03:06:20.781843 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 7b86e66428504217af08f667f660d304 (table default.Table_341a3cd1c8c94ff4b05fc21960a4e9b4 [id=c096ce170874405ea740b9ea829f886e]). Expected version 2 got 0
I20250115 03:06:20.782523 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet c7e9a1d1ada941b68688e0ed6742aa99 (table default.Table_941290b0882541e185286447fca362b9 [id=a2689bcea23843c9bb9e1f08e335e58f]) successfully deleted
W20250115 03:06:20.782634 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet d8f54e9ccd04475f826ee851b0f079c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d8f54e9ccd04475f826ee851b0f079c0 already in progress: creating tablet
I20250115 03:06:20.782671 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a3964469dbd460c8416828043f862c4. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:20.782716 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 62b185f27b5d4002b04dbb9d4ab566ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62b185f27b5d4002b04dbb9d4ab566ca already in progress: creating tablet
I20250115 03:06:20.783150 14812 tablet_bootstrap.cc:492] T 0521973e23984355a77a321b4676b479 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.783226 14812 ts_tablet_manager.cc:1397] T 0521973e23984355a77a321b4676b479 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.217s user 0.002s sys 0.000s
I20250115 03:06:20.783300 14656 tablet_service.cc:1467] Processing CreateTablet for tablet 4ed67a64c93948a2be9d282965966bf6 (DEFAULT_TABLE table=default.Table_40f50e0fd48242a59523563318d43e7a [id=1293a326960a42048ecbd0b9314cf4e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.783528 14656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ed67a64c93948a2be9d282965966bf6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.783648 14812 raft_consensus.cc:357] T 0521973e23984355a77a321b4676b479 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.783758 14812 raft_consensus.cc:383] T 0521973e23984355a77a321b4676b479 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.783789 14812 raft_consensus.cc:738] T 0521973e23984355a77a321b4676b479 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.783912 14812 consensus_queue.cc:260] T 0521973e23984355a77a321b4676b479 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.784171 14812 ts_tablet_manager.cc:1428] T 0521973e23984355a77a321b4676b479 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.784354 14812 tablet_bootstrap.cc:492] T a73d45bf459c4137af6961b680d282ac P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.784587 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 440c8e4d6d474e20ace7f1d55add76da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.785053 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87bcad70ce6b46eda63015a9f1dab1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.785257 14812 tablet_bootstrap.cc:654] T a73d45bf459c4137af6961b680d282ac P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.785511 14671 tablet_service.cc:1467] Processing CreateTablet for tablet 87bcad70ce6b46eda63015a9f1dab1e7 (DEFAULT_TABLE table=default.Table_63bec1dd11364c7fa3f990397751b825 [id=a9a0fb3700d84688808fb0123be3fc66]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.785717 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87bcad70ce6b46eda63015a9f1dab1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.785746 14657 tablet_service.cc:1514] Processing DeleteTablet for tablet 62b185f27b5d4002b04dbb9d4ab566ca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.785966 14659 tablet_service.cc:1467] Processing CreateTablet for tablet c8ba7e498bcf4359a1a588c1daea3fb0 (DEFAULT_TABLE table=default.Table_320deab8a90b44af8922fcc44f378bbf [id=4ec5823613be45c783a075a2b6e73c5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.786085 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 3a3964469dbd460c8416828043f862c4 (DEFAULT_TABLE table=default.Table_019ba03032a745be836deba6e091013e [id=1f8b8805819040a9935767a2f7b49fdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.786147 14659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8ba7e498bcf4359a1a588c1daea3fb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.786283 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a3964469dbd460c8416828043f862c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.787541 14668 tablet_service.cc:1467] Processing CreateTablet for tablet 440c8e4d6d474e20ace7f1d55add76da (DEFAULT_TABLE table=default.Table_46a85c9731254247bbef481a29502e48 [id=d8c11ea4f2dc4c299e4a755429d2703f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.787743 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 440c8e4d6d474e20ace7f1d55add76da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.790251 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet d8f54e9ccd04475f826ee851b0f079c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.790378 14553 raft_consensus.cc:3054] T 15fbf8d5de904fc4b284af664f3bd60f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.792618 14390 tablet_service.cc:1467] Processing CreateTablet for tablet c8ba7e498bcf4359a1a588c1daea3fb0 (DEFAULT_TABLE table=default.Table_320deab8a90b44af8922fcc44f378bbf [id=4ec5823613be45c783a075a2b6e73c5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.792835 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8ba7e498bcf4359a1a588c1daea3fb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.793596 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d6bfa64233ea4a3d87766891d5b37ca9" } modify_external_catalogs: true
I20250115 03:06:20.793694 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d6bfa64233ea4a3d87766891d5b37ca9" } modify_external_catalogs: true
W20250115 03:06:20.781227 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet d8f54e9ccd04475f826ee851b0f079c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d8f54e9ccd04475f826ee851b0f079c0 already in progress: creating tablet
W20250115 03:06:20.795511 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 62b185f27b5d4002b04dbb9d4ab566ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62b185f27b5d4002b04dbb9d4ab566ca already in progress: creating tablet
I20250115 03:06:20.796310 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 2d3157859e684da98bb473d5799b24a6
W20250115 03:06:20.796442 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 62b185f27b5d4002b04dbb9d4ab566ca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62b185f27b5d4002b04dbb9d4ab566ca already in progress: creating tablet
W20250115 03:06:20.796552 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet d8f54e9ccd04475f826ee851b0f079c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d8f54e9ccd04475f826ee851b0f079c0 already in progress: creating tablet
I20250115 03:06:20.797055 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d3157859e684da98bb473d5799b24a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.797235 14823 tablet_replica.cc:331] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:20.797536 14823 raft_consensus.cc:2238] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.797644 14823 raft_consensus.cc:2267] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.797968 14823 ts_tablet_manager.cc:1905] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.799314 14165 master_service.cc:946] ReplaceTablet: received request to replace tablet 8e4410ce33f0464eadead04e8116f773 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:20.800472 14823 ts_tablet_manager.cc:1918] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:20.800556 14823 log.cc:1198] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/2d3157859e684da98bb473d5799b24a6
I20250115 03:06:20.778851 14809 raft_consensus.cc:738] T 815ef85ce5cf4ae0a56c5734837e03ba P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.800669 14809 consensus_queue.cc:260] T 815ef85ce5cf4ae0a56c5734837e03ba P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.800977 14809 ts_tablet_manager.cc:1428] T 815ef85ce5cf4ae0a56c5734837e03ba P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.023s user 0.001s sys 0.000s
I20250115 03:06:20.801139 14809 tablet_bootstrap.cc:492] T 12678da7ffee4a46b4690ac3e94cb19b P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.801841 14809 tablet_bootstrap.cc:654] T 12678da7ffee4a46b4690ac3e94cb19b P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.802664 14809 tablet_bootstrap.cc:492] T 12678da7ffee4a46b4690ac3e94cb19b P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.802740 14809 ts_tablet_manager.cc:1397] T 12678da7ffee4a46b4690ac3e94cb19b P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:20.803076 14809 raft_consensus.cc:357] T 12678da7ffee4a46b4690ac3e94cb19b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.803164 14809 raft_consensus.cc:383] T 12678da7ffee4a46b4690ac3e94cb19b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.803190 14809 raft_consensus.cc:738] T 12678da7ffee4a46b4690ac3e94cb19b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.803189 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 978e86e4ed524512b3cc69c262783b4f
I20250115 03:06:20.806352 14809 consensus_queue.cc:260] T 12678da7ffee4a46b4690ac3e94cb19b P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.806644 14809 ts_tablet_manager.cc:1428] T 12678da7ffee4a46b4690ac3e94cb19b P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:20.806881 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 978e86e4ed524512b3cc69c262783b4f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.806994 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet ef0994b1e51b4c688354998730e7b743 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:20.807325 14809 tablet_bootstrap.cc:492] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.808060 14809 tablet_bootstrap.cc:654] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.801039 14812 tablet_bootstrap.cc:492] T a73d45bf459c4137af6961b680d282ac P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.808347 14812 ts_tablet_manager.cc:1397] T a73d45bf459c4137af6961b680d282ac P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.000s
I20250115 03:06:20.808825 14812 raft_consensus.cc:357] T a73d45bf459c4137af6961b680d282ac P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.808919 14812 raft_consensus.cc:383] T a73d45bf459c4137af6961b680d282ac P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.809003 14812 raft_consensus.cc:738] T a73d45bf459c4137af6961b680d282ac P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.809111 14812 consensus_queue.cc:260] T a73d45bf459c4137af6961b680d282ac P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.809391 14812 ts_tablet_manager.cc:1428] T a73d45bf459c4137af6961b680d282ac P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:20.809410 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 8e4410ce33f0464eadead04e8116f773
I20250115 03:06:20.809581 14812 tablet_bootstrap.cc:492] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.809655 14165 catalog_manager.cc:6596] ReplaceTablet: tablet 8e4410ce33f0464eadead04e8116f773 deleted and replaced by tablet a7e5a09a30314e088f327cee8c9b4fde
I20250115 03:06:20.810099 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e4410ce33f0464eadead04e8116f773 with delete_type TABLET_DATA_DELETED (replaced by tablet a7e5a09a30314e088f327cee8c9b4fde) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.810454 14809 tablet_bootstrap.cc:492] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.810495 14812 tablet_bootstrap.cc:654] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.810524 14809 ts_tablet_manager.cc:1397] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:20.810968 14809 raft_consensus.cc:357] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.811067 14809 raft_consensus.cc:383] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.811097 14809 raft_consensus.cc:738] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.811197 14809 consensus_queue.cc:260] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.811537 14809 ts_tablet_manager.cc:1428] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.811730 14809 tablet_bootstrap.cc:492] T 4122c97f5ba445ee8c79b7d056438229 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.811738 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d3157859e684da98bb473d5799b24a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.813855 14809 tablet_bootstrap.cc:654] T 4122c97f5ba445ee8c79b7d056438229 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.814121 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet ef0994b1e51b4c688354998730e7b743
I20250115 03:06:20.814373 14163 catalog_manager.cc:6596] ReplaceTablet: tablet ef0994b1e51b4c688354998730e7b743 deleted and replaced by tablet db3c5684202e452b92f883d9a45f6e97
I20250115 03:06:20.814975 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet ef0994b1e51b4c688354998730e7b743 with delete_type TABLET_DATA_DELETED (replaced by tablet db3c5684202e452b92f883d9a45f6e97) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.815131 14390 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d3157859e684da98bb473d5799b24a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.815268 14819 tablet_replica.cc:331] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:20.815372 14819 raft_consensus.cc:2238] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:20.815541 14819 raft_consensus.cc:2267] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.815872 14819 ts_tablet_manager.cc:1905] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.816051 14809 tablet_bootstrap.cc:492] T 4122c97f5ba445ee8c79b7d056438229 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.816128 14809 ts_tablet_manager.cc:1397] T 4122c97f5ba445ee8c79b7d056438229 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250115 03:06:20.816532 14809 raft_consensus.cc:357] T 4122c97f5ba445ee8c79b7d056438229 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.816617 14809 raft_consensus.cc:383] T 4122c97f5ba445ee8c79b7d056438229 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.816643 14809 raft_consensus.cc:738] T 4122c97f5ba445ee8c79b7d056438229 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.816773 14809 consensus_queue.cc:260] T 4122c97f5ba445ee8c79b7d056438229 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.817050 14809 ts_tablet_manager.cc:1428] T 4122c97f5ba445ee8c79b7d056438229 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.817198 14809 tablet_bootstrap.cc:492] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.817250 14823 ts_tablet_manager.cc:1939] T 2d3157859e684da98bb473d5799b24a6 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:20.817979 14819 ts_tablet_manager.cc:1918] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:20.818017 14809 tablet_bootstrap.cc:654] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.818040 14819 log.cc:1198] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/2d3157859e684da98bb473d5799b24a6
I20250115 03:06:20.818253 14819 ts_tablet_manager.cc:1939] T 2d3157859e684da98bb473d5799b24a6 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:20.818470 14820 tablet_replica.cc:331] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.818523 14059 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu with pid 14941
W20250115 03:06:20.818535 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 978e86e4ed524512b3cc69c262783b4f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 978e86e4ed524512b3cc69c262783b4f already in progress: creating tablet
I20250115 03:06:20.818614 14820 raft_consensus.cc:2238] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
W20250115 03:06:20.818670 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 8e4410ce33f0464eadead04e8116f773 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e4410ce33f0464eadead04e8116f773 already in progress: creating tablet
I20250115 03:06:20.818723 14820 raft_consensus.cc:2267] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.819550 14820 ts_tablet_manager.cc:1905] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.820230 14809 tablet_bootstrap.cc:492] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.820304 14809 ts_tablet_manager.cc:1397] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:20.820608 14553 raft_consensus.cc:3054] T e5a8293f43134a54b3f8dfb5fb99d6ea P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.820673 14809 raft_consensus.cc:357] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.820772 14809 raft_consensus.cc:383] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.820825 14809 raft_consensus.cc:738] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.820942 14809 consensus_queue.cc:260] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.821195 14809 ts_tablet_manager.cc:1428] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.822679 14809 tablet_bootstrap.cc:492] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.823702 14809 tablet_bootstrap.cc:654] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.826689 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 38d59dfaccb34de5910b5045785b3d7c (DEFAULT_TABLE table=default.Table_b0334b90634c4543b8ced12e02fa7114 [id=2bb0542f45c944f097b9391a10530896]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.826954 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38d59dfaccb34de5910b5045785b3d7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.827225 14809 tablet_bootstrap.cc:492] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.827286 14809 ts_tablet_manager.cc:1397] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20250115 03:06:20.827637 14809 raft_consensus.cc:357] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.827714 14809 raft_consensus.cc:383] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.827750 14809 raft_consensus.cc:738] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.827844 14809 consensus_queue.cc:260] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.828244 14809 ts_tablet_manager.cc:1428] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.828397 14809 tablet_bootstrap.cc:492] T 38f5554b10a44c249e4c3d3825cd7952 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.829380 14809 tablet_bootstrap.cc:654] T 38f5554b10a44c249e4c3d3825cd7952 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.830080 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 863ad53d7fe447a290a43d7f0501cbc1 (DEFAULT_TABLE table=default.Table_e2b439c7d5064b22a1e7d2033f74217b [id=f1d987c9b79e4187aa513d0c3c3af2f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.830349 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 863ad53d7fe447a290a43d7f0501cbc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.830477 14171 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 15fbf8d5de904fc4b284af664f3bd60f (table default.Table_655e9d02aacf4b88ba521f0f57a5f522 [id=6ed01835736b40f8b4caeca7378770d8]). Expected version 1 got 0
I20250115 03:06:20.831166 14527 tablet_service.cc:1467] Processing CreateTablet for tablet a528e049654e4eb78f7d03d8c72ef21d (DEFAULT_TABLE table=default.Table_5e774c70bcb0426fb48e52c32ce75a4c [id=4d79d4a9bc8e40eebfd3e670b6dfdbee]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.831426 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a528e049654e4eb78f7d03d8c72ef21d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.833707 14534 tablet_service.cc:1467] Processing CreateTablet for tablet 0497374d562b4521ac10a624c89d70b0 (DEFAULT_TABLE table=default.Table_bd02870cfaf64d6187493faecb6ff806 [id=59acf63681e347bc9c7c017a4ae97fc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.833956 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0497374d562b4521ac10a624c89d70b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.836289 14820 ts_tablet_manager.cc:1918] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:20.836405 14820 log.cc:1198] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/2d3157859e684da98bb473d5799b24a6
I20250115 03:06:20.836697 14820 ts_tablet_manager.cc:1939] T 2d3157859e684da98bb473d5799b24a6 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.837096 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 2d3157859e684da98bb473d5799b24a6 (table default.Table_02c01532217e4a8d9092ecb9bed7c891 [id=265c248361624c3f9e5ece27144d8e04]) successfully deleted
W20250115 03:06:20.837473 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet ef0994b1e51b4c688354998730e7b743 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ef0994b1e51b4c688354998730e7b743 already in progress: creating tablet
I20250115 03:06:20.837663 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 978e86e4ed524512b3cc69c262783b4f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.838229 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e4410ce33f0464eadead04e8116f773 with delete_type TABLET_DATA_DELETED (replaced by tablet a7e5a09a30314e088f327cee8c9b4fde) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.838267 14390 tablet_service.cc:1514] Processing DeleteTablet for tablet 978e86e4ed524512b3cc69c262783b4f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.838425 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e4410ce33f0464eadead04e8116f773 with delete_type TABLET_DATA_DELETED (replaced by tablet a7e5a09a30314e088f327cee8c9b4fde) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.839077 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet ef0994b1e51b4c688354998730e7b743 with delete_type TABLET_DATA_DELETED (replaced by tablet db3c5684202e452b92f883d9a45f6e97) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.839425 14390 tablet_service.cc:1514] Processing DeleteTablet for tablet ef0994b1e51b4c688354998730e7b743 with delete_type TABLET_DATA_DELETED (replaced by tablet db3c5684202e452b92f883d9a45f6e97) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.840766 14422 raft_consensus.cc:1270] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 5c0dae7aff4b44c699275b95ba639770: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:20.841136 16088 consensus_queue.cc:1035] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:20.841619 14534 tablet_service.cc:1467] Processing CreateTablet for tablet db3c5684202e452b92f883d9a45f6e97 (DEFAULT_TABLE table=default.Table_04c92b10d83444518cc1805f51e8babb [id=ecace87761f746999c8cca0525485c6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.841778 14536 tablet_service.cc:1467] Processing CreateTablet for tablet a7e5a09a30314e088f327cee8c9b4fde (DEFAULT_TABLE table=default.Table_77d4ac27a2c44250a2dcc62121f202c7 [id=6af8f6f562ac47b484eeb4d67ac6c7df]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.841830 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db3c5684202e452b92f883d9a45f6e97. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.841998 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 38d59dfaccb34de5910b5045785b3d7c (DEFAULT_TABLE table=default.Table_b0334b90634c4543b8ced12e02fa7114 [id=2bb0542f45c944f097b9391a10530896]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.842217 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38d59dfaccb34de5910b5045785b3d7c. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:20.843801 14626 consensus_peers.cc:487] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770 -> Peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Couldn't send request to peer 84b1408700ef479cbb5a7b4d2f0bebf1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:20.844595 14390 tablet_service.cc:1467] Processing CreateTablet for tablet 38d59dfaccb34de5910b5045785b3d7c (DEFAULT_TABLE table=default.Table_b0334b90634c4543b8ced12e02fa7114 [id=2bb0542f45c944f097b9391a10530896]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.844619 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 863ad53d7fe447a290a43d7f0501cbc1 (DEFAULT_TABLE table=default.Table_e2b439c7d5064b22a1e7d2033f74217b [id=f1d987c9b79e4187aa513d0c3c3af2f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.844812 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38d59dfaccb34de5910b5045785b3d7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.844812 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 863ad53d7fe447a290a43d7f0501cbc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.844812 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7e5a09a30314e088f327cee8c9b4fde. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.845183 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 863ad53d7fe447a290a43d7f0501cbc1 (DEFAULT_TABLE table=default.Table_e2b439c7d5064b22a1e7d2033f74217b [id=f1d987c9b79e4187aa513d0c3c3af2f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.845191 14659 tablet_service.cc:1467] Processing CreateTablet for tablet a528e049654e4eb78f7d03d8c72ef21d (DEFAULT_TABLE table=default.Table_5e774c70bcb0426fb48e52c32ce75a4c [id=4d79d4a9bc8e40eebfd3e670b6dfdbee]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.845393 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 863ad53d7fe447a290a43d7f0501cbc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.845408 14659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a528e049654e4eb78f7d03d8c72ef21d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.847815 14397 tablet_service.cc:1467] Processing CreateTablet for tablet a528e049654e4eb78f7d03d8c72ef21d (DEFAULT_TABLE table=default.Table_5e774c70bcb0426fb48e52c32ce75a4c [id=4d79d4a9bc8e40eebfd3e670b6dfdbee]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.847827 14659 tablet_service.cc:1467] Processing CreateTablet for tablet 0497374d562b4521ac10a624c89d70b0 (DEFAULT_TABLE table=default.Table_bd02870cfaf64d6187493faecb6ff806 [id=59acf63681e347bc9c7c017a4ae97fc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.847991 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a528e049654e4eb78f7d03d8c72ef21d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.848150 14659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0497374d562b4521ac10a624c89d70b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.850041 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet 103f78d0d9fc42308e1e23316895be33 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:20.850234 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 0497374d562b4521ac10a624c89d70b0 (DEFAULT_TABLE table=default.Table_bd02870cfaf64d6187493faecb6ff806 [id=59acf63681e347bc9c7c017a4ae97fc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.850450 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0497374d562b4521ac10a624c89d70b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.850426 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 2d3157859e684da98bb473d5799b24a6 (table default.Table_02c01532217e4a8d9092ecb9bed7c891 [id=265c248361624c3f9e5ece27144d8e04]) successfully deleted
W20250115 03:06:20.850559 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 8e4410ce33f0464eadead04e8116f773 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e4410ce33f0464eadead04e8116f773 already in progress: creating tablet
W20250115 03:06:20.850669 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 978e86e4ed524512b3cc69c262783b4f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 978e86e4ed524512b3cc69c262783b4f already in progress: creating tablet
W20250115 03:06:20.850775 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet ef0994b1e51b4c688354998730e7b743 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ef0994b1e51b4c688354998730e7b743 already in progress: creating tablet
I20250115 03:06:20.850872 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 2d3157859e684da98bb473d5799b24a6 (table default.Table_02c01532217e4a8d9092ecb9bed7c891 [id=265c248361624c3f9e5ece27144d8e04]) successfully deleted
W20250115 03:06:20.851025 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 978e86e4ed524512b3cc69c262783b4f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 978e86e4ed524512b3cc69c262783b4f already in progress: creating tablet
W20250115 03:06:20.851142 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 8e4410ce33f0464eadead04e8116f773 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8e4410ce33f0464eadead04e8116f773 already in progress: creating tablet
W20250115 03:06:20.851251 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet ef0994b1e51b4c688354998730e7b743 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ef0994b1e51b4c688354998730e7b743 already in progress: creating tablet
I20250115 03:06:20.851950 14391 tablet_service.cc:1467] Processing CreateTablet for tablet db3c5684202e452b92f883d9a45f6e97 (DEFAULT_TABLE table=default.Table_04c92b10d83444518cc1805f51e8babb [id=ecace87761f746999c8cca0525485c6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.852051 14659 tablet_service.cc:1467] Processing CreateTablet for tablet db3c5684202e452b92f883d9a45f6e97 (DEFAULT_TABLE table=default.Table_04c92b10d83444518cc1805f51e8babb [id=ecace87761f746999c8cca0525485c6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.852156 14391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db3c5684202e452b92f883d9a45f6e97. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.852223 14659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db3c5684202e452b92f883d9a45f6e97. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.853116 14393 tablet_service.cc:1467] Processing CreateTablet for tablet a7e5a09a30314e088f327cee8c9b4fde (DEFAULT_TABLE table=default.Table_77d4ac27a2c44250a2dcc62121f202c7 [id=6af8f6f562ac47b484eeb4d67ac6c7df]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.853322 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7e5a09a30314e088f327cee8c9b4fde. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:20.853520 14492 connection.cc:537] client connection to 127.13.186.254:39901 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20250115 03:06:20.853672 14668 tablet_service.cc:1467] Processing CreateTablet for tablet a7e5a09a30314e088f327cee8c9b4fde (DEFAULT_TABLE table=default.Table_77d4ac27a2c44250a2dcc62121f202c7 [id=6af8f6f562ac47b484eeb4d67ac6c7df]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.853895 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7e5a09a30314e088f327cee8c9b4fde. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:20.853879 14357 connection.cc:537] client connection to 127.13.186.254:39901 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20250115 03:06:20.855674 14812 tablet_bootstrap.cc:492] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.855751 14812 ts_tablet_manager.cc:1397] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.046s user 0.001s sys 0.001s
I20250115 03:06:20.856115 14811 tablet_bootstrap.cc:492] T 5589b497b0e24740ad49d1392f5a737b P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.856190 14811 ts_tablet_manager.cc:1397] T 5589b497b0e24740ad49d1392f5a737b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.109s user 0.002s sys 0.000s
I20250115 03:06:20.856223 14812 raft_consensus.cc:357] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.856310 14812 raft_consensus.cc:383] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.856338 14812 raft_consensus.cc:738] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.856437 14812 consensus_queue.cc:260] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.856709 14812 ts_tablet_manager.cc:1428] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.856871 14812 tablet_bootstrap.cc:492] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.857028 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet a73d45bf459c4137af6961b680d282ac with delete_type TABLET_DATA_DELETED (replaced by tablet 15263a3b32be418288770ce348c082d7) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.856997 14811 raft_consensus.cc:357] T 5589b497b0e24740ad49d1392f5a737b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.857091 14811 raft_consensus.cc:383] T 5589b497b0e24740ad49d1392f5a737b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.857121 14811 raft_consensus.cc:738] T 5589b497b0e24740ad49d1392f5a737b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.857236 14811 consensus_queue.cc:260] T 5589b497b0e24740ad49d1392f5a737b P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.857496 14811 ts_tablet_manager.cc:1428] T 5589b497b0e24740ad49d1392f5a737b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20250115 03:06:20.857642 14811 tablet_bootstrap.cc:492] T 85b4eb3b20bb4908884e5716e7b2a369 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.858531 14811 tablet_bootstrap.cc:654] T 85b4eb3b20bb4908884e5716e7b2a369 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.852874 14171 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 0f20bb66f36647e0811d0865b8f414f7 (table default.Table_0326e94494d0447585233c0c907113fd [id=f48964597a90470299a755217c4dbea0]). Expected version 1 got 0
I20250115 03:06:20.861532 14171 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet a4547465f3124e108dc2c4c54e2bc751 (table default.Table_58e0901d0ec94ef9859515f126825621 [id=50c73c26b4ed47a4961348db433b7dd4]). Expected version 3 got 0
W20250115 03:06:20.862263 14171 catalog_manager.cc:5718] Failed to send AlterTable RPC for tablet a4547465f3124e108dc2c4c54e2bc751 (table default.Table_58e0901d0ec94ef9859515f126825621 [id=50c73c26b4ed47a4961348db433b7dd4], current schema version=3): Not found: failed to reset TS proxy: No leader found: Tablet a4547465f3124e108dc2c4c54e2bc751 consensus state has no leader
I20250115 03:06:20.862313 14812 tablet_bootstrap.cc:654] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:20.855700 14157 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.186.254:39901 (error 108) [suppressed 2 similar messages]
I20250115 03:06:20.863081 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 5589b497b0e24740ad49d1392f5a737b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.864864 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 0792e52d171446b48a7c9d1f03d37a79 with delete_type TABLET_DATA_DELETED (replaced by tablet d65bf3745b4f4d92a346b4dd18652432) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.865337 14809 tablet_bootstrap.cc:492] T 38f5554b10a44c249e4c3d3825cd7952 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.865463 14809 ts_tablet_manager.cc:1397] T 38f5554b10a44c249e4c3d3825cd7952 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.000s
I20250115 03:06:20.866168 14809 raft_consensus.cc:357] T 38f5554b10a44c249e4c3d3825cd7952 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.866315 14809 raft_consensus.cc:383] T 38f5554b10a44c249e4c3d3825cd7952 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.866384 14809 raft_consensus.cc:738] T 38f5554b10a44c249e4c3d3825cd7952 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.866550 14809 consensus_queue.cc:260] T 38f5554b10a44c249e4c3d3825cd7952 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.867090 14809 ts_tablet_manager.cc:1428] T 38f5554b10a44c249e4c3d3825cd7952 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:20.867281 14809 tablet_bootstrap.cc:492] T d6cf1a0b9c81413b8f72de0cafc76963 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.868564 14809 tablet_bootstrap.cc:654] T d6cf1a0b9c81413b8f72de0cafc76963 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.868589 14162 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet bfeafbf380df41738ebbcc11c19a7f85 (table default.Table_660872e60c104e56be9287f66d74c387 [id=03ee00da1e024ebab51b3a366d2c0cf3]). Expected version 2 got 0
I20250115 03:06:20.868708 14162 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet f2ca06c402cd46dc8311c8a47da92846 (table default.Table_802f667f1ad246688cbdcffcb3001021 [id=e929ffaba567498c815393836b264a12]). Expected version 1 got 0
I20250115 03:06:20.868809 14162 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 9febec144bb343a8a21817e6d3655ad7 (table default.Table_57fbe77148284b4a9666630570f69321 [id=c9ed0b9c92ef40efa5c51cfdd8389325]). Expected version 2 got 0
I20250115 03:06:20.868950 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ba02ce2e8d4c4274a1d3f75fc1f8558f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.869299 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba02ce2e8d4c4274a1d3f75fc1f8558f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.869335 14819 tablet_replica.cc:331] T 0792e52d171446b48a7c9d1f03d37a79 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:20.869457 14819 raft_consensus.cc:2238] T 0792e52d171446b48a7c9d1f03d37a79 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.869545 14819 raft_consensus.cc:2267] T 0792e52d171446b48a7c9d1f03d37a79 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.869889 14809 tablet_bootstrap.cc:492] T d6cf1a0b9c81413b8f72de0cafc76963 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.869954 14809 ts_tablet_manager.cc:1397] T d6cf1a0b9c81413b8f72de0cafc76963 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20250115 03:06:20.870456 14809 raft_consensus.cc:357] T d6cf1a0b9c81413b8f72de0cafc76963 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.870577 14809 raft_consensus.cc:383] T d6cf1a0b9c81413b8f72de0cafc76963 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.870610 14809 raft_consensus.cc:738] T d6cf1a0b9c81413b8f72de0cafc76963 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.870703 14809 consensus_queue.cc:260] T d6cf1a0b9c81413b8f72de0cafc76963 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.870954 14809 ts_tablet_manager.cc:1428] T d6cf1a0b9c81413b8f72de0cafc76963 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:20.871089 14809 tablet_bootstrap.cc:492] T bb68869eaf55477090cf12f40f76a89c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.871951 14809 tablet_bootstrap.cc:654] T bb68869eaf55477090cf12f40f76a89c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.872129 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet c6fa98c6fb304dc49b8e0ced989cc936 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.873042 14823 tablet_replica.cc:331] T a73d45bf459c4137af6961b680d282ac P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:20.873159 14823 raft_consensus.cc:2238] T a73d45bf459c4137af6961b680d282ac P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.873239 14823 raft_consensus.cc:2267] T a73d45bf459c4137af6961b680d282ac P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.874069 15970 tablet.cc:1721] T 9febec144bb343a8a21817e6d3655ad7 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:20.874365 15617 tablet.cc:1721] T 9febec144bb343a8a21817e6d3655ad7 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:20.874403 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_08cd4bc70fd145d89e52bec66a069523"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.874696 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_08cd4bc70fd145d89e52bec66a069523 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.875377 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_09dd39785939469795df4097f8b9a6a4" } modify_external_catalogs: true
I20250115 03:06:20.875468 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_09dd39785939469795df4097f8b9a6a4" } modify_external_catalogs: true
I20250115 03:06:20.875692 14809 tablet_bootstrap.cc:492] T bb68869eaf55477090cf12f40f76a89c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.875756 14809 ts_tablet_manager.cc:1397] T bb68869eaf55477090cf12f40f76a89c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:20.876164 14809 raft_consensus.cc:357] T bb68869eaf55477090cf12f40f76a89c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.876165 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_fb75e1b5f16f45dc9e9fc586e358dafb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:20.876246 14809 raft_consensus.cc:383] T bb68869eaf55477090cf12f40f76a89c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.876273 14809 raft_consensus.cc:738] T bb68869eaf55477090cf12f40f76a89c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
W20250115 03:06:20.876444 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fb75e1b5f16f45dc9e9fc586e358dafb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.876523 14809 consensus_queue.cc:260] T bb68869eaf55477090cf12f40f76a89c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.876863 14809 ts_tablet_manager.cc:1428] T bb68869eaf55477090cf12f40f76a89c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.877033 14809 tablet_bootstrap.cc:492] T b2629ae9d9854ae4b6e7ddddbb6f341b P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.878137 14809 tablet_bootstrap.cc:654] T b2629ae9d9854ae4b6e7ddddbb6f341b P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.878767 14171 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4ec1365f1a874eb49b19337126347443" } new_table_name: "default.Table_cba2b8d369b84bfd9f72a40955005bd6" modify_external_catalogs: true
I20250115 03:06:20.878799 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_72b70810c1714448bba0bbf17f9d559b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.879048 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_72b70810c1714448bba0bbf17f9d559b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.879441 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f802f82d280b4f29a5fd691e8a6f73f6" } new_table_name: "default.Table_f36db055f5c24120ab65d6c027d792b7" modify_external_catalogs: true
I20250115 03:06:20.880357 14812 tablet_bootstrap.cc:492] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.880489 14812 ts_tablet_manager.cc:1397] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20250115 03:06:20.880847 14823 ts_tablet_manager.cc:1905] T a73d45bf459c4137af6961b680d282ac P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.882411 14812 raft_consensus.cc:357] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.882593 14812 raft_consensus.cc:383] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.882673 14812 raft_consensus.cc:738] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.882850 14812 consensus_queue.cc:260] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.884151 14812 ts_tablet_manager.cc:1428] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:20.884418 14812 tablet_bootstrap.cc:492] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.885352 14812 tablet_bootstrap.cc:654] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.886107 14820 tablet_replica.cc:331] T 5589b497b0e24740ad49d1392f5a737b P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.886291 14820 raft_consensus.cc:2238] T 5589b497b0e24740ad49d1392f5a737b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.886445 14820 raft_consensus.cc:2267] T 5589b497b0e24740ad49d1392f5a737b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.887125 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet d9ddca2936db4adb92c8ccb34693e24c
I20250115 03:06:20.887605 14823 ts_tablet_manager.cc:1918] T a73d45bf459c4137af6961b680d282ac P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.887662 14823 log.cc:1198] T a73d45bf459c4137af6961b680d282ac P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/a73d45bf459c4137af6961b680d282ac
I20250115 03:06:20.887848 14823 ts_tablet_manager.cc:1939] T a73d45bf459c4137af6961b680d282ac P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:20.888607 14555 raft_consensus.cc:3054] T fbd161e290914d9d8671c10becfeaee1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.888895 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet a73d45bf459c4137af6961b680d282ac (table default.Table_70287f2d5a024cc88244993789eb747b [id=88146f846aa74ecfb3977f01ea45df2f]) successfully deleted
I20250115 03:06:20.889806 14390 tablet_service.cc:1514] Processing DeleteTablet for tablet 4122c97f5ba445ee8c79b7d056438229 with delete_type TABLET_DATA_DELETED (replaced by tablet fca14a7d574842ee94a93afb85ca1afd) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.889915 16292 raft_consensus.cc:491] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.889997 16292 raft_consensus.cc:513] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.890175 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet d9ddca2936db4adb92c8ccb34693e24c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.890306 16292 leader_election.cc:290] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:20.890304 14390 tablet_service.cc:1514] Processing DeleteTablet for tablet d9ddca2936db4adb92c8ccb34693e24c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.890378 16292 consensus_queue.cc:1035] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:20.890713 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6fa98c6fb304dc49b8e0ced989cc936" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:20.890759 14819 ts_tablet_manager.cc:1905] T 0792e52d171446b48a7c9d1f03d37a79 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:20.891098 14626 leader_election.cc:343] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:20.892010 14628 leader_election.cc:343] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: c6fa98c6fb304dc49b8e0ced989cc936
I20250115 03:06:20.892077 14628 leader_election.cc:304] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:20.892270 15153 raft_consensus.cc:2743] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.892722 14819 ts_tablet_manager.cc:1918] T 0792e52d171446b48a7c9d1f03d37a79 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.892797 14820 ts_tablet_manager.cc:1905] T 5589b497b0e24740ad49d1392f5a737b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.892894 14819 log.cc:1198] T 0792e52d171446b48a7c9d1f03d37a79 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/0792e52d171446b48a7c9d1f03d37a79
I20250115 03:06:20.893154 14819 ts_tablet_manager.cc:1939] T 0792e52d171446b48a7c9d1f03d37a79 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:20.893702 14819 tablet_replica.cc:331] T 4122c97f5ba445ee8c79b7d056438229 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:20.893868 14819 raft_consensus.cc:2238] T 4122c97f5ba445ee8c79b7d056438229 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.894007 14819 raft_consensus.cc:2267] T 4122c97f5ba445ee8c79b7d056438229 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.895052 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_162fe16a9dfb414784a702bafb5d9938" } new_table_name: "default.Table_920c7cda49a04ab5b030dbccb9fa44d0" modify_external_catalogs: true
I20250115 03:06:20.895186 14820 ts_tablet_manager.cc:1918] T 5589b497b0e24740ad49d1392f5a737b P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.895248 14820 log.cc:1198] T 5589b497b0e24740ad49d1392f5a737b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/5589b497b0e24740ad49d1392f5a737b
I20250115 03:06:20.895432 14820 ts_tablet_manager.cc:1939] T 5589b497b0e24740ad49d1392f5a737b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.895910 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_a8df48231b6f4cdc997f25294c73297c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.896065 16349 negotiation.cc:337] Failed RPC negotiation. Trace:
0115 03:06:20.834854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.186.254:39901 (local address 127.13.186.195:59653)
0115 03:06:20.854956 (+ 20102us) negotiation.cc:107] Waiting for socket to connect
0115 03:06:20.855027 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.186.254:39901: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":20022,"thread_start_us":77,"threads_started":1}
W20250115 03:06:20.896205 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a8df48231b6f4cdc997f25294c73297c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.896224 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_eca9fde960bd4324a474bbda7f6e3317" } new_table_name: "default.Table_84135e72d32d48e5820645a77dc85f72" modify_external_catalogs: true
I20250115 03:06:20.897089 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0c35f2414d284756b9c93ed61e75d451" } new_table_name: "default.Table_3984fd35e6814fd4901d3894b30d7a91" modify_external_catalogs: true
I20250115 03:06:20.897486 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 5589b497b0e24740ad49d1392f5a737b (table default.Table_8daa0abab4d1464488bec21bb7ad375b [id=37d965d8a5eb408ea3f33313bd22c152]) successfully deleted
I20250115 03:06:20.898038 14819 ts_tablet_manager.cc:1905] T 4122c97f5ba445ee8c79b7d056438229 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.898149 14812 tablet_bootstrap.cc:492] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.898221 14812 ts_tablet_manager.cc:1397] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.001s
I20250115 03:06:20.898864 14812 raft_consensus.cc:357] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.898962 14812 raft_consensus.cc:383] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.899000 14812 raft_consensus.cc:738] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.899132 14812 consensus_queue.cc:260] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.899155 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_fa0e824291a14ef5982d469c2a7fe69c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.899420 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fa0e824291a14ef5982d469c2a7fe69c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.900197 14819 ts_tablet_manager.cc:1918] T 4122c97f5ba445ee8c79b7d056438229 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.900350 14823 tablet_replica.cc:331] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:20.900573 14812 ts_tablet_manager.cc:1428] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20250115 03:06:20.900739 14812 tablet_bootstrap.cc:492] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.901684 14812 tablet_bootstrap.cc:654] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.902243 14823 raft_consensus.cc:2238] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.902390 14823 raft_consensus.cc:2267] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:20.903271 14171 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5076e95bfbac445899cb8ae88b49969a consensus state has no leader
I20250115 03:06:20.903677 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet d9ddca2936db4adb92c8ccb34693e24c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.903759 14819 log.cc:1198] T 4122c97f5ba445ee8c79b7d056438229 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/4122c97f5ba445ee8c79b7d056438229
I20250115 03:06:20.904038 14819 ts_tablet_manager.cc:1939] T 4122c97f5ba445ee8c79b7d056438229 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:20.904340 14823 ts_tablet_manager.cc:1905] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:20.905035 14157 consensus_peers.cc:487] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 -> Peer 4225fe2fbfa84937bb41fed702ca66e4 (127.13.186.254:39901): Couldn't send request to peer 4225fe2fbfa84937bb41fed702ca66e4. Status: Network error: Client connection negotiation failed: client connection to 127.13.186.254:39901: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250115 03:06:20.906788 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet d9ddca2936db4adb92c8ccb34693e24c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9ddca2936db4adb92c8ccb34693e24c already in progress: creating tablet
I20250115 03:06:20.907121 14823 ts_tablet_manager.cc:1918] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.907193 14823 log.cc:1198] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/c6fa98c6fb304dc49b8e0ced989cc936
I20250115 03:06:20.907200 14811 tablet_bootstrap.cc:492] T 85b4eb3b20bb4908884e5716e7b2a369 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.907284 14811 ts_tablet_manager.cc:1397] T 85b4eb3b20bb4908884e5716e7b2a369 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.050s user 0.002s sys 0.000s
I20250115 03:06:20.907373 14823 ts_tablet_manager.cc:1939] T c6fa98c6fb304dc49b8e0ced989cc936 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:20.907656 14811 raft_consensus.cc:357] T 85b4eb3b20bb4908884e5716e7b2a369 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.907761 14811 raft_consensus.cc:383] T 85b4eb3b20bb4908884e5716e7b2a369 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.907793 14811 raft_consensus.cc:738] T 85b4eb3b20bb4908884e5716e7b2a369 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.908025 14811 consensus_queue.cc:260] T 85b4eb3b20bb4908884e5716e7b2a369 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.908388 14811 ts_tablet_manager.cc:1428] T 85b4eb3b20bb4908884e5716e7b2a369 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.910640 14162 master_service.cc:946] ReplaceTablet: received request to replace tablet 015ec42caa1b468ea10f25c8f26e278a from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:20.911309 16088 consensus_queue.cc:1035] T fbd161e290914d9d8671c10becfeaee1 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:20.911981 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 91edfeb70a964909bf9049aa1c9fd8a5 consensus state has no leader
W20250115 03:06:20.912432 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9eb56ebdb1a24b8ea8fbc4c129fded95 consensus state has no leader
W20250115 03:06:20.914532 14167 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 578506b0528442cf834a8e65a400c635 consensus state has no leader
I20250115 03:06:20.915177 14170 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_57fbe77148284b4a9666630570f69321 [id=c9ed0b9c92ef40efa5c51cfdd8389325] alter complete (version 2)
I20250115 03:06:20.915577 14812 tablet_bootstrap.cc:492] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.915704 14812 ts_tablet_manager.cc:1397] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20250115 03:06:20.916168 14812 raft_consensus.cc:357] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.916317 14812 raft_consensus.cc:383] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.916381 14812 raft_consensus.cc:738] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.916548 14812 consensus_queue.cc:260] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.916973 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 0792e52d171446b48a7c9d1f03d37a79 (table default.Table_72e82b3e776e4d398fa5d8240afde936 [id=17ae925a7c0942d59058b2fd619ccb45]) successfully deleted
I20250115 03:06:20.917520 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet c6fa98c6fb304dc49b8e0ced989cc936 (table default.Table_9f442bb6e9024025b7fa97a7ddd069f5 [id=8f805d745ece4ec8845a43798122dd4f]) successfully deleted
I20250115 03:06:20.917502 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 73daf2b3a443499d860187ee3c937480 (DEFAULT_TABLE table=default.Table_08cd4bc70fd145d89e52bec66a069523 [id=e956ba9904d64ea7b782a9f60fcf52e6]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:20.917645 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet d9ddca2936db4adb92c8ccb34693e24c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9ddca2936db4adb92c8ccb34693e24c already in progress: creating tablet
I20250115 03:06:20.917745 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 4122c97f5ba445ee8c79b7d056438229 (table default.Table_b169e386c2cc424da59a39503cec548f [id=2b97c2f5b5284cd8a6311247712cc447]) successfully deleted
W20250115 03:06:20.917847 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet d9ddca2936db4adb92c8ccb34693e24c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9ddca2936db4adb92c8ccb34693e24c already in progress: creating tablet
I20250115 03:06:20.918644 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73daf2b3a443499d860187ee3c937480. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.920948 15401 raft_consensus.cc:491] T 0f20bb66f36647e0811d0865b8f414f7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.921025 15401 raft_consensus.cc:513] T 0f20bb66f36647e0811d0865b8f414f7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.921316 15401 leader_election.cc:290] T 0f20bb66f36647e0811d0865b8f414f7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:20.921707 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f20bb66f36647e0811d0865b8f414f7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:20.921991 14356 leader_election.cc:343] T 0f20bb66f36647e0811d0865b8f414f7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.922246 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f20bb66f36647e0811d0865b8f414f7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:20.922401 14811 tablet_bootstrap.cc:492] T c6fa98c6fb304dc49b8e0ced989cc936 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.923321 14811 tablet_bootstrap.cc:654] T c6fa98c6fb304dc49b8e0ced989cc936 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:20.923703 14358 leader_election.cc:343] T 0f20bb66f36647e0811d0865b8f414f7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.923763 14358 leader_election.cc:304] T 0f20bb66f36647e0811d0865b8f414f7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:20.923931 15401 raft_consensus.cc:2743] T 0f20bb66f36647e0811d0865b8f414f7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.924388 15401 raft_consensus.cc:491] T 7043616bd6b34299862336924a6f16bf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.924464 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 73daf2b3a443499d860187ee3c937480 (DEFAULT_TABLE table=default.Table_08cd4bc70fd145d89e52bec66a069523 [id=e956ba9904d64ea7b782a9f60fcf52e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.924572 14390 tablet_service.cc:1514] Processing DeleteTablet for tablet d6cf1a0b9c81413b8f72de0cafc76963 with delete_type TABLET_DATA_DELETED (replaced by tablet a9807cb8c17d4b98be44e6fd5dccc150) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.924676 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73daf2b3a443499d860187ee3c937480. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.924705 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet bb68869eaf55477090cf12f40f76a89c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:20.924785 16088 consensus_queue.cc:1035] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:20.925746 14819 tablet_replica.cc:331] T d6cf1a0b9c81413b8f72de0cafc76963 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:20.925858 14819 raft_consensus.cc:2238] T d6cf1a0b9c81413b8f72de0cafc76963 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.925940 14819 raft_consensus.cc:2267] T d6cf1a0b9c81413b8f72de0cafc76963 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.925915 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 73daf2b3a443499d860187ee3c937480 (DEFAULT_TABLE table=default.Table_08cd4bc70fd145d89e52bec66a069523 [id=e956ba9904d64ea7b782a9f60fcf52e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.926131 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73daf2b3a443499d860187ee3c937480. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.927850 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e8b97b3c1ee748159ca7aa6057de8c9f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.928233 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e8b97b3c1ee748159ca7aa6057de8c9f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.924454 15401 raft_consensus.cc:513] T 7043616bd6b34299862336924a6f16bf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.929468 15401 leader_election.cc:290] T 7043616bd6b34299862336924a6f16bf P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:20.929752 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7043616bd6b34299862336924a6f16bf" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:20.930042 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7043616bd6b34299862336924a6f16bf" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:20.930089 14356 leader_election.cc:343] T 7043616bd6b34299862336924a6f16bf P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.930413 14819 ts_tablet_manager.cc:1905] T d6cf1a0b9c81413b8f72de0cafc76963 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:20.930537 14358 leader_election.cc:343] T 7043616bd6b34299862336924a6f16bf P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.930603 14358 leader_election.cc:304] T 7043616bd6b34299862336924a6f16bf P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:20.930784 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_c265f3104dd946e7bf35035e46c2c029"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:20.930907 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_9fb551f8ee5c451392313c8f1253f5ec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.931061 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c265f3104dd946e7bf35035e46c2c029 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:20.931207 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9fb551f8ee5c451392313c8f1253f5ec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.931923 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e46e737b4c074b7a92903cf0f6a786ca" } modify_external_catalogs: true
I20250115 03:06:20.932024 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e46e737b4c074b7a92903cf0f6a786ca" } modify_external_catalogs: true
I20250115 03:06:20.932277 15401 raft_consensus.cc:2743] T 7043616bd6b34299862336924a6f16bf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.932524 14819 ts_tablet_manager.cc:1918] T d6cf1a0b9c81413b8f72de0cafc76963 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.932518 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_cf7f3fbd11804e94848a7c5357169a86" } modify_external_catalogs: true
I20250115 03:06:20.932588 14819 log.cc:1198] T d6cf1a0b9c81413b8f72de0cafc76963 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/d6cf1a0b9c81413b8f72de0cafc76963
I20250115 03:06:20.932610 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_cf7f3fbd11804e94848a7c5357169a86" } modify_external_catalogs: true
I20250115 03:06:20.932780 14819 ts_tablet_manager.cc:1939] T d6cf1a0b9c81413b8f72de0cafc76963 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:20.934645 14819 tablet_replica.cc:331] T bb68869eaf55477090cf12f40f76a89c P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:20.934804 14819 raft_consensus.cc:2238] T bb68869eaf55477090cf12f40f76a89c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.934922 14819 raft_consensus.cc:2267] T bb68869eaf55477090cf12f40f76a89c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.935482 14812 ts_tablet_manager.cc:1428] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.935830 14809 tablet_bootstrap.cc:492] T b2629ae9d9854ae4b6e7ddddbb6f341b P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.935847 14812 tablet_bootstrap.cc:492] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.935917 14809 ts_tablet_manager.cc:1397] T b2629ae9d9854ae4b6e7ddddbb6f341b P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.059s user 0.002s sys 0.000s
I20250115 03:06:20.936376 14809 raft_consensus.cc:357] T b2629ae9d9854ae4b6e7ddddbb6f341b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.936491 14809 raft_consensus.cc:383] T b2629ae9d9854ae4b6e7ddddbb6f341b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.936522 14809 raft_consensus.cc:738] T b2629ae9d9854ae4b6e7ddddbb6f341b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.936635 14809 consensus_queue.cc:260] T b2629ae9d9854ae4b6e7ddddbb6f341b P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.936874 14812 tablet_bootstrap.cc:654] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.936908 14809 ts_tablet_manager.cc:1428] T b2629ae9d9854ae4b6e7ddddbb6f341b P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.937139 14809 tablet_bootstrap.cc:492] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.938077 14809 tablet_bootstrap.cc:654] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.941852 14819 ts_tablet_manager.cc:1905] T bb68869eaf55477090cf12f40f76a89c P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.943161 14811 tablet_bootstrap.cc:492] T c6fa98c6fb304dc49b8e0ced989cc936 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.943297 14811 ts_tablet_manager.cc:1397] T c6fa98c6fb304dc49b8e0ced989cc936 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.000s
I20250115 03:06:20.943591 14819 ts_tablet_manager.cc:1918] T bb68869eaf55477090cf12f40f76a89c P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.943675 14819 log.cc:1198] T bb68869eaf55477090cf12f40f76a89c P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/bb68869eaf55477090cf12f40f76a89c
I20250115 03:06:20.943871 14819 ts_tablet_manager.cc:1939] T bb68869eaf55477090cf12f40f76a89c P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:20.943867 14811 raft_consensus.cc:357] T c6fa98c6fb304dc49b8e0ced989cc936 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.944052 14811 raft_consensus.cc:383] T c6fa98c6fb304dc49b8e0ced989cc936 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.944113 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet d6cf1a0b9c81413b8f72de0cafc76963 (table default.Table_e51b0b04bb044b36be347afe90845b14 [id=17ea0c263011486f82895eb86b21fe78]) successfully deleted
I20250115 03:06:20.944128 14811 raft_consensus.cc:738] T c6fa98c6fb304dc49b8e0ced989cc936 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.944350 14811 consensus_queue.cc:260] T c6fa98c6fb304dc49b8e0ced989cc936 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.945983 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet bb68869eaf55477090cf12f40f76a89c (table default.Table_0964201dda974eaabef3d5694c40389c [id=4a6fefd2006a4b11b121bfad61cf95ed]) successfully deleted
I20250115 03:06:20.946436 15617 tablet.cc:1721] T e4f688e4c40d4653929432b13508348a P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:20.946990 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet 0aeb6e439d574790a1554f6ff79079c6 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:20.947569 14811 ts_tablet_manager.cc:1428] T c6fa98c6fb304dc49b8e0ced989cc936 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:20.947782 14811 tablet_bootstrap.cc:492] T 9febec144bb343a8a21817e6d3655ad7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.948395 15970 tablet.cc:1721] T e4f688e4c40d4653929432b13508348a P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:20.948634 14811 tablet_bootstrap.cc:654] T 9febec144bb343a8a21817e6d3655ad7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.949031 16265 tablet.cc:1721] T e4f688e4c40d4653929432b13508348a P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:20.949926 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet c6fa98c6fb304dc49b8e0ced989cc936 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.951333 14820 tablet_replica.cc:331] T c6fa98c6fb304dc49b8e0ced989cc936 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:20.951462 14820 raft_consensus.cc:2238] T c6fa98c6fb304dc49b8e0ced989cc936 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.951551 14820 raft_consensus.cc:2267] T c6fa98c6fb304dc49b8e0ced989cc936 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.951956 14820 ts_tablet_manager.cc:1905] T c6fa98c6fb304dc49b8e0ced989cc936 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.953984 14820 ts_tablet_manager.cc:1918] T c6fa98c6fb304dc49b8e0ced989cc936 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.954054 14820 log.cc:1198] T c6fa98c6fb304dc49b8e0ced989cc936 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/c6fa98c6fb304dc49b8e0ced989cc936
I20250115 03:06:20.954231 14820 ts_tablet_manager.cc:1939] T c6fa98c6fb304dc49b8e0ced989cc936 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:20.954993 14811 tablet_bootstrap.cc:492] T 9febec144bb343a8a21817e6d3655ad7 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.955104 14811 ts_tablet_manager.cc:1397] T 9febec144bb343a8a21817e6d3655ad7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20250115 03:06:20.955613 14811 raft_consensus.cc:357] T 9febec144bb343a8a21817e6d3655ad7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.955745 14811 raft_consensus.cc:383] T 9febec144bb343a8a21817e6d3655ad7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.955794 14811 raft_consensus.cc:738] T 9febec144bb343a8a21817e6d3655ad7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.955909 14811 consensus_queue.cc:260] T 9febec144bb343a8a21817e6d3655ad7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.956187 16088 raft_consensus.cc:491] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.956282 14811 ts_tablet_manager.cc:1428] T 9febec144bb343a8a21817e6d3655ad7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:20.956282 16088 raft_consensus.cc:513] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.956667 16088 leader_election.cc:290] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:20.957034 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66b748fc853f499fafc1f476c8f79b8b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:20.957499 14626 leader_election.cc:343] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.957811 14811 tablet_bootstrap.cc:492] T 2c69daff62144976ae9082c89cc58eca P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.959534 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet c6fa98c6fb304dc49b8e0ced989cc936 (table default.Table_9f442bb6e9024025b7fa97a7ddd069f5 [id=8f805d745ece4ec8845a43798122dd4f]) successfully deleted
I20250115 03:06:20.959910 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0a81905ed7fd432ca5ebd5ef48b5f5eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:20.960286 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a81905ed7fd432ca5ebd5ef48b5f5eb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:20.962144 14809 tablet_bootstrap.cc:492] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.962239 14809 ts_tablet_manager.cc:1397] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.000s
I20250115 03:06:20.962605 14812 tablet_bootstrap.cc:492] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:20.962628 14811 tablet_bootstrap.cc:654] T 2c69daff62144976ae9082c89cc58eca P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.962685 14812 ts_tablet_manager.cc:1397] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.000s
I20250115 03:06:20.962673 14809 raft_consensus.cc:357] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.962783 14809 raft_consensus.cc:383] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.962826 14809 raft_consensus.cc:738] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.962939 14809 consensus_queue.cc:260] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.963097 14812 raft_consensus.cc:357] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.963192 14812 raft_consensus.cc:383] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.963208 14809 ts_tablet_manager.cc:1428] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.963222 14812 raft_consensus.cc:738] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.963320 14812 consensus_queue.cc:260] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.963533 14809 tablet_bootstrap.cc:492] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.963575 14812 ts_tablet_manager.cc:1428] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.963727 14812 tablet_bootstrap.cc:492] T b8ec24d2937d44e79b839276bf615fa0 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:20.964406 14809 tablet_bootstrap.cc:654] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.964521 14812 tablet_bootstrap.cc:654] T b8ec24d2937d44e79b839276bf615fa0 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.964963 16001 consensus_queue.cc:1035] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:20.965938 14628 leader_election.cc:343] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 66b748fc853f499fafc1f476c8f79b8b
I20250115 03:06:20.966007 14628 leader_election.cc:304] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:20.966256 16001 raft_consensus.cc:2743] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.968456 14811 tablet_bootstrap.cc:492] T 2c69daff62144976ae9082c89cc58eca P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.968601 14811 ts_tablet_manager.cc:1397] T 2c69daff62144976ae9082c89cc58eca P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.001s
I20250115 03:06:20.969177 14164 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_f36db055f5c24120ab65d6c027d792b7 [id=5beb8c6fadc742c4bd0b5171ed51d0e2] alter complete (version 3)
I20250115 03:06:20.969192 14811 raft_consensus.cc:357] T 2c69daff62144976ae9082c89cc58eca P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.969278 14811 raft_consensus.cc:383] T 2c69daff62144976ae9082c89cc58eca P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.969308 14811 raft_consensus.cc:738] T 2c69daff62144976ae9082c89cc58eca P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.969417 14811 consensus_queue.cc:260] T 2c69daff62144976ae9082c89cc58eca P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.969671 14811 ts_tablet_manager.cc:1428] T 2c69daff62144976ae9082c89cc58eca P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:20.969942 14811 tablet_bootstrap.cc:492] T ada45621f8864caab9189039fae0fae2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:20.970463 16001 consensus_queue.cc:1035] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:20.973249 14811 tablet_bootstrap.cc:654] T ada45621f8864caab9189039fae0fae2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.973613 15390 raft_consensus.cc:491] T 998a515113d04034928c04dc5798a698 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.973685 15390 raft_consensus.cc:513] T 998a515113d04034928c04dc5798a698 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.973986 15390 leader_election.cc:290] T 998a515113d04034928c04dc5798a698 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:20.974206 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "998a515113d04034928c04dc5798a698" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:20.974382 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "998a515113d04034928c04dc5798a698" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:20.974598 14356 leader_election.cc:343] T 998a515113d04034928c04dc5798a698 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.976019 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet b56279bb16054869a97006d88574e381 (table default.Table_23165257844448ce93a97233b487ddc7 [id=080d0a4dfa174d3ba2979bb850d0604a]). Expected version 1 got 0
I20250115 03:06:20.976286 14668 tablet_service.cc:1467] Processing CreateTablet for tablet 660637d2bc474bc6a18a83def9cde185 (DEFAULT_TABLE table=default.Table_72b70810c1714448bba0bbf17f9d559b [id=a8d2d1bb2d32441186605838d7f02069]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.976398 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 15fbf8d5de904fc4b284af664f3bd60f (table default.Table_655e9d02aacf4b88ba521f0f57a5f522 [id=6ed01835736b40f8b4caeca7378770d8]). Expected version 1 got 0
I20250115 03:06:20.976511 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 660637d2bc474bc6a18a83def9cde185. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.976518 14659 tablet_service.cc:1467] Processing CreateTablet for tablet 18310ca8645747a18cafd6d9e647f9e3 (DEFAULT_TABLE table=default.Table_fb75e1b5f16f45dc9e9fc586e358dafb [id=f620e12956364b9db9d46b721915b239]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.977010 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 444a5bef9ab544df913326302867a3a5 (DEFAULT_TABLE table=default.Table_ba02ce2e8d4c4274a1d3f75fc1f8558f [id=b189e18a38b54048921bfddc97551bfb]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:20.977100 14358 leader_election.cc:343] T 998a515113d04034928c04dc5798a698 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:20.977154 14358 leader_election.cc:304] T 998a515113d04034928c04dc5798a698 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:20.977202 14659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18310ca8645747a18cafd6d9e647f9e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.977314 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 444a5bef9ab544df913326302867a3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.977344 15390 raft_consensus.cc:2743] T 998a515113d04034928c04dc5798a698 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.978886 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 9443a671086d47d29e872971461558ee (DEFAULT_TABLE table=default.Table_a8df48231b6f4cdc997f25294c73297c [id=047a004175074346bfcea99719f52a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.979075 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9443a671086d47d29e872971461558ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.979337 16189 raft_consensus.cc:491] T 85b4eb3b20bb4908884e5716e7b2a369 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:20.979416 16189 raft_consensus.cc:513] T 85b4eb3b20bb4908884e5716e7b2a369 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.979753 16189 leader_election.cc:290] T 85b4eb3b20bb4908884e5716e7b2a369 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:20.980173 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b4eb3b20bb4908884e5716e7b2a369" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:20.980252 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b4eb3b20bb4908884e5716e7b2a369" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:20.980304 14419 raft_consensus.cc:2388] T 85b4eb3b20bb4908884e5716e7b2a369 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in current term 1: Already voted for candidate 5c0dae7aff4b44c699275b95ba639770 in this term.
I20250115 03:06:20.980690 14493 leader_election.cc:304] T 85b4eb3b20bb4908884e5716e7b2a369 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:20.980907 16189 raft_consensus.cc:3054] T 85b4eb3b20bb4908884e5716e7b2a369 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.981038 14809 tablet_bootstrap.cc:492] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.981102 14809 ts_tablet_manager.cc:1397] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20250115 03:06:20.981489 14809 raft_consensus.cc:357] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.981611 14809 raft_consensus.cc:383] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.981662 14809 raft_consensus.cc:738] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.981739 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 9febec144bb343a8a21817e6d3655ad7 (table default.Table_57fbe77148284b4a9666630570f69321 [id=c9ed0b9c92ef40efa5c51cfdd8389325]). Expected version 2 got 0
I20250115 03:06:20.981837 16189 raft_consensus.cc:2743] T 85b4eb3b20bb4908884e5716e7b2a369 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:20.981822 14809 consensus_queue.cc:260] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.984091 14390 tablet_service.cc:1467] Processing CreateTablet for tablet 18310ca8645747a18cafd6d9e647f9e3 (DEFAULT_TABLE table=default.Table_fb75e1b5f16f45dc9e9fc586e358dafb [id=f620e12956364b9db9d46b721915b239]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.984148 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 9443a671086d47d29e872971461558ee (DEFAULT_TABLE table=default.Table_a8df48231b6f4cdc997f25294c73297c [id=047a004175074346bfcea99719f52a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.984162 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 660637d2bc474bc6a18a83def9cde185 (DEFAULT_TABLE table=default.Table_72b70810c1714448bba0bbf17f9d559b [id=a8d2d1bb2d32441186605838d7f02069]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.984292 14809 ts_tablet_manager.cc:1428] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:20.984314 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18310ca8645747a18cafd6d9e647f9e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.984658 15153 consensus_queue.cc:1035] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:20.984906 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9443a671086d47d29e872971461558ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.985011 14809 tablet_bootstrap.cc:492] T 8efa998709034fe9bb37c7c608c99e76 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.985801 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet b08e5fc2110e476e897240f9cf3b874d (table default.Table_50c659a42a154d46a26879df0101bf23 [id=f8f2a7ff6fdc40468de6ae9ba28c1427]). Expected version 3 got 0
I20250115 03:06:20.985884 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet b56279bb16054869a97006d88574e381 (table default.Table_23165257844448ce93a97233b487ddc7 [id=080d0a4dfa174d3ba2979bb850d0604a]). Expected version 1 got 0
I20250115 03:06:20.985910 14809 tablet_bootstrap.cc:654] T 8efa998709034fe9bb37c7c608c99e76 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.985989 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet dda418ed197e4af2aa511d3f1851663e (table default.Table_5c360e43ddae405b9a2843fc20a9dfb9 [id=cbab87c958a346199acdad65ab75be3c]). Expected version 3 got 0
W20250115 03:06:20.986655 14163 catalog_manager.cc:5718] Failed to send AlterTable RPC for tablet b08e5fc2110e476e897240f9cf3b874d (table default.Table_50c659a42a154d46a26879df0101bf23 [id=f8f2a7ff6fdc40468de6ae9ba28c1427], current schema version=3): Not found: failed to reset TS proxy: No leader found: Tablet b08e5fc2110e476e897240f9cf3b874d consensus state has no leader
I20250115 03:06:20.987008 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 444a5bef9ab544df913326302867a3a5 (DEFAULT_TABLE table=default.Table_ba02ce2e8d4c4274a1d3f75fc1f8558f [id=b189e18a38b54048921bfddc97551bfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.987210 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 444a5bef9ab544df913326302867a3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.988600 14811 tablet_bootstrap.cc:492] T ada45621f8864caab9189039fae0fae2 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:20.988680 14811 ts_tablet_manager.cc:1397] T ada45621f8864caab9189039fae0fae2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20250115 03:06:20.988824 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet 66b748fc853f499fafc1f476c8f79b8b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:20.989115 14823 tablet_replica.cc:331] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:20.989177 14811 raft_consensus.cc:357] T ada45621f8864caab9189039fae0fae2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.989248 14823 raft_consensus.cc:2238] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:20.989264 14811 raft_consensus.cc:383] T ada45621f8864caab9189039fae0fae2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.989291 14811 raft_consensus.cc:738] T ada45621f8864caab9189039fae0fae2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.989333 14823 raft_consensus.cc:2267] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:20.989899 14823 ts_tablet_manager.cc:1905] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:20.990496 14811 consensus_queue.cc:260] T ada45621f8864caab9189039fae0fae2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.990765 14811 ts_tablet_manager.cc:1428] T ada45621f8864caab9189039fae0fae2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:20.991153 14534 tablet_service.cc:1467] Processing CreateTablet for tablet 660637d2bc474bc6a18a83def9cde185 (DEFAULT_TABLE table=default.Table_72b70810c1714448bba0bbf17f9d559b [id=a8d2d1bb2d32441186605838d7f02069]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.991339 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 18310ca8645747a18cafd6d9e647f9e3 (DEFAULT_TABLE table=default.Table_fb75e1b5f16f45dc9e9fc586e358dafb [id=f620e12956364b9db9d46b721915b239]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.991343 14526 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c69daff62144976ae9082c89cc58eca with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:20.991595 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 444a5bef9ab544df913326302867a3a5 (DEFAULT_TABLE table=default.Table_ba02ce2e8d4c4274a1d3f75fc1f8558f [id=b189e18a38b54048921bfddc97551bfb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.991817 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 9443a671086d47d29e872971461558ee (DEFAULT_TABLE table=default.Table_a8df48231b6f4cdc997f25294c73297c [id=047a004175074346bfcea99719f52a29]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:20.992017 14823 ts_tablet_manager.cc:1918] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:20.992076 14823 log.cc:1198] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/66b748fc853f499fafc1f476c8f79b8b
I20250115 03:06:20.992187 14552 raft_consensus.cc:3054] T b56279bb16054869a97006d88574e381 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:20.992213 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 660637d2bc474bc6a18a83def9cde185. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.992270 14823 ts_tablet_manager.cc:1939] T 66b748fc853f499fafc1f476c8f79b8b P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:20.993014 14809 tablet_bootstrap.cc:492] T 8efa998709034fe9bb37c7c608c99e76 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:20.993088 14809 ts_tablet_manager.cc:1397] T 8efa998709034fe9bb37c7c608c99e76 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20250115 03:06:20.993556 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 660637d2bc474bc6a18a83def9cde185. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.993499 14809 raft_consensus.cc:357] T 8efa998709034fe9bb37c7c608c99e76 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.993625 14809 raft_consensus.cc:383] T 8efa998709034fe9bb37c7c608c99e76 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:20.993677 14809 raft_consensus.cc:738] T 8efa998709034fe9bb37c7c608c99e76 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:20.993786 14809 consensus_queue.cc:260] T 8efa998709034fe9bb37c7c608c99e76 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:20.994045 14809 ts_tablet_manager.cc:1428] T 8efa998709034fe9bb37c7c608c99e76 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:20.994218 14809 tablet_bootstrap.cc:492] T 0e1ce43146ed4e4d82a7c134216e2872 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:20.994524 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 66b748fc853f499fafc1f476c8f79b8b (table default.Table_7eacd8191bf44769b6371caa6d4772f1 [id=ea9e7141223d43de8962caa78ab14914]) successfully deleted
I20250115 03:06:20.994558 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 444a5bef9ab544df913326302867a3a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.997080 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18310ca8645747a18cafd6d9e647f9e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:20.997110 16088 consensus_queue.cc:1035] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:20.997784 14809 tablet_bootstrap.cc:654] T 0e1ce43146ed4e4d82a7c134216e2872 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:20.999109 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 815c5e096e4d4b0cbfb7c15bac571b10
I20250115 03:06:20.999588 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 1fd348b3ae2a45dfa05c52f02536a201
I20250115 03:06:21.000130 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fd348b3ae2a45dfa05c52f02536a201 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.000208 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet 815c5e096e4d4b0cbfb7c15bac571b10 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.000697 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fd348b3ae2a45dfa05c52f02536a201 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:21.000730 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 1fd348b3ae2a45dfa05c52f02536a201 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1fd348b3ae2a45dfa05c52f02536a201 already in progress: creating tablet
W20250115 03:06:21.000874 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 815c5e096e4d4b0cbfb7c15bac571b10 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 815c5e096e4d4b0cbfb7c15bac571b10 already in progress: creating tablet
I20250115 03:06:21.001518 14812 tablet_bootstrap.cc:492] T b8ec24d2937d44e79b839276bf615fa0 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.001597 14812 ts_tablet_manager.cc:1397] T b8ec24d2937d44e79b839276bf615fa0 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.038s user 0.000s sys 0.002s
I20250115 03:06:21.002038 14812 raft_consensus.cc:357] T b8ec24d2937d44e79b839276bf615fa0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.002132 14812 raft_consensus.cc:383] T b8ec24d2937d44e79b839276bf615fa0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.002164 14812 raft_consensus.cc:738] T b8ec24d2937d44e79b839276bf615fa0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.002259 14812 consensus_queue.cc:260] T b8ec24d2937d44e79b839276bf615fa0 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.002547 14812 ts_tablet_manager.cc:1428] T b8ec24d2937d44e79b839276bf615fa0 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.002759 14812 tablet_bootstrap.cc:492] T 85cceb5a3f234d85937ad0bb75348991 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.002839 14162 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet b08e5fc2110e476e897240f9cf3b874d (table default.Table_50c659a42a154d46a26879df0101bf23 [id=f8f2a7ff6fdc40468de6ae9ba28c1427]). Expected version 3 got 0
I20250115 03:06:21.002928 14162 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 8efa998709034fe9bb37c7c608c99e76 (table default.Table_b6002fb5cc1248bba2eb7abbd5a5d407 [id=4c458a80b1a2490da1cf689843cae584]). Expected version 1 got 0
I20250115 03:06:21.002998 14162 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet b56279bb16054869a97006d88574e381 (table default.Table_23165257844448ce93a97233b487ddc7 [id=080d0a4dfa174d3ba2979bb850d0604a]). Expected version 1 got 0
I20250115 03:06:21.003790 14812 tablet_bootstrap.cc:654] T 85cceb5a3f234d85937ad0bb75348991 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.003827 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet b2629ae9d9854ae4b6e7ddddbb6f341b with delete_type TABLET_DATA_DELETED (replaced by tablet 103f78d0d9fc42308e1e23316895be33) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.004065 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 815c5e096e4d4b0cbfb7c15bac571b10 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.004227 14819 tablet_replica.cc:331] T b2629ae9d9854ae4b6e7ddddbb6f341b P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.004357 14819 raft_consensus.cc:2238] T b2629ae9d9854ae4b6e7ddddbb6f341b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.004448 14819 raft_consensus.cc:2267] T b2629ae9d9854ae4b6e7ddddbb6f341b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.005283 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet b56279bb16054869a97006d88574e381 (table default.Table_23165257844448ce93a97233b487ddc7 [id=080d0a4dfa174d3ba2979bb850d0604a]). Expected version 1 got 0
I20250115 03:06:21.005435 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 676ec8b9dc8241d4ae40488d3ce8f61e (table default.Table_0b6cd33c30014981b78899a738418cbf [id=63edd09bfaaa4d56b4314ce760c87d3d]). Expected version 1 got 0
I20250115 03:06:21.005514 14163 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 3bc1f0f72a3b4c79847c0a91fe98ccd1 (table default.Table_b165e5d12ca845b9984c99dde5e2e06e [id=dfe0ca35843c4fc6b975dd1ab87e9a99]). Expected version 1 got 0
I20250115 03:06:21.006004 14819 ts_tablet_manager.cc:1905] T b2629ae9d9854ae4b6e7ddddbb6f341b P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.006405 14858 raft_consensus.cc:491] T 9febec144bb343a8a21817e6d3655ad7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.006487 14858 raft_consensus.cc:513] T 9febec144bb343a8a21817e6d3655ad7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:21.006563 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 1fd348b3ae2a45dfa05c52f02536a201 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1fd348b3ae2a45dfa05c52f02536a201 already in progress: creating tablet
I20250115 03:06:21.006776 14858 leader_election.cc:290] T 9febec144bb343a8a21817e6d3655ad7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.007309 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9febec144bb343a8a21817e6d3655ad7" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.008036 14819 ts_tablet_manager.cc:1918] T b2629ae9d9854ae4b6e7ddddbb6f341b P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.008117 14819 log.cc:1198] T b2629ae9d9854ae4b6e7ddddbb6f341b P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/b2629ae9d9854ae4b6e7ddddbb6f341b
I20250115 03:06:21.008220 14170 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet b56279bb16054869a97006d88574e381 (table default.Table_23165257844448ce93a97233b487ddc7 [id=080d0a4dfa174d3ba2979bb850d0604a]). Expected version 1 got 0
I20250115 03:06:21.008302 14819 ts_tablet_manager.cc:1939] T b2629ae9d9854ae4b6e7ddddbb6f341b P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.009384 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet b2629ae9d9854ae4b6e7ddddbb6f341b (table default.Table_ccd2815995fc485d827b79ae818fedae [id=522c89cadb084de68368a97199ea66db]) successfully deleted
I20250115 03:06:21.009476 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9febec144bb343a8a21817e6d3655ad7" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:21.009518 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 815c5e096e4d4b0cbfb7c15bac571b10 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 815c5e096e4d4b0cbfb7c15bac571b10 already in progress: creating tablet
I20250115 03:06:21.009662 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a222fccf6d3744cab2d0491ce93620be" } modify_external_catalogs: true
I20250115 03:06:21.009764 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a222fccf6d3744cab2d0491ce93620be" } modify_external_catalogs: true
I20250115 03:06:21.011440 14493 leader_election.cc:304] T 9febec144bb343a8a21817e6d3655ad7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:21.011703 16189 raft_consensus.cc:2743] T 9febec144bb343a8a21817e6d3655ad7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.013141 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fd348b3ae2a45dfa05c52f02536a201 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.013257 14527 tablet_service.cc:1514] Processing DeleteTablet for tablet 815c5e096e4d4b0cbfb7c15bac571b10 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.013437 14820 tablet_replica.cc:331] T 2c69daff62144976ae9082c89cc58eca P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:21.013535 14820 raft_consensus.cc:2238] T 2c69daff62144976ae9082c89cc58eca P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.013617 14820 raft_consensus.cc:2267] T 2c69daff62144976ae9082c89cc58eca P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.013949 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9443a671086d47d29e872971461558ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.016713 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 31a04af0754044659a81561d9eddcfa7
I20250115 03:06:21.017472 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 31a04af0754044659a81561d9eddcfa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.017513 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f634ab8b7b7c437f90035122e8f974b9" } modify_external_catalogs: true
I20250115 03:06:21.017602 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f634ab8b7b7c437f90035122e8f974b9" } modify_external_catalogs: true
I20250115 03:06:21.020262 14811 tablet_bootstrap.cc:492] T a73d45bf459c4137af6961b680d282ac P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.021328 14811 tablet_bootstrap.cc:654] T a73d45bf459c4137af6961b680d282ac P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.023103 14820 ts_tablet_manager.cc:1905] T 2c69daff62144976ae9082c89cc58eca P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.023474 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 31a04af0754044659a81561d9eddcfa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.023613 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 31a04af0754044659a81561d9eddcfa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:21.023887 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 31a04af0754044659a81561d9eddcfa7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 31a04af0754044659a81561d9eddcfa7 already in progress: creating tablet
W20250115 03:06:21.024061 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 31a04af0754044659a81561d9eddcfa7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 31a04af0754044659a81561d9eddcfa7 already in progress: creating tablet
I20250115 03:06:21.025007 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet b8ec24d2937d44e79b839276bf615fa0 with delete_type TABLET_DATA_DELETED (replaced by tablet 4febfae2f2794669b5997a1fa7ef7412) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.025147 14820 ts_tablet_manager.cc:1918] T 2c69daff62144976ae9082c89cc58eca P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.025206 14820 log.cc:1198] T 2c69daff62144976ae9082c89cc58eca P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/2c69daff62144976ae9082c89cc58eca
I20250115 03:06:21.025364 14823 tablet_replica.cc:331] T b8ec24d2937d44e79b839276bf615fa0 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:21.025386 14820 ts_tablet_manager.cc:1939] T 2c69daff62144976ae9082c89cc58eca P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:21.025493 14823 raft_consensus.cc:2238] T b8ec24d2937d44e79b839276bf615fa0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.025604 14823 raft_consensus.cc:2267] T b8ec24d2937d44e79b839276bf615fa0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.025651 15390 raft_consensus.cc:491] T 12678da7ffee4a46b4690ac3e94cb19b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.025738 15390 raft_consensus.cc:513] T 12678da7ffee4a46b4690ac3e94cb19b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.026073 15390 leader_election.cc:290] T 12678da7ffee4a46b4690ac3e94cb19b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.026471 14823 ts_tablet_manager.cc:1905] T b8ec24d2937d44e79b839276bf615fa0 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.027477 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12678da7ffee4a46b4690ac3e94cb19b" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:21.027941 14356 leader_election.cc:343] T 12678da7ffee4a46b4690ac3e94cb19b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.028640 14823 ts_tablet_manager.cc:1918] T b8ec24d2937d44e79b839276bf615fa0 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.028708 14823 log.cc:1198] T b8ec24d2937d44e79b839276bf615fa0 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/b8ec24d2937d44e79b839276bf615fa0
I20250115 03:06:21.028890 14823 ts_tablet_manager.cc:1939] T b8ec24d2937d44e79b839276bf615fa0 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:21.029531 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 2c69daff62144976ae9082c89cc58eca (table default.Table_24b2f1f4bbb14d2fa7cba35e089f6707 [id=9c79f5b643664aebbc02a5d815485f61]) successfully deleted
W20250115 03:06:21.029716 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 1fd348b3ae2a45dfa05c52f02536a201 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1fd348b3ae2a45dfa05c52f02536a201 already in progress: creating tablet
I20250115 03:06:21.030344 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet b8ec24d2937d44e79b839276bf615fa0 (table default.Table_83fe43c0616b4351b2b4457195378f0c [id=7bec6149db914c939bc6fd50e47097b2]) successfully deleted
I20250115 03:06:21.030970 16088 raft_consensus.cc:491] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.031029 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12678da7ffee4a46b4690ac3e94cb19b" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.031041 16088 raft_consensus.cc:513] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.031623 16088 leader_election.cc:290] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
W20250115 03:06:21.031643 14358 leader_election.cc:343] T 12678da7ffee4a46b4690ac3e94cb19b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.031715 14358 leader_election.cc:304] T 12678da7ffee4a46b4690ac3e94cb19b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.031997 15390 raft_consensus.cc:2743] T 12678da7ffee4a46b4690ac3e94cb19b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.032063 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b5710268643446b91d04e2f4c7f999e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.032167 14422 raft_consensus.cc:2463] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
W20250115 03:06:21.031149 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 815c5e096e4d4b0cbfb7c15bac571b10 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 815c5e096e4d4b0cbfb7c15bac571b10 already in progress: creating tablet
I20250115 03:06:21.032380 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b5710268643446b91d04e2f4c7f999e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.032541 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet b61cc4b11251486a8089ca9bf44ac91e
I20250115 03:06:21.032747 14628 leader_election.cc:304] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
W20250115 03:06:21.032827 14626 leader_election.cc:343] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.032970 16088 raft_consensus.cc:2798] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.033015 16088 raft_consensus.cc:491] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.033051 16088 raft_consensus.cc:3054] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.033228 15390 raft_consensus.cc:491] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.033295 15390 raft_consensus.cc:513] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.033581 15390 leader_election.cc:290] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.033851 16088 raft_consensus.cc:513] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.034178 16088 leader_election.cc:290] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:21.034415 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b5710268643446b91d04e2f4c7f999e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:21.034866 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4547465f3124e108dc2c4c54e2bc751" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:21.035179 14356 leader_election.cc:343] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.035593 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4547465f3124e108dc2c4c54e2bc751" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.035952 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b5710268643446b91d04e2f4c7f999e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.036047 14422 raft_consensus.cc:3054] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:21.036368 14358 leader_election.cc:343] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:21.036368 14626 leader_election.cc:343] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.036428 14358 leader_election.cc:304] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.036875 15390 raft_consensus.cc:2743] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.036993 14422 raft_consensus.cc:2463] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:21.037523 14628 leader_election.cc:304] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.037813 14555 raft_consensus.cc:3054] T ada45621f8864caab9189039fae0fae2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.038841 16088 raft_consensus.cc:2798] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.038954 16292 consensus_queue.cc:1035] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:21.039268 16088 raft_consensus.cc:695] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:21.039394 16088 consensus_queue.cc:237] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.046272 14812 tablet_bootstrap.cc:492] T 85cceb5a3f234d85937ad0bb75348991 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.046353 14812 ts_tablet_manager.cc:1397] T 85cceb5a3f234d85937ad0bb75348991 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.044s user 0.001s sys 0.001s
I20250115 03:06:21.046797 14812 raft_consensus.cc:357] T 85cceb5a3f234d85937ad0bb75348991 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.046891 14812 raft_consensus.cc:383] T 85cceb5a3f234d85937ad0bb75348991 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.046921 14812 raft_consensus.cc:738] T 85cceb5a3f234d85937ad0bb75348991 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
W20250115 03:06:21.032325 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 31a04af0754044659a81561d9eddcfa7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 31a04af0754044659a81561d9eddcfa7 already in progress: creating tablet
I20250115 03:06:21.047008 14812 consensus_queue.cc:260] T 85cceb5a3f234d85937ad0bb75348991 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.047267 14812 ts_tablet_manager.cc:1428] T 85cceb5a3f234d85937ad0bb75348991 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.047401 16088 raft_consensus.cc:491] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.047430 14812 tablet_bootstrap.cc:492] T 80ede4e9696d491aa385b375fcb5cf3b P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.047472 16088 raft_consensus.cc:513] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.047755 16088 leader_election.cc:290] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:21.047873 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet b61cc4b11251486a8089ca9bf44ac91e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.047879 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet 0e1ce43146ed4e4d82a7c134216e2872 from {username='slave'} at 127.0.0.1:60432
W20250115 03:06:21.048229 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet b61cc4b11251486a8089ca9bf44ac91e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b61cc4b11251486a8089ca9bf44ac91e already in progress: creating tablet
I20250115 03:06:21.048334 14812 tablet_bootstrap.cc:654] T 80ede4e9696d491aa385b375fcb5cf3b P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.048344 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet b61cc4b11251486a8089ca9bf44ac91e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.048863 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676ec8b9dc8241d4ae40488d3ce8f61e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:21.048997 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet b61cc4b11251486a8089ca9bf44ac91e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b61cc4b11251486a8089ca9bf44ac91e already in progress: creating tablet
I20250115 03:06:21.049129 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676ec8b9dc8241d4ae40488d3ce8f61e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.049222 14421 raft_consensus.cc:2463] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:21.049544 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet b61cc4b11251486a8089ca9bf44ac91e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:21.049891 14626 leader_election.cc:343] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:21.050079 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet b61cc4b11251486a8089ca9bf44ac91e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b61cc4b11251486a8089ca9bf44ac91e already in progress: creating tablet
I20250115 03:06:21.051131 14628 leader_election.cc:304] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.052018 14811 tablet_bootstrap.cc:492] T a73d45bf459c4137af6961b680d282ac P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.052098 14811 ts_tablet_manager.cc:1397] T a73d45bf459c4137af6961b680d282ac P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.032s user 0.002s sys 0.000s
I20250115 03:06:21.052322 16292 raft_consensus.cc:2798] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.052371 16292 raft_consensus.cc:491] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.052407 16292 raft_consensus.cc:3054] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.052552 14811 raft_consensus.cc:357] T a73d45bf459c4137af6961b680d282ac P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.052634 14811 raft_consensus.cc:383] T a73d45bf459c4137af6961b680d282ac P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.052659 14811 raft_consensus.cc:738] T a73d45bf459c4137af6961b680d282ac P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.052770 14811 consensus_queue.cc:260] T a73d45bf459c4137af6961b680d282ac P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.053289 16292 raft_consensus.cc:513] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.053351 14811 ts_tablet_manager.cc:1428] T a73d45bf459c4137af6961b680d282ac P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.053606 16292 leader_election.cc:290] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:21.054410 14811 tablet_bootstrap.cc:492] T adbb0ae719f24dcc96709a83981e213d P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.055315 14811 tablet_bootstrap.cc:654] T adbb0ae719f24dcc96709a83981e213d P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.056049 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676ec8b9dc8241d4ae40488d3ce8f61e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:21.056202 14553 raft_consensus.cc:3054] T 6bcfde5fe3d4425e98edb4dc54096718 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:21.056452 14626 leader_election.cc:343] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.058032 14162 master_service.cc:946] ReplaceTablet: received request to replace tablet 0ba157d911a145a2a665eed1ec4fc08b from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:21.058028 16265 tablet.cc:1721] T 6bcfde5fe3d4425e98edb4dc54096718 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:21.058413 14811 tablet_bootstrap.cc:492] T adbb0ae719f24dcc96709a83981e213d P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.058476 14811 ts_tablet_manager.cc:1397] T adbb0ae719f24dcc96709a83981e213d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:21.058836 14811 raft_consensus.cc:357] T adbb0ae719f24dcc96709a83981e213d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.058923 14811 raft_consensus.cc:383] T adbb0ae719f24dcc96709a83981e213d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.058949 14811 raft_consensus.cc:738] T adbb0ae719f24dcc96709a83981e213d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.059047 14811 consensus_queue.cc:260] T adbb0ae719f24dcc96709a83981e213d P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.059267 14811 ts_tablet_manager.cc:1428] T adbb0ae719f24dcc96709a83981e213d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.059408 14811 tablet_bootstrap.cc:492] T bfc34dc76ff94b8eaeb5c2822e6498da P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.060225 14811 tablet_bootstrap.cc:654] T bfc34dc76ff94b8eaeb5c2822e6498da P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.061045 15390 raft_consensus.cc:491] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.061125 15390 raft_consensus.cc:513] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.061448 15390 leader_election.cc:290] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.062039 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfeafbf380df41738ebbcc11c19a7f85" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.064757 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 1a5f62a594ff456eab1a84082068ca68 (DEFAULT_TABLE table=default.Table_0a81905ed7fd432ca5ebd5ef48b5f5eb [id=5c6028aefc20450c9003099409d1aee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.064994 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a5f62a594ff456eab1a84082068ca68. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.067770 14668 tablet_service.cc:1467] Processing CreateTablet for tablet 1a5f62a594ff456eab1a84082068ca68 (DEFAULT_TABLE table=default.Table_0a81905ed7fd432ca5ebd5ef48b5f5eb [id=5c6028aefc20450c9003099409d1aee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.067819 14653 tablet_service.cc:1467] Processing CreateTablet for tablet bef3ccbd409349dba4d3f434473a164d (DEFAULT_TABLE table=default.Table_9fb551f8ee5c451392313c8f1253f5ec [id=adfa3c45c4e74c28827002150637458e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.068012 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 0ba157d911a145a2a665eed1ec4fc08b
I20250115 03:06:21.068039 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a5f62a594ff456eab1a84082068ca68. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.068253 14162 catalog_manager.cc:6596] ReplaceTablet: tablet 0ba157d911a145a2a665eed1ec4fc08b deleted and replaced by tablet e7fb6422afb44ec88db538b74ce74175
I20250115 03:06:21.068264 14399 tablet_service.cc:1467] Processing CreateTablet for tablet bef3ccbd409349dba4d3f434473a164d (DEFAULT_TABLE table=default.Table_9fb551f8ee5c451392313c8f1253f5ec [id=adfa3c45c4e74c28827002150637458e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.068508 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bef3ccbd409349dba4d3f434473a164d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.069074 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bef3ccbd409349dba4d3f434473a164d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.069283 14536 tablet_service.cc:1467] Processing CreateTablet for tablet bef3ccbd409349dba4d3f434473a164d (DEFAULT_TABLE table=default.Table_9fb551f8ee5c451392313c8f1253f5ec [id=adfa3c45c4e74c28827002150637458e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.069499 14526 tablet_service.cc:1467] Processing CreateTablet for tablet de552db1a01443589d95ba8993ef6f06 (DEFAULT_TABLE table=default.Table_fa0e824291a14ef5982d469c2a7fe69c [id=896a0243199e49bdbe6d9169e65f96fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.069617 14534 tablet_service.cc:1467] Processing CreateTablet for tablet e0c864f3ff1c4ce8a07d1b863bf88694 (DEFAULT_TABLE table=default.Table_c265f3104dd946e7bf35035e46c2c029 [id=8de55bf52727418eacaf0129b83036cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.069932 14527 tablet_service.cc:1467] Processing CreateTablet for tablet ea70a042400e4c95bdbc071255ce7d8d (DEFAULT_TABLE table=default.Table_e8b97b3c1ee748159ca7aa6057de8c9f [id=1bb835dda77a4c9784d02e6f9e902bc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.070142 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ba157d911a145a2a665eed1ec4fc08b with delete_type TABLET_DATA_DELETED (replaced by tablet e7fb6422afb44ec88db538b74ce74175) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.070556 16189 raft_consensus.cc:491] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.070642 16189 raft_consensus.cc:513] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.070719 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 1a5f62a594ff456eab1a84082068ca68 (DEFAULT_TABLE table=default.Table_0a81905ed7fd432ca5ebd5ef48b5f5eb [id=5c6028aefc20450c9003099409d1aee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.070917 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a5f62a594ff456eab1a84082068ca68. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.070956 16189 leader_election.cc:290] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:21.071189 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 0ba157d911a145a2a665eed1ec4fc08b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0ba157d911a145a2a665eed1ec4fc08b already in progress: creating tablet
I20250115 03:06:21.071419 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0521973e23984355a77a321b4676b479" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.071514 14689 raft_consensus.cc:2463] T 0521973e23984355a77a321b4676b479 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.071816 14493 leader_election.cc:304] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.071981 16189 raft_consensus.cc:2798] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.072041 16189 raft_consensus.cc:491] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.072077 16189 raft_consensus.cc:3054] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.072649 14809 tablet_bootstrap.cc:492] T 0e1ce43146ed4e4d82a7c134216e2872 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.072726 14809 ts_tablet_manager.cc:1397] T 0e1ce43146ed4e4d82a7c134216e2872 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.079s user 0.002s sys 0.000s
I20250115 03:06:21.072849 16189 raft_consensus.cc:513] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.073158 14809 raft_consensus.cc:357] T 0e1ce43146ed4e4d82a7c134216e2872 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.073216 16189 leader_election.cc:290] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.073243 14809 raft_consensus.cc:383] T 0e1ce43146ed4e4d82a7c134216e2872 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.073272 14809 raft_consensus.cc:738] T 0e1ce43146ed4e4d82a7c134216e2872 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.073395 14809 consensus_queue.cc:260] T 0e1ce43146ed4e4d82a7c134216e2872 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.073673 14809 ts_tablet_manager.cc:1428] T 0e1ce43146ed4e4d82a7c134216e2872 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.073676 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0521973e23984355a77a321b4676b479" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:21.073769 14526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de552db1a01443589d95ba8993ef6f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.073784 14689 raft_consensus.cc:3054] T 0521973e23984355a77a321b4676b479 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.073839 14809 tablet_bootstrap.cc:492] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.073985 14670 tablet_service.cc:1467] Processing CreateTablet for tablet de552db1a01443589d95ba8993ef6f06 (DEFAULT_TABLE table=default.Table_fa0e824291a14ef5982d469c2a7fe69c [id=896a0243199e49bdbe6d9169e65f96fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.074175 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de552db1a01443589d95ba8993ef6f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.074569 14689 raft_consensus.cc:2463] T 0521973e23984355a77a321b4676b479 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.074692 14809 tablet_bootstrap.cc:654] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.075126 14659 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ba157d911a145a2a665eed1ec4fc08b with delete_type TABLET_DATA_DELETED (replaced by tablet e7fb6422afb44ec88db538b74ce74175) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.076359 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ba157d911a145a2a665eed1ec4fc08b with delete_type TABLET_DATA_DELETED (replaced by tablet e7fb6422afb44ec88db538b74ce74175) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:21.076738 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 0ba157d911a145a2a665eed1ec4fc08b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0ba157d911a145a2a665eed1ec4fc08b already in progress: creating tablet
I20250115 03:06:21.076769 14390 tablet_service.cc:1467] Processing CreateTablet for tablet de552db1a01443589d95ba8993ef6f06 (DEFAULT_TABLE table=default.Table_fa0e824291a14ef5982d469c2a7fe69c [id=896a0243199e49bdbe6d9169e65f96fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.076886 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea70a042400e4c95bdbc071255ce7d8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.077008 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de552db1a01443589d95ba8993ef6f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.077478 14399 tablet_service.cc:1467] Processing CreateTablet for tablet ea70a042400e4c95bdbc071255ce7d8d (DEFAULT_TABLE table=default.Table_e8b97b3c1ee748159ca7aa6057de8c9f [id=1bb835dda77a4c9784d02e6f9e902bc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.077674 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea70a042400e4c95bdbc071255ce7d8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.078753 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0c864f3ff1c4ce8a07d1b863bf88694. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.079058 14653 tablet_service.cc:1467] Processing CreateTablet for tablet ea70a042400e4c95bdbc071255ce7d8d (DEFAULT_TABLE table=default.Table_e8b97b3c1ee748159ca7aa6057de8c9f [id=1bb835dda77a4c9784d02e6f9e902bc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.079239 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea70a042400e4c95bdbc071255ce7d8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.079634 14397 tablet_service.cc:1467] Processing CreateTablet for tablet e0c864f3ff1c4ce8a07d1b863bf88694 (DEFAULT_TABLE table=default.Table_c265f3104dd946e7bf35035e46c2c029 [id=8de55bf52727418eacaf0129b83036cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.079829 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0c864f3ff1c4ce8a07d1b863bf88694. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.080359 14668 tablet_service.cc:1467] Processing CreateTablet for tablet e0c864f3ff1c4ce8a07d1b863bf88694 (DEFAULT_TABLE table=default.Table_c265f3104dd946e7bf35035e46c2c029 [id=8de55bf52727418eacaf0129b83036cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.080554 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0c864f3ff1c4ce8a07d1b863bf88694. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.080857 14493 leader_election.cc:304] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.081198 14858 raft_consensus.cc:2798] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.081276 14858 raft_consensus.cc:695] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:21.081387 14858 consensus_queue.cc:237] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.081902 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0521973e23984355a77a321b4676b479" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
W20250115 03:06:21.081956 14356 leader_election.cc:343] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.082000 14419 raft_consensus.cc:2463] T 0521973e23984355a77a321b4676b479 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.082041 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0521973e23984355a77a321b4676b479" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.082126 14422 raft_consensus.cc:3054] T 0521973e23984355a77a321b4676b479 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.082949 14422 raft_consensus.cc:2463] T 0521973e23984355a77a321b4676b479 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.083389 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_06955d84b4e94e5a82ba9e8402877f30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.083762 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_06955d84b4e94e5a82ba9e8402877f30 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.085754 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bef3ccbd409349dba4d3f434473a164d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.088354 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfeafbf380df41738ebbcc11c19a7f85" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.088884 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_da8cef7d7d1b4784aa3d21e269d7c71b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:21.089160 15390 raft_consensus.cc:491] T 7b86e66428504217af08f667f660d304 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20250115 03:06:21.089223 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_da8cef7d7d1b4784aa3d21e269d7c71b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.089237 15390 raft_consensus.cc:513] T 7b86e66428504217af08f667f660d304 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.089421 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5e83ef9eef624e33af51363bb4ed5920" } modify_external_catalogs: true
I20250115 03:06:21.089541 15390 leader_election.cc:290] T 7b86e66428504217af08f667f660d304 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.089665 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676ec8b9dc8241d4ae40488d3ce8f61e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.089648 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_e1daabdd4fac414bb329cb99820de56c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:21.089766 14422 raft_consensus.cc:3054] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:21.089934 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e1daabdd4fac414bb329cb99820de56c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.090188 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b86e66428504217af08f667f660d304" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:21.090543 14356 leader_election.cc:343] T 7b86e66428504217af08f667f660d304 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.090569 14422 raft_consensus.cc:2463] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
W20250115 03:06:21.091437 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 0ba157d911a145a2a665eed1ec4fc08b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0ba157d911a145a2a665eed1ec4fc08b already in progress: creating tablet
W20250115 03:06:21.093087 14358 leader_election.cc:343] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.093152 14358 leader_election.cc:304] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.093382 15401 raft_consensus.cc:2743] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.093912 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5e83ef9eef624e33af51363bb4ed5920" } modify_external_catalogs: true
I20250115 03:06:21.094267 14555 raft_consensus.cc:3054] T 9febec144bb343a8a21817e6d3655ad7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.095993 14628 leader_election.cc:304] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.096263 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1ce43146ed4e4d82a7c134216e2872 with delete_type TABLET_DATA_DELETED (replaced by tablet 578506b0528442cf834a8e65a400c635) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.097083 14819 tablet_replica.cc:331] T 0e1ce43146ed4e4d82a7c134216e2872 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.097482 14819 raft_consensus.cc:2238] T 0e1ce43146ed4e4d82a7c134216e2872 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.097642 14819 raft_consensus.cc:2267] T 0e1ce43146ed4e4d82a7c134216e2872 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.097270 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b86e66428504217af08f667f660d304" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:21.098222 14358 leader_election.cc:343] T 7b86e66428504217af08f667f660d304 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.098285 14358 leader_election.cc:304] T 7b86e66428504217af08f667f660d304 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.098507 15401 raft_consensus.cc:2743] T 7b86e66428504217af08f667f660d304 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.099146 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet a73d45bf459c4137af6961b680d282ac with delete_type TABLET_DATA_DELETED (replaced by tablet 15263a3b32be418288770ce348c082d7) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.099282 16088 raft_consensus.cc:2798] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.099298 14820 tablet_replica.cc:331] T a73d45bf459c4137af6961b680d282ac P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:21.099414 16088 raft_consensus.cc:695] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:21.099445 14820 raft_consensus.cc:2238] T a73d45bf459c4137af6961b680d282ac P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.099548 14820 raft_consensus.cc:2267] T a73d45bf459c4137af6961b680d282ac P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.099869 16088 consensus_queue.cc:237] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.099633 14170 catalog_manager.cc:5526] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:21.100875 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_1bc4f5e4e8a94919b6fc72560065d6af"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.101190 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1bc4f5e4e8a94919b6fc72560065d6af in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.102377 14820 ts_tablet_manager.cc:1905] T a73d45bf459c4137af6961b680d282ac P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.104110 16265 tablet.cc:1721] T 9febec144bb343a8a21817e6d3655ad7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:21.104502 14820 ts_tablet_manager.cc:1918] T a73d45bf459c4137af6961b680d282ac P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.104576 14820 log.cc:1198] T a73d45bf459c4137af6961b680d282ac P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/a73d45bf459c4137af6961b680d282ac
I20250115 03:06:21.104789 14820 ts_tablet_manager.cc:1939] T a73d45bf459c4137af6961b680d282ac P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:21.105551 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet a73d45bf459c4137af6961b680d282ac (table default.Table_70287f2d5a024cc88244993789eb747b [id=88146f846aa74ecfb3977f01ea45df2f]) successfully deleted
I20250115 03:06:21.109220 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 60fd0318f85842818e0b60d5fef60400
I20250115 03:06:21.109755 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 60fd0318f85842818e0b60d5fef60400 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.109759 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet 60fd0318f85842818e0b60d5fef60400 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.109818 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 60fd0318f85842818e0b60d5fef60400 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.109949 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_bfb8ef470a8a43e399bf8541e2171dc4" } new_table_name: "default.Table_1ae3ab02b9eb4ac98b3f5ea457bab6d1" modify_external_catalogs: true
I20250115 03:06:21.110301 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_1fd65820535a4dc7a4fc884c4184bba4" } new_table_name: "default.Table_3e0a77ba78e7413588e3b4c24cfd0de4" modify_external_catalogs: true
W20250115 03:06:21.110617 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 60fd0318f85842818e0b60d5fef60400 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60fd0318f85842818e0b60d5fef60400 already in progress: creating tablet
W20250115 03:06:21.110924 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 60fd0318f85842818e0b60d5fef60400 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60fd0318f85842818e0b60d5fef60400 already in progress: creating tablet
I20250115 03:06:21.111894 14819 ts_tablet_manager.cc:1905] T 0e1ce43146ed4e4d82a7c134216e2872 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.112639 14809 tablet_bootstrap.cc:492] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.112720 14809 ts_tablet_manager.cc:1397] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.039s user 0.001s sys 0.000s
I20250115 03:06:21.113183 14809 raft_consensus.cc:357] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.113274 14809 raft_consensus.cc:383] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.113304 14809 raft_consensus.cc:738] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.113407 14809 consensus_queue.cc:260] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.113634 14809 ts_tablet_manager.cc:1428] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.113799 14809 tablet_bootstrap.cc:492] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.113952 14819 ts_tablet_manager.cc:1918] T 0e1ce43146ed4e4d82a7c134216e2872 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.114007 14819 log.cc:1198] T 0e1ce43146ed4e4d82a7c134216e2872 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/0e1ce43146ed4e4d82a7c134216e2872
I20250115 03:06:21.114176 14819 ts_tablet_manager.cc:1939] T 0e1ce43146ed4e4d82a7c134216e2872 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.114677 14809 tablet_bootstrap.cc:654] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.115991 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 0e1ce43146ed4e4d82a7c134216e2872 (table default.Table_3984fd35e6814fd4901d3894b30d7a91 [id=5775da4995774af39bfc049e48eaf2f0]) successfully deleted
W20250115 03:06:21.116144 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 60fd0318f85842818e0b60d5fef60400 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60fd0318f85842818e0b60d5fef60400 already in progress: creating tablet
I20250115 03:06:21.117009 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_691c78b3563f49c9a7957471bc430389" } new_table_name: "default.Table_dc9387b162a648c0b679957b28558f0d" modify_external_catalogs: true
I20250115 03:06:21.118585 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 72abf6acd9d54f3b983b7098aaf2691d (table default.Table_5e72c1c96c004205bf7304be210f76f1 [id=5ca0a4e7179649d3870fdadc05eaa427]). Expected version 1 got 0
I20250115 03:06:21.121063 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f3e6282c66774e58a910ea88782b43ea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.121338 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f3e6282c66774e58a910ea88782b43ea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:21.125137 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 75854bea6dec40ac9f205fec0ae66f39 consensus state has no leader
W20250115 03:06:21.125705 14165 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet af712cc03750455087ecd381cc97d464 consensus state has no leader
I20250115 03:06:21.126307 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_af3419632bb0478bab77f1fa32e9fb52" } new_table_name: "default.Table_2d473b82a1764ef6afd586a7bf8e17dc" modify_external_catalogs: true
W20250115 03:06:21.126495 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 83a4f95eb9494a5295a81a8a51f58bd3 consensus state has no leader
I20250115 03:06:21.126979 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_50c659a42a154d46a26879df0101bf23" } new_table_name: "default.Table_6b6164bc5f2e4395b5d31047b8ef87d9" modify_external_catalogs: true
I20250115 03:06:21.128362 14811 tablet_bootstrap.cc:492] T bfc34dc76ff94b8eaeb5c2822e6498da P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.128444 14811 ts_tablet_manager.cc:1397] T bfc34dc76ff94b8eaeb5c2822e6498da P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.069s user 0.002s sys 0.000s
I20250115 03:06:21.128897 14811 raft_consensus.cc:357] T bfc34dc76ff94b8eaeb5c2822e6498da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.129024 14811 raft_consensus.cc:383] T bfc34dc76ff94b8eaeb5c2822e6498da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.129060 14811 raft_consensus.cc:738] T bfc34dc76ff94b8eaeb5c2822e6498da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.129173 14811 consensus_queue.cc:260] T bfc34dc76ff94b8eaeb5c2822e6498da P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.129423 14811 ts_tablet_manager.cc:1428] T bfc34dc76ff94b8eaeb5c2822e6498da P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.129593 14811 tablet_bootstrap.cc:492] T 66b748fc853f499fafc1f476c8f79b8b P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.129735 14812 tablet_bootstrap.cc:492] T 80ede4e9696d491aa385b375fcb5cf3b P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.129808 14812 ts_tablet_manager.cc:1397] T 80ede4e9696d491aa385b375fcb5cf3b P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.082s user 0.001s sys 0.000s
I20250115 03:06:21.130239 14812 raft_consensus.cc:357] T 80ede4e9696d491aa385b375fcb5cf3b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.130331 14812 raft_consensus.cc:383] T 80ede4e9696d491aa385b375fcb5cf3b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.130357 14812 raft_consensus.cc:738] T 80ede4e9696d491aa385b375fcb5cf3b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.130460 14812 consensus_queue.cc:260] T 80ede4e9696d491aa385b375fcb5cf3b P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.130659 14811 tablet_bootstrap.cc:654] T 66b748fc853f499fafc1f476c8f79b8b P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.130722 14812 ts_tablet_manager.cc:1428] T 80ede4e9696d491aa385b375fcb5cf3b P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.130889 14812 tablet_bootstrap.cc:492] T 1016cf85491d4ceda132ee57325b48a4 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.131785 14812 tablet_bootstrap.cc:654] T 1016cf85491d4ceda132ee57325b48a4 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.133795 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_a27836029cad44e4b7aecbb06d4a0b93"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.134084 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a27836029cad44e4b7aecbb06d4a0b93 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.135339 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet f72c9bf04a3942ceacc37fd407bff033 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:21.137292 14809 tablet_bootstrap.cc:492] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.137369 14809 ts_tablet_manager.cc:1397] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.000s
I20250115 03:06:21.137773 14809 raft_consensus.cc:357] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.137859 14809 raft_consensus.cc:383] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.137885 14809 raft_consensus.cc:738] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.137990 14809 consensus_queue.cc:260] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.138247 14809 ts_tablet_manager.cc:1428] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:21.138401 14809 tablet_bootstrap.cc:492] T 7f9818d0493547358fd5654a01d4f456 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.138746 15401 raft_consensus.cc:491] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.138818 15401 raft_consensus.cc:513] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.139115 15401 leader_election.cc:290] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.139396 14809 tablet_bootstrap.cc:654] T 7f9818d0493547358fd5654a01d4f456 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.140522 14811 tablet_bootstrap.cc:492] T 66b748fc853f499fafc1f476c8f79b8b P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.140578 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2ca06c402cd46dc8311c8a47da92846" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.140645 14811 ts_tablet_manager.cc:1397] T 66b748fc853f499fafc1f476c8f79b8b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
W20250115 03:06:21.140909 14358 leader_election.cc:343] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.141085 14811 raft_consensus.cc:357] T 66b748fc853f499fafc1f476c8f79b8b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.141242 14811 raft_consensus.cc:383] T 66b748fc853f499fafc1f476c8f79b8b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.141319 14811 raft_consensus.cc:738] T 66b748fc853f499fafc1f476c8f79b8b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.141508 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2ca06c402cd46dc8311c8a47da92846" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:21.141988 14356 leader_election.cc:343] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.142055 14356 leader_election.cc:304] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
W20250115 03:06:21.143138 14163 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b08e5fc2110e476e897240f9cf3b874d consensus state has no leader
I20250115 03:06:21.143515 15401 raft_consensus.cc:2743] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.144832 14167 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5db9d5f6cecb454a90ad931d8d165bcc (table default.Table_e68b21fe987c4ea49523f24440585fce [id=07019baad4a74fd0b468bbd9b60d54c8]). Expected version 1 got 0
I20250115 03:06:21.141995 14809 tablet_bootstrap.cc:492] T 7f9818d0493547358fd5654a01d4f456 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.145931 14809 ts_tablet_manager.cc:1397] T 7f9818d0493547358fd5654a01d4f456 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20250115 03:06:21.146361 14809 raft_consensus.cc:357] T 7f9818d0493547358fd5654a01d4f456 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.146509 14809 raft_consensus.cc:383] T 7f9818d0493547358fd5654a01d4f456 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.146575 14809 raft_consensus.cc:738] T 7f9818d0493547358fd5654a01d4f456 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.146728 14809 consensus_queue.cc:260] T 7f9818d0493547358fd5654a01d4f456 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.147107 14809 ts_tablet_manager.cc:1428] T 7f9818d0493547358fd5654a01d4f456 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.147444 14809 tablet_bootstrap.cc:492] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.149286 14809 tablet_bootstrap.cc:654] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.141516 14811 consensus_queue.cc:260] T 66b748fc853f499fafc1f476c8f79b8b P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.154107 14811 ts_tablet_manager.cc:1428] T 66b748fc853f499fafc1f476c8f79b8b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.013s user 0.001s sys 0.000s
I20250115 03:06:21.155683 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_9db226106c5648bd82ff07e877c08ad7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.156013 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9db226106c5648bd82ff07e877c08ad7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.156250 14811 tablet_bootstrap.cc:492] T 3b5710268643446b91d04e2f4c7f999e P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.156706 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_51c53a8a98774de291a3d57f87eefe5d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.157004 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_51c53a8a98774de291a3d57f87eefe5d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.158288 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_47111cefc12c46d2aa88d3280df7245b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.158563 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_47111cefc12c46d2aa88d3280df7245b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.159053 14811 tablet_bootstrap.cc:654] T 3b5710268643446b91d04e2f4c7f999e P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.162262 15617 tablet.cc:1721] T 341b24d9b63f4761b2013035023fcd98 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:21.162618 16265 tablet.cc:1721] T 341b24d9b63f4761b2013035023fcd98 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:21.163120 15970 tablet.cc:1721] T 341b24d9b63f4761b2013035023fcd98 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:21.164183 14811 tablet_bootstrap.cc:492] T 3b5710268643446b91d04e2f4c7f999e P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.164309 14811 ts_tablet_manager.cc:1397] T 3b5710268643446b91d04e2f4c7f999e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20250115 03:06:21.164773 14811 raft_consensus.cc:357] T 3b5710268643446b91d04e2f4c7f999e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.164944 14811 raft_consensus.cc:383] T 3b5710268643446b91d04e2f4c7f999e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.164996 14811 raft_consensus.cc:738] T 3b5710268643446b91d04e2f4c7f999e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.165144 14811 consensus_queue.cc:260] T 3b5710268643446b91d04e2f4c7f999e P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.165504 14811 ts_tablet_manager.cc:1428] T 3b5710268643446b91d04e2f4c7f999e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.165699 14811 tablet_bootstrap.cc:492] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.166654 14811 tablet_bootstrap.cc:654] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.171662 14811 tablet_bootstrap.cc:492] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.171789 14811 ts_tablet_manager.cc:1397] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250115 03:06:21.172273 14811 raft_consensus.cc:357] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.172417 14811 raft_consensus.cc:383] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.172699 14811 raft_consensus.cc:738] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.172899 14811 consensus_queue.cc:260] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.173799 14811 ts_tablet_manager.cc:1428] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:21.174199 14165 master_service.cc:946] ReplaceTablet: received request to replace tablet 6004f4adca8e47ddb91f7e39c4cf9089 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:21.174258 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_20607cbd2ddf454bb006aa12bb3eca45"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.174541 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_20607cbd2ddf454bb006aa12bb3eca45 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.174794 14811 tablet_bootstrap.cc:492] T 676ec8b9dc8241d4ae40488d3ce8f61e P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.175577 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 9febec144bb343a8a21817e6d3655ad7 (table default.Table_57fbe77148284b4a9666630570f69321 [id=c9ed0b9c92ef40efa5c51cfdd8389325]). Expected version 2 got 0
I20250115 03:06:21.175742 14162 catalog_manager.cc:5526] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:21.176424 14397 tablet_service.cc:1467] Processing CreateTablet for tablet e7fb6422afb44ec88db538b74ce74175 (DEFAULT_TABLE table=default.Table_8f491acf518347ee9b94a67183cfde02 [id=e0946f0e55bc4367ba654cac0c43d65b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.176680 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7fb6422afb44ec88db538b74ce74175. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.177007 14536 tablet_service.cc:1467] Processing CreateTablet for tablet e7fb6422afb44ec88db538b74ce74175 (DEFAULT_TABLE table=default.Table_8f491acf518347ee9b94a67183cfde02 [id=e0946f0e55bc4367ba654cac0c43d65b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.177029 14668 tablet_service.cc:1467] Processing CreateTablet for tablet e7fb6422afb44ec88db538b74ce74175 (DEFAULT_TABLE table=default.Table_8f491acf518347ee9b94a67183cfde02 [id=e0946f0e55bc4367ba654cac0c43d65b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.177202 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7fb6422afb44ec88db538b74ce74175. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.177263 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7fb6422afb44ec88db538b74ce74175. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.179495 14811 tablet_bootstrap.cc:654] T 676ec8b9dc8241d4ae40488d3ce8f61e P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.180040 16411 raft_consensus.cc:491] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.180153 16411 raft_consensus.cc:513] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.180438 14811 tablet_bootstrap.cc:492] T 676ec8b9dc8241d4ae40488d3ce8f61e P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.180548 14811 ts_tablet_manager.cc:1397] T 676ec8b9dc8241d4ae40488d3ce8f61e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250115 03:06:21.180980 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda418ed197e4af2aa511d3f1851663e" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.180999 14811 raft_consensus.cc:357] T 676ec8b9dc8241d4ae40488d3ce8f61e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.181115 14811 raft_consensus.cc:383] T 676ec8b9dc8241d4ae40488d3ce8f61e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.181144 14811 raft_consensus.cc:738] T 676ec8b9dc8241d4ae40488d3ce8f61e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
W20250115 03:06:21.181285 14356 leader_election.cc:343] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.181376 14811 consensus_queue.cc:260] T 676ec8b9dc8241d4ae40488d3ce8f61e P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.181730 14811 ts_tablet_manager.cc:1428] T 676ec8b9dc8241d4ae40488d3ce8f61e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.181826 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda418ed197e4af2aa511d3f1851663e" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.181931 14811 tablet_bootstrap.cc:492] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
W20250115 03:06:21.182291 14358 leader_election.cc:343] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.182358 14358 leader_election.cc:304] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.182754 14811 tablet_bootstrap.cc:654] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.182641 14167 catalog_manager.cc:5526] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:21.182828 14167 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 676ec8b9dc8241d4ae40488d3ce8f61e (table default.Table_0b6cd33c30014981b78899a738418cbf [id=63edd09bfaaa4d56b4314ce760c87d3d]). Expected version 1 got 0
I20250115 03:06:21.183461 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet adbb0ae719f24dcc96709a83981e213d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.184562 14820 tablet_replica.cc:331] T adbb0ae719f24dcc96709a83981e213d P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:21.184693 14820 raft_consensus.cc:2238] T adbb0ae719f24dcc96709a83981e213d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.184799 14820 raft_consensus.cc:2267] T adbb0ae719f24dcc96709a83981e213d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.185261 14820 ts_tablet_manager.cc:1905] T adbb0ae719f24dcc96709a83981e213d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.185303 16406 raft_consensus.cc:2743] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.185565 16411 leader_election.cc:290] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.186158 14811 tablet_bootstrap.cc:492] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.186273 14811 ts_tablet_manager.cc:1397] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250115 03:06:21.186692 14811 raft_consensus.cc:357] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.186833 14811 raft_consensus.cc:383] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.186899 14811 raft_consensus.cc:738] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.187062 14811 consensus_queue.cc:260] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.187529 14809 tablet_bootstrap.cc:492] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.187568 14820 ts_tablet_manager.cc:1918] T adbb0ae719f24dcc96709a83981e213d P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.187613 14809 ts_tablet_manager.cc:1397] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.000s
I20250115 03:06:21.187633 14820 log.cc:1198] T adbb0ae719f24dcc96709a83981e213d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/adbb0ae719f24dcc96709a83981e213d
I20250115 03:06:21.187825 14820 ts_tablet_manager.cc:1939] T adbb0ae719f24dcc96709a83981e213d P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:21.188041 14809 raft_consensus.cc:357] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.188133 14809 raft_consensus.cc:383] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.188163 14809 raft_consensus.cc:738] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.188268 14809 consensus_queue.cc:260] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.188308 14811 ts_tablet_manager.cc:1428] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:21.188519 14809 ts_tablet_manager.cc:1428] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.188686 14809 tablet_bootstrap.cc:492] T a8e41e31ffae4bb092cb08d83eda16b5 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.189028 14811 tablet_bootstrap.cc:492] T b8ec24d2937d44e79b839276bf615fa0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.189586 14809 tablet_bootstrap.cc:654] T a8e41e31ffae4bb092cb08d83eda16b5 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.190850 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet adbb0ae719f24dcc96709a83981e213d (table default.Table_e73230225df744a2a11f246bea89b8b0 [id=407331115fba4ca28b006a587584ea0d]) successfully deleted
I20250115 03:06:21.191114 14811 tablet_bootstrap.cc:654] T b8ec24d2937d44e79b839276bf615fa0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.192023 14811 tablet_bootstrap.cc:492] T b8ec24d2937d44e79b839276bf615fa0 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.192096 14811 ts_tablet_manager.cc:1397] T b8ec24d2937d44e79b839276bf615fa0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:21.192457 14811 raft_consensus.cc:357] T b8ec24d2937d44e79b839276bf615fa0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.192546 14811 raft_consensus.cc:383] T b8ec24d2937d44e79b839276bf615fa0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.192574 14811 raft_consensus.cc:738] T b8ec24d2937d44e79b839276bf615fa0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.192664 14811 consensus_queue.cc:260] T b8ec24d2937d44e79b839276bf615fa0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.192896 14811 ts_tablet_manager.cc:1428] T b8ec24d2937d44e79b839276bf615fa0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.193053 14811 tablet_bootstrap.cc:492] T 85cceb5a3f234d85937ad0bb75348991 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.193854 14811 tablet_bootstrap.cc:654] T 85cceb5a3f234d85937ad0bb75348991 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.194072 14165 master_service.cc:946] ReplaceTablet: received request to replace tablet 6f53ad530584487e946120a724f99cfa from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:21.195640 14809 tablet_bootstrap.cc:492] T a8e41e31ffae4bb092cb08d83eda16b5 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.195726 14809 ts_tablet_manager.cc:1397] T a8e41e31ffae4bb092cb08d83eda16b5 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20250115 03:06:21.196120 14809 raft_consensus.cc:357] T a8e41e31ffae4bb092cb08d83eda16b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.196206 14809 raft_consensus.cc:383] T a8e41e31ffae4bb092cb08d83eda16b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.196236 14809 raft_consensus.cc:738] T a8e41e31ffae4bb092cb08d83eda16b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.196334 14809 consensus_queue.cc:260] T a8e41e31ffae4bb092cb08d83eda16b5 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.196635 14811 tablet_bootstrap.cc:492] T 85cceb5a3f234d85937ad0bb75348991 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.196635 14809 ts_tablet_manager.cc:1428] T a8e41e31ffae4bb092cb08d83eda16b5 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.196700 14811 ts_tablet_manager.cc:1397] T 85cceb5a3f234d85937ad0bb75348991 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:21.196784 14809 tablet_bootstrap.cc:492] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.197062 14811 raft_consensus.cc:357] T 85cceb5a3f234d85937ad0bb75348991 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.197151 14811 raft_consensus.cc:383] T 85cceb5a3f234d85937ad0bb75348991 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.197178 14811 raft_consensus.cc:738] T 85cceb5a3f234d85937ad0bb75348991 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.197263 14811 consensus_queue.cc:260] T 85cceb5a3f234d85937ad0bb75348991 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.197495 14811 ts_tablet_manager.cc:1428] T 85cceb5a3f234d85937ad0bb75348991 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.197641 14811 tablet_bootstrap.cc:492] T 80ede4e9696d491aa385b375fcb5cf3b P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.197646 14809 tablet_bootstrap.cc:654] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.198408 14811 tablet_bootstrap.cc:654] T 80ede4e9696d491aa385b375fcb5cf3b P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.198942 16189 raft_consensus.cc:491] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.199009 16189 raft_consensus.cc:513] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.199276 16189 leader_election.cc:290] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.200089 14809 tablet_bootstrap.cc:492] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.200157 14809 ts_tablet_manager.cc:1397] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:21.200551 14809 raft_consensus.cc:357] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.200635 14809 raft_consensus.cc:383] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.200665 14809 raft_consensus.cc:738] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.200747 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc1f0f72a3b4c79847c0a91fe98ccd1" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.200755 14809 consensus_queue.cc:260] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.200852 14689 raft_consensus.cc:2463] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.201067 14809 ts_tablet_manager.cc:1428] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.201156 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc1f0f72a3b4c79847c0a91fe98ccd1" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.201267 14422 raft_consensus.cc:2463] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.201376 14493 leader_election.cc:304] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.201524 14853 raft_consensus.cc:2798] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.201565 14853 raft_consensus.cc:491] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.201601 14853 raft_consensus.cc:3054] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.202409 14853 raft_consensus.cc:513] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.202705 14853 leader_election.cc:290] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.203114 14809 tablet_bootstrap.cc:492] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.203276 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 3bc1f0f72a3b4c79847c0a91fe98ccd1 (table default.Table_b165e5d12ca845b9984c99dde5e2e06e [id=dfe0ca35843c4fc6b975dd1ab87e9a99]). Expected version 1 got 0
I20250115 03:06:21.203408 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 676ec8b9dc8241d4ae40488d3ce8f61e (table default.Table_0b6cd33c30014981b78899a738418cbf [id=63edd09bfaaa4d56b4314ce760c87d3d]). Expected version 1 got 0
I20250115 03:06:21.203963 14809 tablet_bootstrap.cc:654] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.204208 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet 85cceb5a3f234d85937ad0bb75348991 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.204339 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 80ede4e9696d491aa385b375fcb5cf3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.204460 14823 tablet_replica.cc:331] T 85cceb5a3f234d85937ad0bb75348991 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:21.204578 14823 raft_consensus.cc:2238] T 85cceb5a3f234d85937ad0bb75348991 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.204672 14823 raft_consensus.cc:2267] T 85cceb5a3f234d85937ad0bb75348991 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.205173 14823 ts_tablet_manager.cc:1905] T 85cceb5a3f234d85937ad0bb75348991 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.205724 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc1f0f72a3b4c79847c0a91fe98ccd1" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:21.205808 14689 raft_consensus.cc:3054] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.205938 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4af064a5c1cc4ad7bc2b1d802e014110" } modify_external_catalogs: true
I20250115 03:06:21.206024 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4af064a5c1cc4ad7bc2b1d802e014110" } modify_external_catalogs: true
I20250115 03:06:21.206568 14689 raft_consensus.cc:2463] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.206657 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_84135e72d32d48e5820645a77dc85f72" } modify_external_catalogs: true
I20250115 03:06:21.206748 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_84135e72d32d48e5820645a77dc85f72" } modify_external_catalogs: true
I20250115 03:06:21.207342 14823 ts_tablet_manager.cc:1918] T 85cceb5a3f234d85937ad0bb75348991 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.207403 14823 log.cc:1198] T 85cceb5a3f234d85937ad0bb75348991 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/85cceb5a3f234d85937ad0bb75348991
I20250115 03:06:21.207494 14493 leader_election.cc:304] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.207590 14823 ts_tablet_manager.cc:1939] T 85cceb5a3f234d85937ad0bb75348991 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:21.207649 14853 raft_consensus.cc:2798] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.207710 14853 raft_consensus.cc:695] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:21.207811 14853 consensus_queue.cc:237] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.208092 14823 tablet_replica.cc:331] T 80ede4e9696d491aa385b375fcb5cf3b P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:21.208202 14823 raft_consensus.cc:2238] T 80ede4e9696d491aa385b375fcb5cf3b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.208292 14823 raft_consensus.cc:2267] T 80ede4e9696d491aa385b375fcb5cf3b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.208477 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bc1f0f72a3b4c79847c0a91fe98ccd1" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.208570 14422 raft_consensus.cc:3054] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.208698 14809 tablet_bootstrap.cc:492] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.208765 14809 ts_tablet_manager.cc:1397] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:21.209049 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 85cceb5a3f234d85937ad0bb75348991 (table default.Table_eec974cf89de4f45930dbf47fa754f2a [id=36660de3890e411e974c41b7f8f85946]) successfully deleted
I20250115 03:06:21.209179 14809 raft_consensus.cc:357] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.209264 14809 raft_consensus.cc:383] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.209293 14809 raft_consensus.cc:738] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.209391 14809 consensus_queue.cc:260] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.209491 14422 raft_consensus.cc:2463] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.209664 14809 ts_tablet_manager.cc:1428] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.209812 14809 tablet_bootstrap.cc:492] T c2afa048b3fd4362b50c8a6da306c25a P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.210039 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 6f53ad530584487e946120a724f99cfa
I20250115 03:06:21.210156 14811 tablet_bootstrap.cc:492] T 80ede4e9696d491aa385b375fcb5cf3b P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.210224 14811 ts_tablet_manager.cc:1397] T 80ede4e9696d491aa385b375fcb5cf3b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20250115 03:06:21.210278 14165 catalog_manager.cc:6596] ReplaceTablet: tablet 6f53ad530584487e946120a724f99cfa deleted and replaced by tablet a142f9f7321a484ba7fde981ecc417e4
I20250115 03:06:21.210608 14811 raft_consensus.cc:357] T 80ede4e9696d491aa385b375fcb5cf3b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.210670 14809 tablet_bootstrap.cc:654] T c2afa048b3fd4362b50c8a6da306c25a P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.210692 14811 raft_consensus.cc:383] T 80ede4e9696d491aa385b375fcb5cf3b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.210719 14811 raft_consensus.cc:738] T 80ede4e9696d491aa385b375fcb5cf3b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.210816 14811 consensus_queue.cc:260] T 80ede4e9696d491aa385b375fcb5cf3b P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.211047 14811 ts_tablet_manager.cc:1428] T 80ede4e9696d491aa385b375fcb5cf3b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.211215 14811 tablet_bootstrap.cc:492] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.211527 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f53ad530584487e946120a724f99cfa with delete_type TABLET_DATA_DELETED (replaced by tablet a142f9f7321a484ba7fde981ecc417e4) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.212245 14811 tablet_bootstrap.cc:654] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.212615 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f53ad530584487e946120a724f99cfa with delete_type TABLET_DATA_DELETED (replaced by tablet a142f9f7321a484ba7fde981ecc417e4) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.213057 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f53ad530584487e946120a724f99cfa with delete_type TABLET_DATA_DELETED (replaced by tablet a142f9f7321a484ba7fde981ecc417e4) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:21.213454 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 6f53ad530584487e946120a724f99cfa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f53ad530584487e946120a724f99cfa already in progress: creating tablet
W20250115 03:06:21.214356 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 6f53ad530584487e946120a724f99cfa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f53ad530584487e946120a724f99cfa already in progress: creating tablet
I20250115 03:06:21.214586 14823 ts_tablet_manager.cc:1905] T 80ede4e9696d491aa385b375fcb5cf3b P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.214635 16088 consensus_queue.cc:1035] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:21.216729 14809 tablet_bootstrap.cc:492] T c2afa048b3fd4362b50c8a6da306c25a P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.216802 14809 ts_tablet_manager.cc:1397] T c2afa048b3fd4362b50c8a6da306c25a P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20250115 03:06:21.217199 14809 raft_consensus.cc:357] T c2afa048b3fd4362b50c8a6da306c25a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.217283 14809 raft_consensus.cc:383] T c2afa048b3fd4362b50c8a6da306c25a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.217311 14809 raft_consensus.cc:738] T c2afa048b3fd4362b50c8a6da306c25a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.217319 14812 tablet_bootstrap.cc:492] T 1016cf85491d4ceda132ee57325b48a4 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.217406 14812 ts_tablet_manager.cc:1397] T 1016cf85491d4ceda132ee57325b48a4 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.087s user 0.001s sys 0.000s
I20250115 03:06:21.217410 14809 consensus_queue.cc:260] T c2afa048b3fd4362b50c8a6da306c25a P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.217655 14809 ts_tablet_manager.cc:1428] T c2afa048b3fd4362b50c8a6da306c25a P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.217818 14809 tablet_bootstrap.cc:492] T 6ed47b26cdf744acb28fb657a1b41087 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.217801 14812 raft_consensus.cc:357] T 1016cf85491d4ceda132ee57325b48a4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.217890 14812 raft_consensus.cc:383] T 1016cf85491d4ceda132ee57325b48a4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.217918 14812 raft_consensus.cc:738] T 1016cf85491d4ceda132ee57325b48a4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.218011 14812 consensus_queue.cc:260] T 1016cf85491d4ceda132ee57325b48a4 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.218268 14812 ts_tablet_manager.cc:1428] T 1016cf85491d4ceda132ee57325b48a4 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.218421 14812 tablet_bootstrap.cc:492] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.218271 14823 ts_tablet_manager.cc:1918] T 80ede4e9696d491aa385b375fcb5cf3b P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.218654 14809 tablet_bootstrap.cc:654] T 6ed47b26cdf744acb28fb657a1b41087 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.218689 14823 log.cc:1198] T 80ede4e9696d491aa385b375fcb5cf3b P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/80ede4e9696d491aa385b375fcb5cf3b
I20250115 03:06:21.218880 14823 ts_tablet_manager.cc:1939] T 80ede4e9696d491aa385b375fcb5cf3b P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:21.219134 14812 tablet_bootstrap.cc:654] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.219401 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 80ede4e9696d491aa385b375fcb5cf3b (table default.Table_75cd7b01fc534652975eb593912b88bc [id=111fc67094164d3f99965c246ae12b24]) successfully deleted
I20250115 03:06:21.220090 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ec52a51cbc9e40f9bd1f983f932c4698" } modify_external_catalogs: true
I20250115 03:06:21.220177 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ec52a51cbc9e40f9bd1f983f932c4698" } modify_external_catalogs: true
W20250115 03:06:21.220574 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 6f53ad530584487e946120a724f99cfa because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6f53ad530584487e946120a724f99cfa already in progress: creating tablet
I20250115 03:06:21.220942 16001 consensus_queue.cc:1035] T 85b4eb3b20bb4908884e5716e7b2a369 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:21.221423 14162 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_2d473b82a1764ef6afd586a7bf8e17dc [id=369c42ec23ec4a27a71a54be23a410a6] alter complete (version 1)
I20250115 03:06:21.222734 14534 tablet_service.cc:1514] Processing DeleteTablet for tablet bfc34dc76ff94b8eaeb5c2822e6498da with delete_type TABLET_DATA_DELETED (replaced by tablet f2ca06c402cd46dc8311c8a47da92846) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.222885 14820 tablet_replica.cc:331] T bfc34dc76ff94b8eaeb5c2822e6498da P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:21.222991 14820 raft_consensus.cc:2238] T bfc34dc76ff94b8eaeb5c2822e6498da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.223081 14820 raft_consensus.cc:2267] T bfc34dc76ff94b8eaeb5c2822e6498da P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.223269 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 66b748fc853f499fafc1f476c8f79b8b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.223585 14820 ts_tablet_manager.cc:1905] T bfc34dc76ff94b8eaeb5c2822e6498da P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.224898 14809 tablet_bootstrap.cc:492] T 6ed47b26cdf744acb28fb657a1b41087 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.224999 14809 ts_tablet_manager.cc:1397] T 6ed47b26cdf744acb28fb657a1b41087 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20250115 03:06:21.225353 14809 raft_consensus.cc:357] T 6ed47b26cdf744acb28fb657a1b41087 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.225445 14809 raft_consensus.cc:383] T 6ed47b26cdf744acb28fb657a1b41087 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.225474 14809 raft_consensus.cc:738] T 6ed47b26cdf744acb28fb657a1b41087 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.225565 14809 consensus_queue.cc:260] T 6ed47b26cdf744acb28fb657a1b41087 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.225662 14820 ts_tablet_manager.cc:1918] T bfc34dc76ff94b8eaeb5c2822e6498da P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.225718 14820 log.cc:1198] T bfc34dc76ff94b8eaeb5c2822e6498da P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/bfc34dc76ff94b8eaeb5c2822e6498da
I20250115 03:06:21.225782 14809 ts_tablet_manager.cc:1428] T 6ed47b26cdf744acb28fb657a1b41087 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.225903 14820 ts_tablet_manager.cc:1939] T bfc34dc76ff94b8eaeb5c2822e6498da P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:21.225908 14809 tablet_bootstrap.cc:492] T fca14a7d574842ee94a93afb85ca1afd P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.226333 14820 tablet_replica.cc:331] T 66b748fc853f499fafc1f476c8f79b8b P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:21.226714 14809 tablet_bootstrap.cc:654] T fca14a7d574842ee94a93afb85ca1afd P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.226837 14820 raft_consensus.cc:2238] T 66b748fc853f499fafc1f476c8f79b8b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.226931 14820 raft_consensus.cc:2267] T 66b748fc853f499fafc1f476c8f79b8b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.227326 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet bfc34dc76ff94b8eaeb5c2822e6498da (table default.Table_802f667f1ad246688cbdcffcb3001021 [id=e929ffaba567498c815393836b264a12]) successfully deleted
I20250115 03:06:21.229367 14812 tablet_bootstrap.cc:492] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.229439 14812 ts_tablet_manager.cc:1397] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20250115 03:06:21.229846 14812 raft_consensus.cc:357] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.229943 14812 raft_consensus.cc:383] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.229974 14812 raft_consensus.cc:738] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.230069 14812 consensus_queue.cc:260] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.230314 14812 ts_tablet_manager.cc:1428] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.230469 14812 tablet_bootstrap.cc:492] T 12678da7ffee4a46b4690ac3e94cb19b P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.230904 16411 raft_consensus.cc:491] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.230973 16411 raft_consensus.cc:513] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.231257 16411 leader_election.cc:290] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.231333 14812 tablet_bootstrap.cc:654] T 12678da7ffee4a46b4690ac3e94cb19b P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.231642 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad0e9216bd154f4a8a37874d3d8a08c2" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.231926 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad0e9216bd154f4a8a37874d3d8a08c2" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:21.232127 14356 leader_election.cc:343] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:21.232662 14358 leader_election.cc:343] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.232734 14358 leader_election.cc:304] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.232956 16411 raft_consensus.cc:2743] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.233570 14820 ts_tablet_manager.cc:1905] T 66b748fc853f499fafc1f476c8f79b8b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.234418 16411 raft_consensus.cc:491] T 815ef85ce5cf4ae0a56c5734837e03ba P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.234509 16411 raft_consensus.cc:513] T 815ef85ce5cf4ae0a56c5734837e03ba P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.234724 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 9eb56ebdb1a24b8ea8fbc4c129fded95
I20250115 03:06:21.234831 16411 leader_election.cc:290] T 815ef85ce5cf4ae0a56c5734837e03ba P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.235239 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "815ef85ce5cf4ae0a56c5734837e03ba" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.235582 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "815ef85ce5cf4ae0a56c5734837e03ba" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:21.235853 14356 leader_election.cc:343] T 815ef85ce5cf4ae0a56c5734837e03ba P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:21.235932 14358 leader_election.cc:343] T 815ef85ce5cf4ae0a56c5734837e03ba P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.235996 14358 leader_election.cc:304] T 815ef85ce5cf4ae0a56c5734837e03ba P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.236202 16411 raft_consensus.cc:2743] T 815ef85ce5cf4ae0a56c5734837e03ba P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.236828 14820 ts_tablet_manager.cc:1918] T 66b748fc853f499fafc1f476c8f79b8b P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.236968 14820 log.cc:1198] T 66b748fc853f499fafc1f476c8f79b8b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/66b748fc853f499fafc1f476c8f79b8b
I20250115 03:06:21.237248 16411 raft_consensus.cc:491] T 38f5554b10a44c249e4c3d3825cd7952 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.237316 16411 raft_consensus.cc:513] T 38f5554b10a44c249e4c3d3825cd7952 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.237444 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 51b412277e634ea39bb6d2b90a43b875
I20250115 03:06:21.237625 16411 leader_election.cc:290] T 38f5554b10a44c249e4c3d3825cd7952 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.238000 14820 ts_tablet_manager.cc:1939] T 66b748fc853f499fafc1f476c8f79b8b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:21.238346 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f5554b10a44c249e4c3d3825cd7952" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:21.238641 14358 leader_election.cc:343] T 38f5554b10a44c249e4c3d3825cd7952 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.239109 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f5554b10a44c249e4c3d3825cd7952" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:21.239414 14356 leader_election.cc:343] T 38f5554b10a44c249e4c3d3825cd7952 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.239486 14356 leader_election.cc:304] T 38f5554b10a44c249e4c3d3825cd7952 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.239512 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 66b748fc853f499fafc1f476c8f79b8b (table default.Table_7eacd8191bf44769b6371caa6d4772f1 [id=ea9e7141223d43de8962caa78ab14914]) successfully deleted
I20250115 03:06:21.239647 16411 raft_consensus.cc:2743] T 38f5554b10a44c249e4c3d3825cd7952 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.239828 16411 raft_consensus.cc:491] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.239859 16189 raft_consensus.cc:491] T 3b5710268643446b91d04e2f4c7f999e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.239882 16411 raft_consensus.cc:513] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.239936 16189 raft_consensus.cc:513] T 3b5710268643446b91d04e2f4c7f999e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.240164 16411 leader_election.cc:290] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.240247 16189 leader_election.cc:290] T 3b5710268643446b91d04e2f4c7f999e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.240535 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eb56ebdb1a24b8ea8fbc4c129fded95 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.240628 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b5710268643446b91d04e2f4c7f999e" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.240772 14653 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eb56ebdb1a24b8ea8fbc4c129fded95 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.240888 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet 51b412277e634ea39bb6d2b90a43b875 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.241144 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eb56ebdb1a24b8ea8fbc4c129fded95 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:21.241186 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 9eb56ebdb1a24b8ea8fbc4c129fded95 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9eb56ebdb1a24b8ea8fbc4c129fded95 already in progress: creating tablet
I20250115 03:06:21.241217 14823 tablet_replica.cc:331] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:21.241324 14823 raft_consensus.cc:2238] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:21.241590 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 51b412277e634ea39bb6d2b90a43b875 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.241652 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b5710268643446b91d04e2f4c7f999e" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.241773 14419 raft_consensus.cc:2388] T 3b5710268643446b91d04e2f4c7f999e P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in current term 1: Already voted for candidate 5c0dae7aff4b44c699275b95ba639770 in this term.
I20250115 03:06:21.241760 14823 raft_consensus.cc:2267] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.241894 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 51b412277e634ea39bb6d2b90a43b875 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.241986 14493 leader_election.cc:304] T 3b5710268643446b91d04e2f4c7f999e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:21.242046 14819 tablet_replica.cc:331] T 51b412277e634ea39bb6d2b90a43b875 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.242156 14819 raft_consensus.cc:2238] T 51b412277e634ea39bb6d2b90a43b875 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.242190 14823 ts_tablet_manager.cc:1905] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.242241 14819 raft_consensus.cc:2267] T 51b412277e634ea39bb6d2b90a43b875 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.242311 14812 tablet_bootstrap.cc:492] T 12678da7ffee4a46b4690ac3e94cb19b P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
W20250115 03:06:21.242295 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 9eb56ebdb1a24b8ea8fbc4c129fded95 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9eb56ebdb1a24b8ea8fbc4c129fded95 already in progress: creating tablet
I20250115 03:06:21.242374 14812 ts_tablet_manager.cc:1397] T 12678da7ffee4a46b4690ac3e94cb19b P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20250115 03:06:21.242431 16189 raft_consensus.cc:3054] T 3b5710268643446b91d04e2f4c7f999e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.242558 14819 ts_tablet_manager.cc:1905] T 51b412277e634ea39bb6d2b90a43b875 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.242770 14812 raft_consensus.cc:357] T 12678da7ffee4a46b4690ac3e94cb19b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.242852 14812 raft_consensus.cc:383] T 12678da7ffee4a46b4690ac3e94cb19b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.242877 14812 raft_consensus.cc:738] T 12678da7ffee4a46b4690ac3e94cb19b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.242981 14812 consensus_queue.cc:260] T 12678da7ffee4a46b4690ac3e94cb19b P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.243237 14812 ts_tablet_manager.cc:1428] T 12678da7ffee4a46b4690ac3e94cb19b P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:21.243286 16189 raft_consensus.cc:2743] T 3b5710268643446b91d04e2f4c7f999e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.243405 14812 tablet_bootstrap.cc:492] T 0792e52d171446b48a7c9d1f03d37a79 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.241910 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72abf6acd9d54f3b983b7098aaf2691d" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.244096 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72abf6acd9d54f3b983b7098aaf2691d" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.244243 14823 ts_tablet_manager.cc:1918] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:21.244319 14823 log.cc:1198] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/51b412277e634ea39bb6d2b90a43b875
I20250115 03:06:21.244251 14812 tablet_bootstrap.cc:654] T 0792e52d171446b48a7c9d1f03d37a79 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:21.244483 14358 leader_election.cc:343] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.244598 14819 ts_tablet_manager.cc:1918] T 51b412277e634ea39bb6d2b90a43b875 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20250115 03:06:21.244649 14356 leader_election.cc:343] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.244702 14356 leader_election.cc:304] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.245076 14820 tablet_replica.cc:331] T 51b412277e634ea39bb6d2b90a43b875 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:21.245185 14820 raft_consensus.cc:2238] T 51b412277e634ea39bb6d2b90a43b875 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.245263 14820 raft_consensus.cc:2267] T 51b412277e634ea39bb6d2b90a43b875 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:21.245353 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 9eb56ebdb1a24b8ea8fbc4c129fded95 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9eb56ebdb1a24b8ea8fbc4c129fded95 already in progress: creating tablet
I20250115 03:06:21.245563 14820 ts_tablet_manager.cc:1905] T 51b412277e634ea39bb6d2b90a43b875 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.245750 16411 raft_consensus.cc:2743] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.244661 14819 log.cc:1198] T 51b412277e634ea39bb6d2b90a43b875 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/51b412277e634ea39bb6d2b90a43b875
I20250115 03:06:21.246479 14819 ts_tablet_manager.cc:1939] T 51b412277e634ea39bb6d2b90a43b875 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.247581 14820 ts_tablet_manager.cc:1918] T 51b412277e634ea39bb6d2b90a43b875 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:21.247643 14820 log.cc:1198] T 51b412277e634ea39bb6d2b90a43b875 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/51b412277e634ea39bb6d2b90a43b875
I20250115 03:06:21.247867 14820 ts_tablet_manager.cc:1939] T 51b412277e634ea39bb6d2b90a43b875 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:21.248385 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 51b412277e634ea39bb6d2b90a43b875 (table default.Table_4af064a5c1cc4ad7bc2b1d802e014110 [id=752d9e6368564d1ca34397e8e3eea7e7]) successfully deleted
I20250115 03:06:21.248530 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 51b412277e634ea39bb6d2b90a43b875 (table default.Table_4af064a5c1cc4ad7bc2b1d802e014110 [id=752d9e6368564d1ca34397e8e3eea7e7]) successfully deleted
I20250115 03:06:21.248576 14823 ts_tablet_manager.cc:1939] T 51b412277e634ea39bb6d2b90a43b875 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:21.249209 14162 catalog_manager.cc:5526] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:21.249418 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 3bc1f0f72a3b4c79847c0a91fe98ccd1 (table default.Table_b165e5d12ca845b9984c99dde5e2e06e [id=dfe0ca35843c4fc6b975dd1ab87e9a99]). Expected version 1 got 0
I20250115 03:06:21.250700 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 51b412277e634ea39bb6d2b90a43b875 (table default.Table_4af064a5c1cc4ad7bc2b1d802e014110 [id=752d9e6368564d1ca34397e8e3eea7e7]) successfully deleted
I20250115 03:06:21.250927 14668 tablet_service.cc:1467] Processing CreateTablet for tablet 45a4618ca707448aa273c610eab3c7d7 (DEFAULT_TABLE table=default.Table_51c53a8a98774de291a3d57f87eefe5d [id=80f5675756f344ee81dc9147f628ac79]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.251051 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a6170a1e9f9546abb2af9ba24456080d" } modify_external_catalogs: true
I20250115 03:06:21.251140 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45a4618ca707448aa273c610eab3c7d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.251137 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a6170a1e9f9546abb2af9ba24456080d" } modify_external_catalogs: true
I20250115 03:06:21.256140 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 8315091d2cae492d93023656149d8ef5 (DEFAULT_TABLE table=default.Table_e1daabdd4fac414bb329cb99820de56c [id=f6ff6e8bfd80445cb1d449124b13f63d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.256229 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 745a6e03a1544f6f99120d8cc4e7acb0 (DEFAULT_TABLE table=default.Table_da8cef7d7d1b4784aa3d21e269d7c71b [id=edf3da73687345c18e8c25fa5e9d446e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.256371 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8315091d2cae492d93023656149d8ef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.256531 14393 tablet_service.cc:1467] Processing CreateTablet for tablet ccce4613a9e74109b53e3ced7c5983de (DEFAULT_TABLE table=default.Table_1bc4f5e4e8a94919b6fc72560065d6af [id=3c8b27f353f941d69101280471649dc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.256605 14391 tablet_service.cc:1467] Processing CreateTablet for tablet 87e79385cac549b49e6c3477020ba8d1 (DEFAULT_TABLE table=default.Table_f3e6282c66774e58a910ea88782b43ea [id=f781e4a062f74ee195d56412231862bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.256718 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccce4613a9e74109b53e3ced7c5983de. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.256829 14391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87e79385cac549b49e6c3477020ba8d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.257030 14398 tablet_service.cc:1467] Processing CreateTablet for tablet d1b9208a0ecd4323987f1aabcc733546 (DEFAULT_TABLE table=default.Table_a27836029cad44e4b7aecbb06d4a0b93 [id=72c6cc76f06e40c6a8936fdee4fccc01]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.257117 14394 tablet_service.cc:1467] Processing CreateTablet for tablet b67739bf7afb4631ac5beec896337774 (DEFAULT_TABLE table=default.Table_9db226106c5648bd82ff07e877c08ad7 [id=b65a3b58b69e44afbef34fe8eeb2cded]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.257218 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1b9208a0ecd4323987f1aabcc733546. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.257325 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b67739bf7afb4631ac5beec896337774. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.257536 14395 tablet_service.cc:1467] Processing CreateTablet for tablet a6a03c2ab1494278bb0b5069e4f0f18f (DEFAULT_TABLE table=default.Table_47111cefc12c46d2aa88d3280df7245b [id=9c6b8a87a48b4315bbf8d02edd452ee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.257714 14395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6a03c2ab1494278bb0b5069e4f0f18f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.257968 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 45a4618ca707448aa273c610eab3c7d7 (DEFAULT_TABLE table=default.Table_51c53a8a98774de291a3d57f87eefe5d [id=80f5675756f344ee81dc9147f628ac79]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.258183 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45a4618ca707448aa273c610eab3c7d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.258448 14390 tablet_service.cc:1467] Processing CreateTablet for tablet 459440ce87bd4e0eb78cf6b9197122d8 (DEFAULT_TABLE table=default.Table_06955d84b4e94e5a82ba9e8402877f30 [id=9203e8aaaab84849bb9586d7abb030a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.258637 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 459440ce87bd4e0eb78cf6b9197122d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.259323 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 745a6e03a1544f6f99120d8cc4e7acb0 (DEFAULT_TABLE table=default.Table_da8cef7d7d1b4784aa3d21e269d7c71b [id=edf3da73687345c18e8c25fa5e9d446e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.259541 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 745a6e03a1544f6f99120d8cc4e7acb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.259598 14659 tablet_service.cc:1467] Processing CreateTablet for tablet ccce4613a9e74109b53e3ced7c5983de (DEFAULT_TABLE table=default.Table_1bc4f5e4e8a94919b6fc72560065d6af [id=3c8b27f353f941d69101280471649dc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.259800 14659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccce4613a9e74109b53e3ced7c5983de. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.260151 14534 tablet_service.cc:1467] Processing CreateTablet for tablet 8315091d2cae492d93023656149d8ef5 (DEFAULT_TABLE table=default.Table_e1daabdd4fac414bb329cb99820de56c [id=f6ff6e8bfd80445cb1d449124b13f63d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.260358 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8315091d2cae492d93023656149d8ef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.260419 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 459440ce87bd4e0eb78cf6b9197122d8 (DEFAULT_TABLE table=default.Table_06955d84b4e94e5a82ba9e8402877f30 [id=9203e8aaaab84849bb9586d7abb030a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.260622 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 459440ce87bd4e0eb78cf6b9197122d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.261539 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 87e79385cac549b49e6c3477020ba8d1 (DEFAULT_TABLE table=default.Table_f3e6282c66774e58a910ea88782b43ea [id=f781e4a062f74ee195d56412231862bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.261688 14671 tablet_service.cc:1467] Processing CreateTablet for tablet 87e79385cac549b49e6c3477020ba8d1 (DEFAULT_TABLE table=default.Table_f3e6282c66774e58a910ea88782b43ea [id=f781e4a062f74ee195d56412231862bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.261734 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87e79385cac549b49e6c3477020ba8d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.261870 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87e79385cac549b49e6c3477020ba8d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.262275 14527 tablet_service.cc:1467] Processing CreateTablet for tablet 459440ce87bd4e0eb78cf6b9197122d8 (DEFAULT_TABLE table=default.Table_06955d84b4e94e5a82ba9e8402877f30 [id=9203e8aaaab84849bb9586d7abb030a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.262472 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 459440ce87bd4e0eb78cf6b9197122d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.262465 14657 tablet_service.cc:1467] Processing CreateTablet for tablet b67739bf7afb4631ac5beec896337774 (DEFAULT_TABLE table=default.Table_9db226106c5648bd82ff07e877c08ad7 [id=b65a3b58b69e44afbef34fe8eeb2cded]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.262656 14657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b67739bf7afb4631ac5beec896337774. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.263621 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 45a4618ca707448aa273c610eab3c7d7 (DEFAULT_TABLE table=default.Table_51c53a8a98774de291a3d57f87eefe5d [id=80f5675756f344ee81dc9147f628ac79]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.263702 14656 tablet_service.cc:1467] Processing CreateTablet for tablet d1b9208a0ecd4323987f1aabcc733546 (DEFAULT_TABLE table=default.Table_a27836029cad44e4b7aecbb06d4a0b93 [id=72c6cc76f06e40c6a8936fdee4fccc01]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.263814 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45a4618ca707448aa273c610eab3c7d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.263904 14656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1b9208a0ecd4323987f1aabcc733546. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.264343 14537 tablet_service.cc:1467] Processing CreateTablet for tablet ccce4613a9e74109b53e3ced7c5983de (DEFAULT_TABLE table=default.Table_1bc4f5e4e8a94919b6fc72560065d6af [id=3c8b27f353f941d69101280471649dc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.264407 14655 tablet_service.cc:1467] Processing CreateTablet for tablet a6a03c2ab1494278bb0b5069e4f0f18f (DEFAULT_TABLE table=default.Table_47111cefc12c46d2aa88d3280df7245b [id=9c6b8a87a48b4315bbf8d02edd452ee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.264539 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccce4613a9e74109b53e3ced7c5983de. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.264596 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6a03c2ab1494278bb0b5069e4f0f18f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.265707 14526 tablet_service.cc:1467] Processing CreateTablet for tablet 745a6e03a1544f6f99120d8cc4e7acb0 (DEFAULT_TABLE table=default.Table_da8cef7d7d1b4784aa3d21e269d7c71b [id=edf3da73687345c18e8c25fa5e9d446e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.265901 14526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 745a6e03a1544f6f99120d8cc4e7acb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.266484 14518 tablet_service.cc:1467] Processing CreateTablet for tablet a6a03c2ab1494278bb0b5069e4f0f18f (DEFAULT_TABLE table=default.Table_47111cefc12c46d2aa88d3280df7245b [id=9c6b8a87a48b4315bbf8d02edd452ee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.266621 14811 tablet_bootstrap.cc:492] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.266675 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6a03c2ab1494278bb0b5069e4f0f18f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.266695 14811 ts_tablet_manager.cc:1397] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.056s user 0.001s sys 0.000s
I20250115 03:06:21.267114 14811 raft_consensus.cc:357] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.267204 14811 raft_consensus.cc:383] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.267234 14811 raft_consensus.cc:738] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.267328 14811 consensus_queue.cc:260] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.267592 14811 ts_tablet_manager.cc:1428] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.267768 14811 tablet_bootstrap.cc:492] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.268625 14811 tablet_bootstrap.cc:654] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.269057 14668 tablet_service.cc:1467] Processing CreateTablet for tablet 8315091d2cae492d93023656149d8ef5 (DEFAULT_TABLE table=default.Table_e1daabdd4fac414bb329cb99820de56c [id=f6ff6e8bfd80445cb1d449124b13f63d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.269085 14530 tablet_service.cc:1467] Processing CreateTablet for tablet b67739bf7afb4631ac5beec896337774 (DEFAULT_TABLE table=default.Table_9db226106c5648bd82ff07e877c08ad7 [id=b65a3b58b69e44afbef34fe8eeb2cded]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.269260 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8315091d2cae492d93023656149d8ef5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.269310 14533 tablet_service.cc:1467] Processing CreateTablet for tablet d1b9208a0ecd4323987f1aabcc733546 (DEFAULT_TABLE table=default.Table_a27836029cad44e4b7aecbb06d4a0b93 [id=72c6cc76f06e40c6a8936fdee4fccc01]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.269482 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 745a6e03a1544f6f99120d8cc4e7acb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.274636 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 85fcafa24fa1448c995d30dd93b0737f
I20250115 03:06:21.275111 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b67739bf7afb4631ac5beec896337774. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.275199 14526 tablet_service.cc:1514] Processing DeleteTablet for tablet 85fcafa24fa1448c995d30dd93b0737f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.276032 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet 85fcafa24fa1448c995d30dd93b0737f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:21.276394 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 85fcafa24fa1448c995d30dd93b0737f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 85fcafa24fa1448c995d30dd93b0737f already in progress: creating tablet
I20250115 03:06:21.277690 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1b9208a0ecd4323987f1aabcc733546. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:21.277969 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 85fcafa24fa1448c995d30dd93b0737f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 85fcafa24fa1448c995d30dd93b0737f already in progress: creating tablet
I20250115 03:06:21.279553 14809 tablet_bootstrap.cc:492] T fca14a7d574842ee94a93afb85ca1afd P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.279624 14809 ts_tablet_manager.cc:1397] T fca14a7d574842ee94a93afb85ca1afd P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.054s user 0.001s sys 0.000s
I20250115 03:06:21.280025 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet d4f76d0b3a394a5f8aa33ad17b11da34
I20250115 03:06:21.280056 14809 raft_consensus.cc:357] T fca14a7d574842ee94a93afb85ca1afd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.280138 14809 raft_consensus.cc:383] T fca14a7d574842ee94a93afb85ca1afd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.280165 14809 raft_consensus.cc:738] T fca14a7d574842ee94a93afb85ca1afd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.280267 14809 consensus_queue.cc:260] T fca14a7d574842ee94a93afb85ca1afd P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.280524 14809 ts_tablet_manager.cc:1428] T fca14a7d574842ee94a93afb85ca1afd P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:21.281327 14809 tablet_bootstrap.cc:492] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.282297 16400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "147c86b98f604d518e71f2ba3121fc21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4225fe2fbfa84937bb41fed702ca66e4" member_type: VOTER last_known_addr { host: "127.13.186.254" port: 39901 } } peers { permanent_uuid: "147c86b98f604d518e71f2ba3121fc21" member_type: VOTER last_known_addr { host: "127.13.186.253" port: 35411 } } peers { permanent_uuid: "2db8cff032af46db986bf52b21b52246" member_type: VOTER last_known_addr { host: "127.13.186.252" port: 41855 } } }
I20250115 03:06:21.282378 14809 tablet_bootstrap.cc:654] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.282366 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 6bd4e4200f9b489ea5dd2154aa6f382d (table default.Table_83b1fd83225a4454b29b7c61bc4639d4 [id=299ac71cfb98480f873eb659419e522c]). Expected version 1 got 0
I20250115 03:06:21.282925 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet d4f76d0b3a394a5f8aa33ad17b11da34 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.283070 14823 tablet_replica.cc:331] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:21.283195 14823 raft_consensus.cc:2238] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.283300 14823 raft_consensus.cc:2267] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.282394 16400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21 [sys.catalog]: This master's current role is: LEADER
I20250115 03:06:21.283632 14823 ts_tablet_manager.cc:1905] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.283779 16406 raft_consensus.cc:491] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.283851 16406 raft_consensus.cc:513] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.283892 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet d4f76d0b3a394a5f8aa33ad17b11da34 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.284094 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 85fcafa24fa1448c995d30dd93b0737f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.284168 16406 leader_election.cc:290] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.284257 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet d4f76d0b3a394a5f8aa33ad17b11da34 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.284443 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ed47b26cdf744acb28fb657a1b41087 with delete_type TABLET_DATA_DELETED (replaced by tablet fe8d94c8fecc4077ba052390810c7430) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.284523 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5db9d5f6cecb454a90ad931d8d165bcc" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.284615 14819 tablet_replica.cc:331] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.284721 14819 raft_consensus.cc:2238] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.284806 14819 raft_consensus.cc:2267] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.285341 14819 ts_tablet_manager.cc:1905] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.286233 14823 ts_tablet_manager.cc:1918] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:21.286299 14823 log.cc:1198] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/d4f76d0b3a394a5f8aa33ad17b11da34
I20250115 03:06:21.286533 14823 ts_tablet_manager.cc:1939] T d4f76d0b3a394a5f8aa33ad17b11da34 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:21.286841 14820 tablet_replica.cc:331] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:21.286970 14820 raft_consensus.cc:2238] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:21.287125 14820 raft_consensus.cc:2267] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:21.287132 14358 leader_election.cc:343] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.287266 14391 tablet_service.cc:1514] Processing DeleteTablet for tablet d891020ed0bb4fc49155bd9f2f6ed35c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.287426 14820 ts_tablet_manager.cc:1905] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.287456 14819 ts_tablet_manager.cc:1918] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:21.287520 14819 log.cc:1198] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/d4f76d0b3a394a5f8aa33ad17b11da34
I20250115 03:06:21.287842 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5db9d5f6cecb454a90ad931d8d165bcc" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.288795 14526 tablet_service.cc:1514] Processing DeleteTablet for tablet 85cceb5a3f234d85937ad0bb75348991 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.289045 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet b8ec24d2937d44e79b839276bf615fa0 with delete_type TABLET_DATA_DELETED (replaced by tablet 4febfae2f2794669b5997a1fa7ef7412) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.289206 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 80ede4e9696d491aa385b375fcb5cf3b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:16 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.289538 14819 ts_tablet_manager.cc:1939] T d4f76d0b3a394a5f8aa33ad17b11da34 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.290112 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet d4f76d0b3a394a5f8aa33ad17b11da34 (table default.Table_a6170a1e9f9546abb2af9ba24456080d [id=488a9a6268f64669ae17e1a67caa73a5]) successfully deleted
W20250115 03:06:21.290666 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 85fcafa24fa1448c995d30dd93b0737f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 85fcafa24fa1448c995d30dd93b0737f already in progress: creating tablet
I20250115 03:06:21.290894 14819 tablet_replica.cc:331] T 6ed47b26cdf744acb28fb657a1b41087 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.291041 14819 raft_consensus.cc:2238] T 6ed47b26cdf744acb28fb657a1b41087 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.291420 14820 ts_tablet_manager.cc:1918] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20250115 03:06:21.291491 14820 log.cc:1198] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/d4f76d0b3a394a5f8aa33ad17b11da34
I20250115 03:06:21.291738 14820 ts_tablet_manager.cc:1939] T d4f76d0b3a394a5f8aa33ad17b11da34 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
W20250115 03:06:21.291719 14356 leader_election.cc:343] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.291783 14356 leader_election.cc:304] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.291939 16406 raft_consensus.cc:2743] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.292007 14820 tablet_replica.cc:331] T 85cceb5a3f234d85937ad0bb75348991 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:21.292110 14820 raft_consensus.cc:2238] T 85cceb5a3f234d85937ad0bb75348991 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.292155 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet c2afa048b3fd4362b50c8a6da306c25a with delete_type TABLET_DATA_DELETED (replaced by tablet 6501c82435154d6e8f2c12b7c82da296) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.292184 14820 raft_consensus.cc:2267] T 85cceb5a3f234d85937ad0bb75348991 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.292276 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet d4f76d0b3a394a5f8aa33ad17b11da34 (table default.Table_a6170a1e9f9546abb2af9ba24456080d [id=488a9a6268f64669ae17e1a67caa73a5]) successfully deleted
I20250115 03:06:21.294193 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet a4547465f3124e108dc2c4c54e2bc751 (table default.Table_58e0901d0ec94ef9859515f126825621 [id=50c73c26b4ed47a4961348db433b7dd4]). Expected version 3 got 0
I20250115 03:06:21.295172 14819 raft_consensus.cc:2267] T 6ed47b26cdf744acb28fb657a1b41087 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.295174 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet d4f76d0b3a394a5f8aa33ad17b11da34 (table default.Table_a6170a1e9f9546abb2af9ba24456080d [id=488a9a6268f64669ae17e1a67caa73a5]) successfully deleted
I20250115 03:06:21.295542 14809 tablet_bootstrap.cc:492] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.295627 14809 ts_tablet_manager.cc:1397] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20250115 03:06:21.296046 14809 raft_consensus.cc:357] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.296144 14809 raft_consensus.cc:383] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.296178 14809 raft_consensus.cc:738] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.296280 14809 consensus_queue.cc:260] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.296676 14809 ts_tablet_manager.cc:1428] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.296945 14819 ts_tablet_manager.cc:1905] T 6ed47b26cdf744acb28fb657a1b41087 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.297004 14820 ts_tablet_manager.cc:1905] T 85cceb5a3f234d85937ad0bb75348991 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.297577 14812 tablet_bootstrap.cc:492] T 0792e52d171446b48a7c9d1f03d37a79 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.297662 14812 ts_tablet_manager.cc:1397] T 0792e52d171446b48a7c9d1f03d37a79 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.054s user 0.001s sys 0.001s
I20250115 03:06:21.298110 14809 tablet_bootstrap.cc:492] T ca117aca95f640be880b640c85e18cd4 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.298133 16411 raft_consensus.cc:491] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.298098 14812 raft_consensus.cc:357] T 0792e52d171446b48a7c9d1f03d37a79 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.298190 14812 raft_consensus.cc:383] T 0792e52d171446b48a7c9d1f03d37a79 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.298220 14812 raft_consensus.cc:738] T 0792e52d171446b48a7c9d1f03d37a79 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.298197 16411 raft_consensus.cc:513] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.298336 14812 consensus_queue.cc:260] T 0792e52d171446b48a7c9d1f03d37a79 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.298538 16411 leader_election.cc:290] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.298593 14812 ts_tablet_manager.cc:1428] T 0792e52d171446b48a7c9d1f03d37a79 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.298760 14812 tablet_bootstrap.cc:492] T a4547465f3124e108dc2c4c54e2bc751 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.298954 14809 tablet_bootstrap.cc:654] T ca117aca95f640be880b640c85e18cd4 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.299393 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d891020ed0bb4fc49155bd9f2f6ed35c" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.299589 14812 tablet_bootstrap.cc:654] T a4547465f3124e108dc2c4c54e2bc751 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:21.299705 14358 leader_election.cc:343] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.299916 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2e6e02769ab14002924b5cc60596fddc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.300243 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2e6e02769ab14002924b5cc60596fddc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.300709 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d891020ed0bb4fc49155bd9f2f6ed35c" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:21.301133 14356 leader_election.cc:343] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.301191 14356 leader_election.cc:304] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.301455 16411 raft_consensus.cc:2743] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.301649 14820 ts_tablet_manager.cc:1918] T 85cceb5a3f234d85937ad0bb75348991 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.301707 14820 log.cc:1198] T 85cceb5a3f234d85937ad0bb75348991 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/85cceb5a3f234d85937ad0bb75348991
I20250115 03:06:21.301879 14820 ts_tablet_manager.cc:1939] T 85cceb5a3f234d85937ad0bb75348991 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:21.302726 16189 raft_consensus.cc:491] T 676ec8b9dc8241d4ae40488d3ce8f61e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.302817 16189 raft_consensus.cc:513] T 676ec8b9dc8241d4ae40488d3ce8f61e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.302955 14820 tablet_replica.cc:331] T b8ec24d2937d44e79b839276bf615fa0 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:21.303048 14820 raft_consensus.cc:2238] T b8ec24d2937d44e79b839276bf615fa0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.303125 14820 raft_consensus.cc:2267] T b8ec24d2937d44e79b839276bf615fa0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.303298 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 85cceb5a3f234d85937ad0bb75348991 (table default.Table_eec974cf89de4f45930dbf47fa754f2a [id=36660de3890e411e974c41b7f8f85946]) successfully deleted
I20250115 03:06:21.303128 16189 leader_election.cc:290] T 676ec8b9dc8241d4ae40488d3ce8f61e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.303642 14820 ts_tablet_manager.cc:1905] T b8ec24d2937d44e79b839276bf615fa0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.303994 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676ec8b9dc8241d4ae40488d3ce8f61e" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.305222 14811 tablet_bootstrap.cc:492] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.305305 14811 ts_tablet_manager.cc:1397] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.038s user 0.000s sys 0.001s
I20250115 03:06:21.305691 14811 raft_consensus.cc:357] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.305794 14811 raft_consensus.cc:383] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.305830 14811 raft_consensus.cc:738] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.305981 14811 consensus_queue.cc:260] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.306247 14811 ts_tablet_manager.cc:1428] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:21.306185 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_89657e3d79dc42d182839fa60ce9190a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:21.306661 14811 tablet_bootstrap.cc:492] T 0f20bb66f36647e0811d0865b8f414f7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.307379 14812 tablet_bootstrap.cc:492] T a4547465f3124e108dc2c4c54e2bc751 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.307446 14812 ts_tablet_manager.cc:1397] T a4547465f3124e108dc2c4c54e2bc751 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:21.307521 14811 tablet_bootstrap.cc:654] T 0f20bb66f36647e0811d0865b8f414f7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.307866 14165 master_service.cc:946] ReplaceTablet: received request to replace tablet 015ec42caa1b468ea10f25c8f26e278a from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:21.307870 14812 raft_consensus.cc:357] T a4547465f3124e108dc2c4c54e2bc751 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.307904 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "676ec8b9dc8241d4ae40488d3ce8f61e" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.307952 14812 raft_consensus.cc:383] T a4547465f3124e108dc2c4c54e2bc751 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.307984 14812 raft_consensus.cc:738] T a4547465f3124e108dc2c4c54e2bc751 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.308022 14422 raft_consensus.cc:2388] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in current term 1: Already voted for candidate 5c0dae7aff4b44c699275b95ba639770 in this term.
I20250115 03:06:21.308086 14812 consensus_queue.cc:260] T a4547465f3124e108dc2c4c54e2bc751 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.308328 14812 ts_tablet_manager.cc:1428] T a4547465f3124e108dc2c4c54e2bc751 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:21.308595 14812 tablet_bootstrap.cc:492] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.309342 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5a2894e4736b4191963a61d908bc673c (table default.Table_df79ed42f26f4b68b197877c445b9665 [id=bb6453f2ab97404cbd3ba32a87976e52]). Expected version 1 got 0
W20250115 03:06:21.309778 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_89657e3d79dc42d182839fa60ce9190a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.311105 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5e774c70bcb0426fb48e52c32ce75a4c" } modify_external_catalogs: true
I20250115 03:06:21.311201 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5e774c70bcb0426fb48e52c32ce75a4c" } modify_external_catalogs: true
I20250115 03:06:21.311303 14820 ts_tablet_manager.cc:1918] T b8ec24d2937d44e79b839276bf615fa0 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.311362 14820 log.cc:1198] T b8ec24d2937d44e79b839276bf615fa0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/b8ec24d2937d44e79b839276bf615fa0
I20250115 03:06:21.311545 14820 ts_tablet_manager.cc:1939] T b8ec24d2937d44e79b839276bf615fa0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:21.311859 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet b8ec24d2937d44e79b839276bf615fa0 (table default.Table_83fe43c0616b4351b2b4457195378f0c [id=7bec6149db914c939bc6fd50e47097b2]) successfully deleted
I20250115 03:06:21.311985 14164 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet f2ca06c402cd46dc8311c8a47da92846 (table default.Table_802f667f1ad246688cbdcffcb3001021 [id=e929ffaba567498c815393836b264a12]). Expected version 1 got 0
I20250115 03:06:21.312040 14820 tablet_replica.cc:331] T 80ede4e9696d491aa385b375fcb5cf3b P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:21.312147 14820 raft_consensus.cc:2238] T 80ede4e9696d491aa385b375fcb5cf3b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.312224 14820 raft_consensus.cc:2267] T 80ede4e9696d491aa385b375fcb5cf3b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.312559 14493 leader_election.cc:304] T 676ec8b9dc8241d4ae40488d3ce8f61e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:21.312764 16189 raft_consensus.cc:3054] T 676ec8b9dc8241d4ae40488d3ce8f61e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.313298 14820 ts_tablet_manager.cc:1905] T 80ede4e9696d491aa385b375fcb5cf3b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.313666 16189 raft_consensus.cc:2743] T 676ec8b9dc8241d4ae40488d3ce8f61e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.314705 14811 tablet_bootstrap.cc:492] T 0f20bb66f36647e0811d0865b8f414f7 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.314779 14811 ts_tablet_manager.cc:1397] T 0f20bb66f36647e0811d0865b8f414f7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20250115 03:06:21.315510 14812 tablet_bootstrap.cc:654] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.316901 14811 raft_consensus.cc:357] T 0f20bb66f36647e0811d0865b8f414f7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.317027 14811 raft_consensus.cc:383] T 0f20bb66f36647e0811d0865b8f414f7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.317060 14811 raft_consensus.cc:738] T 0f20bb66f36647e0811d0865b8f414f7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.317251 14811 consensus_queue.cc:260] T 0f20bb66f36647e0811d0865b8f414f7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.317589 14811 ts_tablet_manager.cc:1428] T 0f20bb66f36647e0811d0865b8f414f7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20250115 03:06:21.317782 14811 tablet_bootstrap.cc:492] T 7b86e66428504217af08f667f660d304 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.318301 14812 tablet_bootstrap.cc:492] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.318378 14812 ts_tablet_manager.cc:1397] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.001s
I20250115 03:06:21.318791 14812 raft_consensus.cc:357] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.318874 14812 raft_consensus.cc:383] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.318902 14812 raft_consensus.cc:738] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.318993 14812 consensus_queue.cc:260] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.319224 14812 ts_tablet_manager.cc:1428] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.319360 14812 tablet_bootstrap.cc:492] T 815ef85ce5cf4ae0a56c5734837e03ba P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.319394 14811 tablet_bootstrap.cc:654] T 7b86e66428504217af08f667f660d304 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.319572 14164 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet f2ca06c402cd46dc8311c8a47da92846 (table default.Table_802f667f1ad246688cbdcffcb3001021 [id=e929ffaba567498c815393836b264a12]). Expected version 1 got 0
I20250115 03:06:21.320181 14812 tablet_bootstrap.cc:654] T 815ef85ce5cf4ae0a56c5734837e03ba P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.320689 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_c1a47c048ec54b68aecd90dd20ea1ea5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:21.320904 14820 ts_tablet_manager.cc:1918] T 80ede4e9696d491aa385b375fcb5cf3b P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.320995 14820 log.cc:1198] T 80ede4e9696d491aa385b375fcb5cf3b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/80ede4e9696d491aa385b375fcb5cf3b
W20250115 03:06:21.321156 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c1a47c048ec54b68aecd90dd20ea1ea5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.321180 14820 ts_tablet_manager.cc:1939] T 80ede4e9696d491aa385b375fcb5cf3b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:21.321408 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet a528e049654e4eb78f7d03d8c72ef21d
I20250115 03:06:21.321521 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 80ede4e9696d491aa385b375fcb5cf3b (table default.Table_75cd7b01fc534652975eb593912b88bc [id=111fc67094164d3f99965c246ae12b24]) successfully deleted
I20250115 03:06:21.321961 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet a528e049654e4eb78f7d03d8c72ef21d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.322110 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet a528e049654e4eb78f7d03d8c72ef21d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.322747 14811 tablet_bootstrap.cc:492] T 7b86e66428504217af08f667f660d304 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.322830 14811 ts_tablet_manager.cc:1397] T 7b86e66428504217af08f667f660d304 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
W20250115 03:06:21.322937 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet a528e049654e4eb78f7d03d8c72ef21d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a528e049654e4eb78f7d03d8c72ef21d already in progress: creating tablet
I20250115 03:06:21.323220 14811 raft_consensus.cc:357] T 7b86e66428504217af08f667f660d304 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.323314 14811 raft_consensus.cc:383] T 7b86e66428504217af08f667f660d304 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.323336 14812 tablet_bootstrap.cc:492] T 815ef85ce5cf4ae0a56c5734837e03ba P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.323343 14811 raft_consensus.cc:738] T 7b86e66428504217af08f667f660d304 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.323356 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet a528e049654e4eb78f7d03d8c72ef21d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.323407 14812 ts_tablet_manager.cc:1397] T 815ef85ce5cf4ae0a56c5734837e03ba P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:21.323426 14811 consensus_queue.cc:260] T 7b86e66428504217af08f667f660d304 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.323674 14811 ts_tablet_manager.cc:1428] T 7b86e66428504217af08f667f660d304 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:21.323818 14811 tablet_bootstrap.cc:492] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.323797 14812 raft_consensus.cc:357] T 815ef85ce5cf4ae0a56c5734837e03ba P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.323881 14812 raft_consensus.cc:383] T 815ef85ce5cf4ae0a56c5734837e03ba P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.323908 14812 raft_consensus.cc:738] T 815ef85ce5cf4ae0a56c5734837e03ba P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.324007 14812 consensus_queue.cc:260] T 815ef85ce5cf4ae0a56c5734837e03ba P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.324246 14812 ts_tablet_manager.cc:1428] T 815ef85ce5cf4ae0a56c5734837e03ba P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.324378 14812 tablet_bootstrap.cc:492] T f2ca06c402cd46dc8311c8a47da92846 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.324612 14811 tablet_bootstrap.cc:654] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.325225 14812 tablet_bootstrap.cc:654] T f2ca06c402cd46dc8311c8a47da92846 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.326499 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_a96d43a0d2eb4f5c86b28defd2c3de4f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:21.326642 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f92458688aee4eaf9c942f6778309a2b" } new_table_name: "default.Table_fe370dfcf3f942ecb770ab59fd23d25c" modify_external_catalogs: true
W20250115 03:06:21.326804 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a96d43a0d2eb4f5c86b28defd2c3de4f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.327687 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5a9c638d77a84fefb016a56854ac897f" } new_table_name: "default.Table_6c2031ba4104492f959f905cab4861dc" modify_external_catalogs: true
I20250115 03:06:21.327747 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_544b91f835734146acd099765f743a9d" } new_table_name: "default.Table_a56fd016ce294130baac03a9ef333cf8" modify_external_catalogs: true
I20250115 03:06:21.328251 14812 tablet_bootstrap.cc:492] T f2ca06c402cd46dc8311c8a47da92846 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.328318 14812 ts_tablet_manager.cc:1397] T f2ca06c402cd46dc8311c8a47da92846 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:21.328742 14812 raft_consensus.cc:357] T f2ca06c402cd46dc8311c8a47da92846 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.328836 14812 raft_consensus.cc:383] T f2ca06c402cd46dc8311c8a47da92846 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.328871 14812 raft_consensus.cc:738] T f2ca06c402cd46dc8311c8a47da92846 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.328994 14812 consensus_queue.cc:260] T f2ca06c402cd46dc8311c8a47da92846 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.329257 14812 ts_tablet_manager.cc:1428] T f2ca06c402cd46dc8311c8a47da92846 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.329411 14812 tablet_bootstrap.cc:492] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:21.329896 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet a528e049654e4eb78f7d03d8c72ef21d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a528e049654e4eb78f7d03d8c72ef21d already in progress: creating tablet
I20250115 03:06:21.330392 14812 tablet_bootstrap.cc:654] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.331027 14819 ts_tablet_manager.cc:1918] T 6ed47b26cdf744acb28fb657a1b41087 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.331090 14819 log.cc:1198] T 6ed47b26cdf744acb28fb657a1b41087 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/6ed47b26cdf744acb28fb657a1b41087
I20250115 03:06:21.331275 14819 ts_tablet_manager.cc:1939] T 6ed47b26cdf744acb28fb657a1b41087 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.332280 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 6ed47b26cdf744acb28fb657a1b41087 (table default.Table_4f5d0494ec384c479419c10b7ff36dc0 [id=b9026c7e60a14240b2f5c596d31ba517]) successfully deleted
I20250115 03:06:21.332877 14812 tablet_bootstrap.cc:492] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.333012 14812 ts_tablet_manager.cc:1397] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250115 03:06:21.333448 14812 raft_consensus.cc:357] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.333547 14812 raft_consensus.cc:383] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.333575 14812 raft_consensus.cc:738] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.333702 14812 consensus_queue.cc:260] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.334035 14812 ts_tablet_manager.cc:1428] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.334862 16189 raft_consensus.cc:491] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.334944 16189 raft_consensus.cc:513] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.335183 14812 tablet_bootstrap.cc:492] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.335280 16189 leader_election.cc:290] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.335641 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2ca06c402cd46dc8311c8a47da92846" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.335753 14689 raft_consensus.cc:2463] T f2ca06c402cd46dc8311c8a47da92846 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.336062 14493 leader_election.cc:304] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.336267 16189 raft_consensus.cc:2798] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.336323 16189 raft_consensus.cc:491] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.336360 16189 raft_consensus.cc:3054] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.337245 16189 raft_consensus.cc:513] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.337533 16189 leader_election.cc:290] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.338084 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_6df122bc9bdc45aea95cb4f07ee90dfa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:21.338129 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2ca06c402cd46dc8311c8a47da92846" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:21.338236 14689 raft_consensus.cc:3054] T f2ca06c402cd46dc8311c8a47da92846 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:21.338388 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6df122bc9bdc45aea95cb4f07ee90dfa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.339114 14812 tablet_bootstrap.cc:654] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.339181 14689 raft_consensus.cc:2463] T f2ca06c402cd46dc8311c8a47da92846 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.339869 14493 leader_election.cc:304] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.340049 16189 raft_consensus.cc:2798] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.335565 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2ca06c402cd46dc8311c8a47da92846" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.340767 14422 raft_consensus.cc:2463] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.339067 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2ca06c402cd46dc8311c8a47da92846" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.341019 14419 raft_consensus.cc:3054] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.341924 14419 raft_consensus.cc:2463] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.332477 14819 tablet_replica.cc:331] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.342156 14819 raft_consensus.cc:2238] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.342240 14819 raft_consensus.cc:2267] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:21.342546 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fe0dd7895a6547e8a105492083f2ec22 consensus state has no leader
I20250115 03:06:21.342912 14811 tablet_bootstrap.cc:492] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.342981 14811 ts_tablet_manager.cc:1397] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20250115 03:06:21.343398 14811 raft_consensus.cc:357] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.343518 14811 raft_consensus.cc:383] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.343567 14811 raft_consensus.cc:738] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.343672 14811 consensus_queue.cc:260] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.343956 14811 ts_tablet_manager.cc:1428] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.344105 14811 tablet_bootstrap.cc:492] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.344527 16411 raft_consensus.cc:491] T fca14a7d574842ee94a93afb85ca1afd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.344597 16411 raft_consensus.cc:513] T fca14a7d574842ee94a93afb85ca1afd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.344890 16411 leader_election.cc:290] T fca14a7d574842ee94a93afb85ca1afd P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.345145 16411 raft_consensus.cc:491] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.345220 16411 raft_consensus.cc:513] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.345501 16411 leader_election.cc:290] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.345876 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b08e5fc2110e476e897240f9cf3b874d" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:21.347251 14358 leader_election.cc:343] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.347798 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca14a7d574842ee94a93afb85ca1afd" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:21.348099 14358 leader_election.cc:343] T fca14a7d574842ee94a93afb85ca1afd P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:21.348408 14167 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet beb2fd90c41f4841bfa2147bdcd3e017 consensus state has no leader
I20250115 03:06:21.349325 16189 raft_consensus.cc:695] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:21.349391 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca14a7d574842ee94a93afb85ca1afd" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.349454 16189 consensus_queue.cc:237] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.349611 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b08e5fc2110e476e897240f9cf3b874d" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.352087 15153 raft_consensus.cc:491] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.352166 15153 raft_consensus.cc:513] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.352512 15153 leader_election.cc:290] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:21.352793 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f20bb66f36647e0811d0865b8f414f7" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.352797 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f20bb66f36647e0811d0865b8f414f7" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.352887 14422 raft_consensus.cc:2463] T 0f20bb66f36647e0811d0865b8f414f7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:21.352897 14552 raft_consensus.cc:2463] T 0f20bb66f36647e0811d0865b8f414f7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:21.353154 14628 leader_election.cc:304] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:21.353333 15153 raft_consensus.cc:2798] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.353382 15153 raft_consensus.cc:491] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.353418 15153 raft_consensus.cc:3054] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.353772 14819 ts_tablet_manager.cc:1905] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.354317 15153 raft_consensus.cc:513] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.354642 15153 leader_election.cc:290] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:21.354893 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f20bb66f36647e0811d0865b8f414f7" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:21.354980 14552 raft_consensus.cc:3054] T 0f20bb66f36647e0811d0865b8f414f7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.354967 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f20bb66f36647e0811d0865b8f414f7" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.355062 14422 raft_consensus.cc:3054] T 0f20bb66f36647e0811d0865b8f414f7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.355495 14809 tablet_bootstrap.cc:492] T ca117aca95f640be880b640c85e18cd4 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.355580 14809 ts_tablet_manager.cc:1397] T ca117aca95f640be880b640c85e18cd4 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.058s user 0.001s sys 0.000s
I20250115 03:06:21.355885 14552 raft_consensus.cc:2463] T 0f20bb66f36647e0811d0865b8f414f7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:21.355949 14422 raft_consensus.cc:2463] T 0f20bb66f36647e0811d0865b8f414f7 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:21.356026 14809 raft_consensus.cc:357] T ca117aca95f640be880b640c85e18cd4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.356115 14809 raft_consensus.cc:383] T ca117aca95f640be880b640c85e18cd4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.356145 14809 raft_consensus.cc:738] T ca117aca95f640be880b640c85e18cd4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.356237 14628 leader_election.cc:304] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
W20250115 03:06:21.356233 14356 leader_election.cc:343] T fca14a7d574842ee94a93afb85ca1afd P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.356303 14356 leader_election.cc:304] T fca14a7d574842ee94a93afb85ca1afd P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.355942 14819 ts_tablet_manager.cc:1918] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.356427 14819 log.cc:1198] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/d891020ed0bb4fc49155bd9f2f6ed35c
W20250115 03:06:21.356520 14356 leader_election.cc:343] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.356567 14356 leader_election.cc:304] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.356563 16001 raft_consensus.cc:2798] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.356612 14819 ts_tablet_manager.cc:1939] T d891020ed0bb4fc49155bd9f2f6ed35c P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.356637 16001 raft_consensus.cc:695] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:21.356683 16406 raft_consensus.cc:2743] T fca14a7d574842ee94a93afb85ca1afd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.356747 16411 raft_consensus.cc:2743] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.356739 16001 consensus_queue.cc:237] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.357199 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet d891020ed0bb4fc49155bd9f2f6ed35c (table default.Table_73f837db71d74dae9606eb6995ad7270 [id=ff302d72ca26434f9a86f478f3346318]) successfully deleted
I20250115 03:06:21.358772 14819 tablet_replica.cc:331] T c2afa048b3fd4362b50c8a6da306c25a P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.358891 14819 raft_consensus.cc:2238] T c2afa048b3fd4362b50c8a6da306c25a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.356248 14809 consensus_queue.cc:260] T ca117aca95f640be880b640c85e18cd4 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.358964 14819 raft_consensus.cc:2267] T c2afa048b3fd4362b50c8a6da306c25a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.359119 14809 ts_tablet_manager.cc:1428] T ca117aca95f640be880b640c85e18cd4 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:21.359280 14809 tablet_bootstrap.cc:492] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.360008 14819 ts_tablet_manager.cc:1905] T c2afa048b3fd4362b50c8a6da306c25a P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.360146 14809 tablet_bootstrap.cc:654] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.361111 14809 tablet_bootstrap.cc:492] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.361191 14809 ts_tablet_manager.cc:1397] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:21.361552 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4b1d421298a845129d33322313978924" } new_table_name: "default.Table_0654e6fa86284deab15640918a465fbc" modify_external_catalogs: true
I20250115 03:06:21.362329 14166 master_service.cc:946] ReplaceTablet: received request to replace tablet a528e049654e4eb78f7d03d8c72ef21d from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:21.362684 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_86a356644d764518ab84bb6ff8d67991"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:21.362931 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_27a225477c7141f5a427f30387298506"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.363194 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_27a225477c7141f5a427f30387298506 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:21.362983 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86a356644d764518ab84bb6ff8d67991 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.364218 14819 ts_tablet_manager.cc:1918] T c2afa048b3fd4362b50c8a6da306c25a P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.364272 14819 log.cc:1198] T c2afa048b3fd4362b50c8a6da306c25a P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/c2afa048b3fd4362b50c8a6da306c25a
I20250115 03:06:21.364442 14819 ts_tablet_manager.cc:1939] T c2afa048b3fd4362b50c8a6da306c25a P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.364401 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e72d1ee69400410c9da9082d4bd8bc9a" } new_table_name: "default.Table_013cb9366ef94a75a90ef4dfed00ff41" modify_external_catalogs: true
I20250115 03:06:21.364970 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet c2afa048b3fd4362b50c8a6da306c25a (table default.Table_c95d9b4b8beb4ebea3e620d35b56bd40 [id=74853e82e7024da6a9e4e1dd23bd043d]) successfully deleted
W20250115 03:06:21.365334 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet a528e049654e4eb78f7d03d8c72ef21d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a528e049654e4eb78f7d03d8c72ef21d already in progress: creating tablet
I20250115 03:06:21.367007 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ed091b2f479b4435bd71157b60d837e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.367270 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed091b2f479b4435bd71157b60d837e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.368299 14171 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5a2894e4736b4191963a61d908bc673c (table default.Table_df79ed42f26f4b68b197877c445b9665 [id=bb6453f2ab97404cbd3ba32a87976e52]). Expected version 1 got 0
I20250115 03:06:21.368659 14171 master_service.cc:946] ReplaceTablet: received request to replace tablet bb68869eaf55477090cf12f40f76a89c from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:21.369808 14165 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 0f20bb66f36647e0811d0865b8f414f7 (table default.Table_0326e94494d0447585233c0c907113fd [id=f48964597a90470299a755217c4dbea0]). Expected version 1 got 0
I20250115 03:06:21.370055 14397 tablet_service.cc:1467] Processing CreateTablet for tablet c8c73857d8184dc8a535b562b966ff4f (DEFAULT_TABLE table=default.Table_20607cbd2ddf454bb006aa12bb3eca45 [id=6dfda059b834468b87f459466e6172cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.370290 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8c73857d8184dc8a535b562b966ff4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.370416 14668 tablet_service.cc:1467] Processing CreateTablet for tablet c8c73857d8184dc8a535b562b966ff4f (DEFAULT_TABLE table=default.Table_20607cbd2ddf454bb006aa12bb3eca45 [id=6dfda059b834468b87f459466e6172cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.370550 14170 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 0f20bb66f36647e0811d0865b8f414f7 (table default.Table_0326e94494d0447585233c0c907113fd [id=f48964597a90470299a755217c4dbea0]). Expected version 1 got 0
I20250115 03:06:21.370630 14811 tablet_bootstrap.cc:654] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.370651 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8c73857d8184dc8a535b562b966ff4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.370658 14170 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 15fbf8d5de904fc4b284af664f3bd60f (table default.Table_655e9d02aacf4b88ba521f0f57a5f522 [id=6ed01835736b40f8b4caeca7378770d8]). Expected version 1 got 0
I20250115 03:06:21.370725 14170 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet f2ca06c402cd46dc8311c8a47da92846 (table default.Table_802f667f1ad246688cbdcffcb3001021 [id=e929ffaba567498c815393836b264a12]). Expected version 1 got 0
I20250115 03:06:21.370790 14170 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet a4547465f3124e108dc2c4c54e2bc751 (table default.Table_58e0901d0ec94ef9859515f126825621 [id=50c73c26b4ed47a4961348db433b7dd4]). Expected version 3 got 0
I20250115 03:06:21.371235 14530 tablet_service.cc:1467] Processing CreateTablet for tablet c8c73857d8184dc8a535b562b966ff4f (DEFAULT_TABLE table=default.Table_20607cbd2ddf454bb006aa12bb3eca45 [id=6dfda059b834468b87f459466e6172cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.371315 14655 tablet_service.cc:1467] Processing CreateTablet for tablet a142f9f7321a484ba7fde981ecc417e4 (DEFAULT_TABLE table=default.Table_47fd03c657214fea893d4881cbbff5f1 [id=b4b9e2dd19f1416ca06cd5f53ffa5109]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.371484 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8c73857d8184dc8a535b562b966ff4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.371495 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a142f9f7321a484ba7fde981ecc417e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.372392 14391 tablet_service.cc:1467] Processing CreateTablet for tablet a142f9f7321a484ba7fde981ecc417e4 (DEFAULT_TABLE table=default.Table_47fd03c657214fea893d4881cbbff5f1 [id=b4b9e2dd19f1416ca06cd5f53ffa5109]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.372606 14391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a142f9f7321a484ba7fde981ecc417e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.373279 14533 tablet_service.cc:1467] Processing CreateTablet for tablet a142f9f7321a484ba7fde981ecc417e4 (DEFAULT_TABLE table=default.Table_47fd03c657214fea893d4881cbbff5f1 [id=b4b9e2dd19f1416ca06cd5f53ffa5109]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.373499 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a142f9f7321a484ba7fde981ecc417e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.375196 14656 tablet_service.cc:1514] Processing DeleteTablet for tablet 0792e52d171446b48a7c9d1f03d37a79 with delete_type TABLET_DATA_DELETED (replaced by tablet d65bf3745b4f4d92a346b4dd18652432) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.376142 14823 tablet_replica.cc:331] T 0792e52d171446b48a7c9d1f03d37a79 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:21.376279 14823 raft_consensus.cc:2238] T 0792e52d171446b48a7c9d1f03d37a79 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.376377 14823 raft_consensus.cc:2267] T 0792e52d171446b48a7c9d1f03d37a79 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.376909 14823 ts_tablet_manager.cc:1905] T 0792e52d171446b48a7c9d1f03d37a79 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.378300 14809 raft_consensus.cc:357] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.378396 14809 raft_consensus.cc:383] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.378424 14809 raft_consensus.cc:738] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.378517 14809 consensus_queue.cc:260] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.379084 14823 ts_tablet_manager.cc:1918] T 0792e52d171446b48a7c9d1f03d37a79 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.379153 14823 log.cc:1198] T 0792e52d171446b48a7c9d1f03d37a79 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/0792e52d171446b48a7c9d1f03d37a79
I20250115 03:06:21.379339 14823 ts_tablet_manager.cc:1939] T 0792e52d171446b48a7c9d1f03d37a79 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:21.379855 14809 ts_tablet_manager.cc:1428] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.019s user 0.001s sys 0.000s
I20250115 03:06:21.380786 14809 tablet_bootstrap.cc:492] T b61cc4b11251486a8089ca9bf44ac91e P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.380864 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 0792e52d171446b48a7c9d1f03d37a79 (table default.Table_72e82b3e776e4d398fa5d8240afde936 [id=17ae925a7c0942d59058b2fd619ccb45]) successfully deleted
I20250115 03:06:21.384126 15617 tablet.cc:1721] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:21.385123 16265 tablet.cc:1721] T dcdb5282e41c43dbb235ae78465d19f9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:21.386348 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2fabe46e26c64fdf9c13c69f53a754f4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.386626 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2fabe46e26c64fdf9c13c69f53a754f4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.387506 16411 raft_consensus.cc:491] T 7f9818d0493547358fd5654a01d4f456 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.387574 16411 raft_consensus.cc:513] T 7f9818d0493547358fd5654a01d4f456 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.387888 16411 leader_election.cc:290] T 7f9818d0493547358fd5654a01d4f456 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.387977 15970 tablet.cc:1721] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:21.388487 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9818d0493547358fd5654a01d4f456" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.390344 14812 tablet_bootstrap.cc:492] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.390419 14812 ts_tablet_manager.cc:1397] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.055s user 0.002s sys 0.000s
I20250115 03:06:21.390839 14812 raft_consensus.cc:357] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.390934 14812 raft_consensus.cc:383] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.390965 14812 raft_consensus.cc:738] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.391053 14812 consensus_queue.cc:260] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.391571 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9818d0493547358fd5654a01d4f456" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:21.392140 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d5e4743e016e49ed8ba26b1ac96b1470 consensus state has no leader
W20250115 03:06:21.395320 14358 leader_election.cc:343] T 7f9818d0493547358fd5654a01d4f456 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.395382 14811 tablet_bootstrap.cc:492] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.395450 14811 ts_tablet_manager.cc:1397] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.051s user 0.002s sys 0.000s
I20250115 03:06:21.395467 14812 ts_tablet_manager.cc:1428] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:21.395619 15970 tablet.cc:1721] T e63e854c49b34c1ab286f1f97fe443ba P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:21.395866 14811 raft_consensus.cc:357] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.395953 14811 raft_consensus.cc:383] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.395982 14811 raft_consensus.cc:738] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.396006 15617 tablet.cc:1721] T e63e854c49b34c1ab286f1f97fe443ba P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:21.396087 14811 consensus_queue.cc:260] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.396335 14811 ts_tablet_manager.cc:1428] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.397150 14812 tablet_bootstrap.cc:492] T 4122c97f5ba445ee8c79b7d056438229 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.400359 14812 tablet_bootstrap.cc:654] T 4122c97f5ba445ee8c79b7d056438229 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.400885 16265 tablet.cc:1721] T e63e854c49b34c1ab286f1f97fe443ba P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:21.401257 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ef1516de522f4c0aa275c9752c4fe452"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.401539 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ef1516de522f4c0aa275c9752c4fe452 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:21.401705 14356 leader_election.cc:343] T 7f9818d0493547358fd5654a01d4f456 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.401777 14356 leader_election.cc:304] T 7f9818d0493547358fd5654a01d4f456 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.401930 16411 raft_consensus.cc:2743] T 7f9818d0493547358fd5654a01d4f456 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.406651 14811 tablet_bootstrap.cc:492] T 998a515113d04034928c04dc5798a698 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.407533 14811 tablet_bootstrap.cc:654] T 998a515113d04034928c04dc5798a698 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.407932 16455 tablet.cc:1721] T b56279bb16054869a97006d88574e381 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:21.408802 14811 tablet_bootstrap.cc:492] T 998a515113d04034928c04dc5798a698 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.408878 14811 ts_tablet_manager.cc:1397] T 998a515113d04034928c04dc5798a698 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:21.409390 14811 raft_consensus.cc:357] T 998a515113d04034928c04dc5798a698 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.409488 14811 raft_consensus.cc:383] T 998a515113d04034928c04dc5798a698 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.409520 14811 raft_consensus.cc:738] T 998a515113d04034928c04dc5798a698 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.409572 14391 tablet_service.cc:1514] Processing DeleteTablet for tablet ca117aca95f640be880b640c85e18cd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.409622 14811 consensus_queue.cc:260] T 998a515113d04034928c04dc5798a698 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.409898 14811 ts_tablet_manager.cc:1428] T 998a515113d04034928c04dc5798a698 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.410044 14811 tablet_bootstrap.cc:492] T 7043616bd6b34299862336924a6f16bf P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.410769 15970 tablet.cc:1721] T b56279bb16054869a97006d88574e381 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:21.410851 14811 tablet_bootstrap.cc:654] T 7043616bd6b34299862336924a6f16bf P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.413975 16265 tablet.cc:1721] T b56279bb16054869a97006d88574e381 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:21.414147 14819 tablet_replica.cc:331] T ca117aca95f640be880b640c85e18cd4 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.414263 14819 raft_consensus.cc:2238] T ca117aca95f640be880b640c85e18cd4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.414345 14819 raft_consensus.cc:2267] T ca117aca95f640be880b640c85e18cd4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.414959 14811 tablet_bootstrap.cc:492] T 7043616bd6b34299862336924a6f16bf P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.415026 14811 ts_tablet_manager.cc:1397] T 7043616bd6b34299862336924a6f16bf P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20250115 03:06:21.415431 14811 raft_consensus.cc:357] T 7043616bd6b34299862336924a6f16bf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.415541 14811 raft_consensus.cc:383] T 7043616bd6b34299862336924a6f16bf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.415573 14811 raft_consensus.cc:738] T 7043616bd6b34299862336924a6f16bf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.415676 14811 consensus_queue.cc:260] T 7043616bd6b34299862336924a6f16bf P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.415946 14811 ts_tablet_manager.cc:1428] T 7043616bd6b34299862336924a6f16bf P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.418393 14819 ts_tablet_manager.cc:1905] T ca117aca95f640be880b640c85e18cd4 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.418757 14811 tablet_bootstrap.cc:492] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.419741 14811 tablet_bootstrap.cc:654] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.420413 14819 ts_tablet_manager.cc:1918] T ca117aca95f640be880b640c85e18cd4 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.420468 14819 log.cc:1198] T ca117aca95f640be880b640c85e18cd4 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/ca117aca95f640be880b640c85e18cd4
I20250115 03:06:21.420641 14809 tablet_bootstrap.cc:654] T b61cc4b11251486a8089ca9bf44ac91e P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.420656 14819 ts_tablet_manager.cc:1939] T ca117aca95f640be880b640c85e18cd4 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.421612 14811 tablet_bootstrap.cc:492] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.421694 14811 ts_tablet_manager.cc:1397] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:21.422086 14811 raft_consensus.cc:357] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.422180 14811 raft_consensus.cc:383] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.422210 14811 raft_consensus.cc:738] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.422329 14811 consensus_queue.cc:260] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.422580 14811 ts_tablet_manager.cc:1428] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.422739 14811 tablet_bootstrap.cc:492] T d6cf1a0b9c81413b8f72de0cafc76963 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.422822 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet ca117aca95f640be880b640c85e18cd4 (table default.Table_a82405d576ec40228fe5ad55b98753e8 [id=9ce40fc0fe094c2da26bf1941aeaa657]) successfully deleted
I20250115 03:06:21.423548 14811 tablet_bootstrap.cc:654] T d6cf1a0b9c81413b8f72de0cafc76963 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.424476 16411 raft_consensus.cc:491] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.424548 16411 raft_consensus.cc:513] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.424862 16411 leader_election.cc:290] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.427305 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bd4e4200f9b489ea5dd2154aa6f382d" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:21.427634 14356 leader_election.cc:343] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.429502 14166 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 0b5fd91a0ea64b3b85b7571e5daa08b2 (table default.Table_1f92c58a90374663a4c6368b14f71d43 [id=526370a40a0d4f7aa96c8348412d8b26]). Expected version 2 got 0
I20250115 03:06:21.430094 14811 tablet_bootstrap.cc:492] T d6cf1a0b9c81413b8f72de0cafc76963 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.430176 14811 ts_tablet_manager.cc:1397] T d6cf1a0b9c81413b8f72de0cafc76963 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:21.430282 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bd4e4200f9b489ea5dd2154aa6f382d" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.430543 14811 raft_consensus.cc:357] T d6cf1a0b9c81413b8f72de0cafc76963 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:21.430578 14358 leader_election.cc:343] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.430626 14811 raft_consensus.cc:383] T d6cf1a0b9c81413b8f72de0cafc76963 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.430636 14358 leader_election.cc:304] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.430655 14811 raft_consensus.cc:738] T d6cf1a0b9c81413b8f72de0cafc76963 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.430758 14811 consensus_queue.cc:260] T d6cf1a0b9c81413b8f72de0cafc76963 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.430801 16411 raft_consensus.cc:2743] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.430980 14811 ts_tablet_manager.cc:1428] T d6cf1a0b9c81413b8f72de0cafc76963 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.431155 14811 tablet_bootstrap.cc:492] T 38f5554b10a44c249e4c3d3825cd7952 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.434054 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_0654e6fa86284deab15640918a465fbc [id=b298dce7ab664327893633f821ed4d89] alter complete (version 3)
I20250115 03:06:21.434441 14170 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 7b86e66428504217af08f667f660d304 (table default.Table_341a3cd1c8c94ff4b05fc21960a4e9b4 [id=c096ce170874405ea740b9ea829f886e]). Expected version 2 got 0
I20250115 03:06:21.434569 14170 catalog_manager.cc:5526] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:21.434758 14170 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet f2ca06c402cd46dc8311c8a47da92846 (table default.Table_802f667f1ad246688cbdcffcb3001021 [id=e929ffaba567498c815393836b264a12]). Expected version 1 got 0
I20250115 03:06:21.435559 14809 tablet_bootstrap.cc:492] T b61cc4b11251486a8089ca9bf44ac91e P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.435632 14809 ts_tablet_manager.cc:1397] T b61cc4b11251486a8089ca9bf44ac91e P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.055s user 0.002s sys 0.000s
I20250115 03:06:21.436028 14809 raft_consensus.cc:357] T b61cc4b11251486a8089ca9bf44ac91e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.436111 14809 raft_consensus.cc:383] T b61cc4b11251486a8089ca9bf44ac91e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.436137 14809 raft_consensus.cc:738] T b61cc4b11251486a8089ca9bf44ac91e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.436295 14809 consensus_queue.cc:260] T b61cc4b11251486a8089ca9bf44ac91e P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.436661 14809 ts_tablet_manager.cc:1428] T b61cc4b11251486a8089ca9bf44ac91e P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.436806 14809 tablet_bootstrap.cc:492] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.437122 14533 tablet_service.cc:1467] Processing CreateTablet for tablet b438faf62e154afe9aec2c0c44d59ba3 (DEFAULT_TABLE table=default.Table_6df122bc9bdc45aea95cb4f07ee90dfa [id=ba43efe090254ca48a75a4de394b4c9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.437664 14809 tablet_bootstrap.cc:654] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.438632 14809 tablet_bootstrap.cc:492] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.438702 14809 ts_tablet_manager.cc:1397] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:21.439092 14809 raft_consensus.cc:357] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.439198 14809 raft_consensus.cc:383] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.439173 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a8df48231b6f4cdc997f25294c73297c" } modify_external_catalogs: true
I20250115 03:06:21.439227 14809 raft_consensus.cc:738] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.439261 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a8df48231b6f4cdc997f25294c73297c" } modify_external_catalogs: true
I20250115 03:06:21.439319 14809 consensus_queue.cc:260] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.439529 14809 ts_tablet_manager.cc:1428] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.439669 14809 tablet_bootstrap.cc:492] T 398b93f1db0741a9b54837bca43b1bad P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.440435 14809 tablet_bootstrap.cc:654] T 398b93f1db0741a9b54837bca43b1bad P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.441679 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_166b9ddd229c4e079ebf7b12ae5e54dc" } modify_external_catalogs: true
I20250115 03:06:21.441781 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_166b9ddd229c4e079ebf7b12ae5e54dc" } modify_external_catalogs: true
I20250115 03:06:21.442679 14809 tablet_bootstrap.cc:492] T 398b93f1db0741a9b54837bca43b1bad P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.442763 14809 ts_tablet_manager.cc:1397] T 398b93f1db0741a9b54837bca43b1bad P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:21.443244 14809 raft_consensus.cc:357] T 398b93f1db0741a9b54837bca43b1bad P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.443362 14809 raft_consensus.cc:383] T 398b93f1db0741a9b54837bca43b1bad P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.443390 14809 raft_consensus.cc:738] T 398b93f1db0741a9b54837bca43b1bad P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.443480 14809 consensus_queue.cc:260] T 398b93f1db0741a9b54837bca43b1bad P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.443701 16406 raft_consensus.cc:491] T 8efa998709034fe9bb37c7c608c99e76 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.443702 14809 ts_tablet_manager.cc:1428] T 398b93f1db0741a9b54837bca43b1bad P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.443805 16406 raft_consensus.cc:513] T 8efa998709034fe9bb37c7c608c99e76 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.443976 14809 tablet_bootstrap.cc:492] T 103f78d0d9fc42308e1e23316895be33 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.444186 16406 leader_election.cc:290] T 8efa998709034fe9bb37c7c608c99e76 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.444331 14162 master_service.cc:946] ReplaceTablet: received request to replace tablet c127b5ac695a456aa38479a5f9a8fe82 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:21.444587 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8efa998709034fe9bb37c7c608c99e76" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.444790 14809 tablet_bootstrap.cc:654] T 103f78d0d9fc42308e1e23316895be33 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:21.444906 14358 leader_election.cc:343] T 8efa998709034fe9bb37c7c608c99e76 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.445760 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8efa998709034fe9bb37c7c608c99e76" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:21.446328 14356 leader_election.cc:343] T 8efa998709034fe9bb37c7c608c99e76 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.446393 14356 leader_election.cc:304] T 8efa998709034fe9bb37c7c608c99e76 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.446517 16406 raft_consensus.cc:2743] T 8efa998709034fe9bb37c7c608c99e76 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.447188 14812 tablet_bootstrap.cc:492] T 4122c97f5ba445ee8c79b7d056438229 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.447242 14809 tablet_bootstrap.cc:492] T 103f78d0d9fc42308e1e23316895be33 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.447263 14812 ts_tablet_manager.cc:1397] T 4122c97f5ba445ee8c79b7d056438229 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.050s user 0.002s sys 0.000s
I20250115 03:06:21.447306 14809 ts_tablet_manager.cc:1397] T 103f78d0d9fc42308e1e23316895be33 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:21.447652 14812 raft_consensus.cc:357] T 4122c97f5ba445ee8c79b7d056438229 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.447662 14809 raft_consensus.cc:357] T 103f78d0d9fc42308e1e23316895be33 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.447742 14812 raft_consensus.cc:383] T 4122c97f5ba445ee8c79b7d056438229 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.447752 14809 raft_consensus.cc:383] T 103f78d0d9fc42308e1e23316895be33 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.447777 14812 raft_consensus.cc:738] T 4122c97f5ba445ee8c79b7d056438229 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.447783 14809 raft_consensus.cc:738] T 103f78d0d9fc42308e1e23316895be33 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.447870 14812 consensus_queue.cc:260] T 4122c97f5ba445ee8c79b7d056438229 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.447880 14809 consensus_queue.cc:260] T 103f78d0d9fc42308e1e23316895be33 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.448103 14809 ts_tablet_manager.cc:1428] T 103f78d0d9fc42308e1e23316895be33 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.448105 14812 ts_tablet_manager.cc:1428] T 4122c97f5ba445ee8c79b7d056438229 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.448238 14809 tablet_bootstrap.cc:492] T f47da0e523d14388898a4be9c37bb809 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.448256 14812 tablet_bootstrap.cc:492] T bfeafbf380df41738ebbcc11c19a7f85 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.449072 14809 tablet_bootstrap.cc:654] T f47da0e523d14388898a4be9c37bb809 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.449223 14812 tablet_bootstrap.cc:654] T bfeafbf380df41738ebbcc11c19a7f85 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.449906 14668 tablet_service.cc:1467] Processing CreateTablet for tablet b438faf62e154afe9aec2c0c44d59ba3 (DEFAULT_TABLE table=default.Table_6df122bc9bdc45aea95cb4f07ee90dfa [id=ba43efe090254ca48a75a4de394b4c9e]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:21.449997 14157 catalog_manager.cc:4670] Async tablet task AlterTable RPC for tablet d5e4743e016e49ed8ba26b1ac96b1470 (table default.Table_013cb9366ef94a75a90ef4dfed00ff41 [id=6ab343bee1254fb19a3024ebafa44464], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet d5e4743e016e49ed8ba26b1ac96b1470 consensus state has no leader
I20250115 03:06:21.450063 16189 raft_consensus.cc:491] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.450129 16189 raft_consensus.cc:513] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.450424 16189 leader_election.cc:290] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.450628 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 54eb137af68b4ff79f2f6f4743952390 (DEFAULT_TABLE table=default.Table_c1a47c048ec54b68aecd90dd20ea1ea5 [id=97ce4eb35dda47e6b98607757a57d215]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.451210 14391 tablet_service.cc:1467] Processing CreateTablet for tablet b438faf62e154afe9aec2c0c44d59ba3 (DEFAULT_TABLE table=default.Table_6df122bc9bdc45aea95cb4f07ee90dfa [id=ba43efe090254ca48a75a4de394b4c9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.451428 14391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b438faf62e154afe9aec2c0c44d59ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.452013 14655 tablet_service.cc:1467] Processing CreateTablet for tablet 54eb137af68b4ff79f2f6f4743952390 (DEFAULT_TABLE table=default.Table_c1a47c048ec54b68aecd90dd20ea1ea5 [id=97ce4eb35dda47e6b98607757a57d215]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.452512 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 54eb137af68b4ff79f2f6f4743952390 (DEFAULT_TABLE table=default.Table_c1a47c048ec54b68aecd90dd20ea1ea5 [id=97ce4eb35dda47e6b98607757a57d215]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.452592 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b438faf62e154afe9aec2c0c44d59ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.452725 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54eb137af68b4ff79f2f6f4743952390. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.453863 14656 tablet_service.cc:1467] Processing CreateTablet for tablet 71137cf62a374522af620e0ddb782fa8 (DEFAULT_TABLE table=default.Table_a96d43a0d2eb4f5c86b28defd2c3de4f [id=34b04825b8d84733b647947b68cf8510]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.454162 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54eb137af68b4ff79f2f6f4743952390. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.454646 14526 tablet_service.cc:1467] Processing CreateTablet for tablet 71137cf62a374522af620e0ddb782fa8 (DEFAULT_TABLE table=default.Table_a96d43a0d2eb4f5c86b28defd2c3de4f [id=34b04825b8d84733b647947b68cf8510]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.454852 14526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71137cf62a374522af620e0ddb782fa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.455740 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 9443a671086d47d29e872971461558ee
I20250115 03:06:21.456492 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 9443a671086d47d29e872971461558ee with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.456648 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b438faf62e154afe9aec2c0c44d59ba3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.456677 14165 catalog_manager.cc:5526] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:21.456817 14657 tablet_service.cc:1514] Processing DeleteTablet for tablet 9443a671086d47d29e872971461558ee with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.456864 14165 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 0f20bb66f36647e0811d0865b8f414f7 (table default.Table_0326e94494d0447585233c0c907113fd [id=f48964597a90470299a755217c4dbea0]). Expected version 1 got 0
W20250115 03:06:21.456885 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 9443a671086d47d29e872971461558ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9443a671086d47d29e872971461558ee already in progress: creating tablet
I20250115 03:06:21.457018 14165 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 7b86e66428504217af08f667f660d304 (table default.Table_341a3cd1c8c94ff4b05fc21960a4e9b4 [id=c096ce170874405ea740b9ea829f886e]). Expected version 2 got 0
I20250115 03:06:21.457324 16406 raft_consensus.cc:491] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.457393 16406 raft_consensus.cc:513] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.457652 16406 leader_election.cc:290] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.457746 14811 tablet_bootstrap.cc:654] T 38f5554b10a44c249e4c3d3825cd7952 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.457975 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a2894e4736b4191963a61d908bc673c" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:21.458266 14358 leader_election.cc:343] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.458377 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_23165257844448ce93a97233b487ddc7 [id=080d0a4dfa174d3ba2979bb850d0604a] alter complete (version 1)
W20250115 03:06:21.458483 14493 leader_election.cc:343] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: 0792e52d171446b48a7c9d1f03d37a79
I20250115 03:06:21.458935 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a2894e4736b4191963a61d908bc673c" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:21.459245 14493 leader_election.cc:343] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 0792e52d171446b48a7c9d1f03d37a79
I20250115 03:06:21.459306 14493 leader_election.cc:304] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:21.459403 14656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71137cf62a374522af620e0ddb782fa8. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:21.459578 14356 leader_election.cc:343] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.459636 14356 leader_election.cc:304] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.459765 16406 raft_consensus.cc:2743] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.459822 16189 raft_consensus.cc:2743] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.461117 14812 tablet_bootstrap.cc:492] T bfeafbf380df41738ebbcc11c19a7f85 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.461194 14812 ts_tablet_manager.cc:1397] T bfeafbf380df41738ebbcc11c19a7f85 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20250115 03:06:21.461567 14812 raft_consensus.cc:357] T bfeafbf380df41738ebbcc11c19a7f85 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.461664 14812 raft_consensus.cc:383] T bfeafbf380df41738ebbcc11c19a7f85 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.461696 14812 raft_consensus.cc:738] T bfeafbf380df41738ebbcc11c19a7f85 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.461803 14812 consensus_queue.cc:260] T bfeafbf380df41738ebbcc11c19a7f85 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.461869 14526 tablet_service.cc:1514] Processing DeleteTablet for tablet 0792e52d171446b48a7c9d1f03d37a79 with delete_type TABLET_DATA_DELETED (replaced by tablet d65bf3745b4f4d92a346b4dd18652432) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.462021 14820 tablet_replica.cc:331] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:21.462056 14812 ts_tablet_manager.cc:1428] T bfeafbf380df41738ebbcc11c19a7f85 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.462141 14820 raft_consensus.cc:2238] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.462195 14812 tablet_bootstrap.cc:492] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.462227 14820 raft_consensus.cc:2267] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.462888 14820 ts_tablet_manager.cc:1905] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.463021 14812 tablet_bootstrap.cc:654] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.464751 14820 ts_tablet_manager.cc:1918] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.464809 14820 log.cc:1198] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/0792e52d171446b48a7c9d1f03d37a79
I20250115 03:06:21.466784 14809 tablet_bootstrap.cc:492] T f47da0e523d14388898a4be9c37bb809 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.466856 14809 ts_tablet_manager.cc:1397] T f47da0e523d14388898a4be9c37bb809 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.000s
I20250115 03:06:21.467218 14809 raft_consensus.cc:357] T f47da0e523d14388898a4be9c37bb809 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.467294 14809 raft_consensus.cc:383] T f47da0e523d14388898a4be9c37bb809 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.467319 14809 raft_consensus.cc:738] T f47da0e523d14388898a4be9c37bb809 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.467694 14812 tablet_bootstrap.cc:492] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.467775 14812 ts_tablet_manager.cc:1397] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:21.467738 14809 consensus_queue.cc:260] T f47da0e523d14388898a4be9c37bb809 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.468423 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54eb137af68b4ff79f2f6f4743952390. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.468566 14809 ts_tablet_manager.cc:1428] T f47da0e523d14388898a4be9c37bb809 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20250115 03:06:21.468575 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 9443a671086d47d29e872971461558ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9443a671086d47d29e872971461558ee already in progress: creating tablet
I20250115 03:06:21.469173 14820 ts_tablet_manager.cc:1939] T 0792e52d171446b48a7c9d1f03d37a79 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:21.469558 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 0792e52d171446b48a7c9d1f03d37a79 (table default.Table_72e82b3e776e4d398fa5d8240afde936 [id=17ae925a7c0942d59058b2fd619ccb45]) successfully deleted
I20250115 03:06:21.469744 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 9443a671086d47d29e872971461558ee with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
W20250115 03:06:21.470119 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 9443a671086d47d29e872971461558ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9443a671086d47d29e872971461558ee already in progress: creating tablet
I20250115 03:06:21.468155 14812 raft_consensus.cc:357] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.472170 14812 raft_consensus.cc:383] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.472174 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet c127b5ac695a456aa38479a5f9a8fe82
I20250115 03:06:21.472213 14812 raft_consensus.cc:738] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.472313 14812 consensus_queue.cc:260] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.472378 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 0675b412073741128f11b84c0e0582f1
I20250115 03:06:21.472438 14162 catalog_manager.cc:6596] ReplaceTablet: tablet c127b5ac695a456aa38479a5f9a8fe82 deleted and replaced by tablet a5eda11ab1a24b5dbe7b177a49671cf5
I20250115 03:06:21.473016 14809 tablet_bootstrap.cc:492] T 75c581f11be84e6293a3d14966fd584f P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.473141 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet c127b5ac695a456aa38479a5f9a8fe82 with delete_type TABLET_DATA_DELETED (replaced by tablet a5eda11ab1a24b5dbe7b177a49671cf5) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.473553 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 0675b412073741128f11b84c0e0582f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.473903 14809 tablet_bootstrap.cc:654] T 75c581f11be84e6293a3d14966fd584f P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:21.473969 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet c127b5ac695a456aa38479a5f9a8fe82 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c127b5ac695a456aa38479a5f9a8fe82 already in progress: creating tablet
W20250115 03:06:21.474109 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 0675b412073741128f11b84c0e0582f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0675b412073741128f11b84c0e0582f1 already in progress: creating tablet
I20250115 03:06:21.475210 14655 tablet_service.cc:1514] Processing DeleteTablet for tablet c127b5ac695a456aa38479a5f9a8fe82 with delete_type TABLET_DATA_DELETED (replaced by tablet a5eda11ab1a24b5dbe7b177a49671cf5) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:21.475641 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet c127b5ac695a456aa38479a5f9a8fe82 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c127b5ac695a456aa38479a5f9a8fe82 already in progress: creating tablet
I20250115 03:06:21.476034 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet 0675b412073741128f11b84c0e0582f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.476569 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_86a356644d764518ab84bb6ff8d67991" } modify_external_catalogs: true
I20250115 03:06:21.476666 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_86a356644d764518ab84bb6ff8d67991" } modify_external_catalogs: true
I20250115 03:06:21.477149 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet c127b5ac695a456aa38479a5f9a8fe82 with delete_type TABLET_DATA_DELETED (replaced by tablet a5eda11ab1a24b5dbe7b177a49671cf5) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.477870 14526 tablet_service.cc:1514] Processing DeleteTablet for tablet 0675b412073741128f11b84c0e0582f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.478520 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_a56fd016ce294130baac03a9ef333cf8 [id=8460d1b93fbc4c3e987f38acc1d00818] alter complete (version 1)
W20250115 03:06:21.479322 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 0675b412073741128f11b84c0e0582f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0675b412073741128f11b84c0e0582f1 already in progress: creating tablet
I20250115 03:06:21.480068 14812 ts_tablet_manager.cc:1428] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.012s user 0.001s sys 0.000s
I20250115 03:06:21.482327 14165 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet bfeafbf380df41738ebbcc11c19a7f85 (table default.Table_660872e60c104e56be9287f66d74c387 [id=03ee00da1e024ebab51b3a366d2c0cf3]). Expected version 2 got 0
I20250115 03:06:21.482904 14391 tablet_service.cc:1514] Processing DeleteTablet for tablet 398b93f1db0741a9b54837bca43b1bad with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.483031 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet b61cc4b11251486a8089ca9bf44ac91e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.483144 14819 tablet_replica.cc:331] T 398b93f1db0741a9b54837bca43b1bad P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.483095 16189 consensus_queue.cc:1035] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:21.483233 14819 raft_consensus.cc:2238] T 398b93f1db0741a9b54837bca43b1bad P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.483296 14819 raft_consensus.cc:2267] T 398b93f1db0741a9b54837bca43b1bad P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.483381 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet f47da0e523d14388898a4be9c37bb809 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.483513 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 103f78d0d9fc42308e1e23316895be33 with delete_type TABLET_DATA_DELETED (replaced by tablet 9c436975bd034068a6d58d5831db1896) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.483599 14812 tablet_bootstrap.cc:492] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.483826 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet 4122c97f5ba445ee8c79b7d056438229 with delete_type TABLET_DATA_DELETED (replaced by tablet fca14a7d574842ee94a93afb85ca1afd) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.483934 14819 ts_tablet_manager.cc:1905] T 398b93f1db0741a9b54837bca43b1bad P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.483983 14823 tablet_replica.cc:331] T 4122c97f5ba445ee8c79b7d056438229 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:21.484098 14823 raft_consensus.cc:2238] T 4122c97f5ba445ee8c79b7d056438229 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.484180 14823 raft_consensus.cc:2267] T 4122c97f5ba445ee8c79b7d056438229 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:21.484256 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet c127b5ac695a456aa38479a5f9a8fe82 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c127b5ac695a456aa38479a5f9a8fe82 already in progress: creating tablet
W20250115 03:06:21.484395 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 0675b412073741128f11b84c0e0582f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0675b412073741128f11b84c0e0582f1 already in progress: creating tablet
I20250115 03:06:21.485101 14526 tablet_service.cc:1514] Processing DeleteTablet for tablet d6cf1a0b9c81413b8f72de0cafc76963 with delete_type TABLET_DATA_DELETED (replaced by tablet a9807cb8c17d4b98be44e6fd5dccc150) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.485154 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4b95b36b8d744822b248adc43a061bb1" } modify_external_catalogs: true
I20250115 03:06:21.485267 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4b95b36b8d744822b248adc43a061bb1" } modify_external_catalogs: true
I20250115 03:06:21.485539 14812 tablet_bootstrap.cc:654] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.486060 14819 ts_tablet_manager.cc:1918] T 398b93f1db0741a9b54837bca43b1bad P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.486130 14819 log.cc:1198] T 398b93f1db0741a9b54837bca43b1bad P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/398b93f1db0741a9b54837bca43b1bad
I20250115 03:06:21.486312 14819 ts_tablet_manager.cc:1939] T 398b93f1db0741a9b54837bca43b1bad P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.486666 14812 tablet_bootstrap.cc:492] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.486743 14812 ts_tablet_manager.cc:1397] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:21.487134 14812 raft_consensus.cc:357] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.487221 14812 raft_consensus.cc:383] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.487253 14812 raft_consensus.cc:738] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.487356 14812 consensus_queue.cc:260] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.487601 14812 ts_tablet_manager.cc:1428] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.487664 14820 tablet_replica.cc:331] T d6cf1a0b9c81413b8f72de0cafc76963 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:21.487758 14812 tablet_bootstrap.cc:492] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.487774 14820 raft_consensus.cc:2238] T d6cf1a0b9c81413b8f72de0cafc76963 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.487856 14820 raft_consensus.cc:2267] T d6cf1a0b9c81413b8f72de0cafc76963 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.487855 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 398b93f1db0741a9b54837bca43b1bad (table default.Table_84b19a144ef44b0380ee4e6c66a26370 [id=9f9e695a3cee41ef872d4b6f3566b6c9]) successfully deleted
I20250115 03:06:21.488034 14819 tablet_replica.cc:331] T b61cc4b11251486a8089ca9bf44ac91e P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.488145 14819 raft_consensus.cc:2238] T b61cc4b11251486a8089ca9bf44ac91e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.488225 14819 raft_consensus.cc:2267] T b61cc4b11251486a8089ca9bf44ac91e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.488669 14819 ts_tablet_manager.cc:1905] T b61cc4b11251486a8089ca9bf44ac91e P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.488685 14812 tablet_bootstrap.cc:654] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.489044 16292 raft_consensus.cc:491] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.489117 16292 raft_consensus.cc:513] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.489403 16292 leader_election.cc:290] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:21.490371 14823 ts_tablet_manager.cc:1905] T 4122c97f5ba445ee8c79b7d056438229 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.490417 14819 ts_tablet_manager.cc:1918] T b61cc4b11251486a8089ca9bf44ac91e P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.490466 14819 log.cc:1198] T b61cc4b11251486a8089ca9bf44ac91e P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/b61cc4b11251486a8089ca9bf44ac91e
W20250115 03:06:21.490587 14628 leader_election.cc:343] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: bb68869eaf55477090cf12f40f76a89c
I20250115 03:06:21.490648 14819 ts_tablet_manager.cc:1939] T b61cc4b11251486a8089ca9bf44ac91e P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.490880 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb68869eaf55477090cf12f40f76a89c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.490916 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 71137cf62a374522af620e0ddb782fa8 (DEFAULT_TABLE table=default.Table_a96d43a0d2eb4f5c86b28defd2c3de4f [id=34b04825b8d84733b647947b68cf8510]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.491135 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71137cf62a374522af620e0ddb782fa8. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:21.491183 14626 leader_election.cc:343] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.491173 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet b61cc4b11251486a8089ca9bf44ac91e (table default.Table_f634ab8b7b7c437f90035122e8f974b9 [id=555a4a4508a1409fb8febc47add30a1a]) successfully deleted
I20250115 03:06:21.491233 14626 leader_election.cc:304] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:21.491386 14819 tablet_replica.cc:331] T f47da0e523d14388898a4be9c37bb809 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.491436 16001 raft_consensus.cc:2743] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.491488 14819 raft_consensus.cc:2238] T f47da0e523d14388898a4be9c37bb809 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.491575 14819 raft_consensus.cc:2267] T f47da0e523d14388898a4be9c37bb809 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.492339 14823 ts_tablet_manager.cc:1918] T 4122c97f5ba445ee8c79b7d056438229 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.492399 14823 log.cc:1198] T 4122c97f5ba445ee8c79b7d056438229 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/4122c97f5ba445ee8c79b7d056438229
I20250115 03:06:21.492576 14823 ts_tablet_manager.cc:1939] T 4122c97f5ba445ee8c79b7d056438229 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:21.493352 14820 ts_tablet_manager.cc:1905] T d6cf1a0b9c81413b8f72de0cafc76963 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.493579 14526 tablet_service.cc:1467] Processing CreateTablet for tablet e227869626a84098be33c5ed58b0a0c2 (DEFAULT_TABLE table=default.Table_2e6e02769ab14002924b5cc60596fddc [id=4d7ed579e41e40ebbbddc92c8bfae8c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.493695 14819 ts_tablet_manager.cc:1905] T f47da0e523d14388898a4be9c37bb809 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.494155 14398 tablet_service.cc:1467] Processing CreateTablet for tablet e227869626a84098be33c5ed58b0a0c2 (DEFAULT_TABLE table=default.Table_2e6e02769ab14002924b5cc60596fddc [id=4d7ed579e41e40ebbbddc92c8bfae8c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.494359 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e227869626a84098be33c5ed58b0a0c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.494370 14526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e227869626a84098be33c5ed58b0a0c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.495435 14168 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet eb19e79635654bd49d7bba95b4559c67
I20250115 03:06:21.495491 14819 ts_tablet_manager.cc:1918] T f47da0e523d14388898a4be9c37bb809 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.495558 14819 log.cc:1198] T f47da0e523d14388898a4be9c37bb809 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/f47da0e523d14388898a4be9c37bb809
I20250115 03:06:21.495738 14819 ts_tablet_manager.cc:1939] T f47da0e523d14388898a4be9c37bb809 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.496198 14819 tablet_replica.cc:331] T 103f78d0d9fc42308e1e23316895be33 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.496196 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 4122c97f5ba445ee8c79b7d056438229 (table default.Table_b169e386c2cc424da59a39503cec548f [id=2b97c2f5b5284cd8a6311247712cc447]) successfully deleted
I20250115 03:06:21.496310 14819 raft_consensus.cc:2238] T 103f78d0d9fc42308e1e23316895be33 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.496393 14819 raft_consensus.cc:2267] T 103f78d0d9fc42308e1e23316895be33 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.496420 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet f47da0e523d14388898a4be9c37bb809 (table default.Table_59afdaba8b114e13b37772ac27949580 [id=ca82b44382294c8da6a2ac0f0e8ccf81]) successfully deleted
I20250115 03:06:21.496768 14668 tablet_service.cc:1467] Processing CreateTablet for tablet e227869626a84098be33c5ed58b0a0c2 (DEFAULT_TABLE table=default.Table_2e6e02769ab14002924b5cc60596fddc [id=4d7ed579e41e40ebbbddc92c8bfae8c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.496992 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e227869626a84098be33c5ed58b0a0c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.498365 14820 ts_tablet_manager.cc:1918] T d6cf1a0b9c81413b8f72de0cafc76963 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.498433 14820 log.cc:1198] T d6cf1a0b9c81413b8f72de0cafc76963 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/d6cf1a0b9c81413b8f72de0cafc76963
I20250115 03:06:21.498605 14820 ts_tablet_manager.cc:1939] T d6cf1a0b9c81413b8f72de0cafc76963 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:21.501178 16292 raft_consensus.cc:491] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.501255 16292 raft_consensus.cc:513] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.501554 16292 leader_election.cc:290] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:21.502014 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "998a515113d04034928c04dc5798a698" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.502120 14422 raft_consensus.cc:2463] T 998a515113d04034928c04dc5798a698 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:21.502612 14809 tablet_bootstrap.cc:492] T 75c581f11be84e6293a3d14966fd584f P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.502693 14809 ts_tablet_manager.cc:1397] T 75c581f11be84e6293a3d14966fd584f P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.000s
I20250115 03:06:21.503114 14809 raft_consensus.cc:357] T 75c581f11be84e6293a3d14966fd584f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.503201 14809 raft_consensus.cc:383] T 75c581f11be84e6293a3d14966fd584f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.503230 14809 raft_consensus.cc:738] T 75c581f11be84e6293a3d14966fd584f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.503320 14809 consensus_queue.cc:260] T 75c581f11be84e6293a3d14966fd584f P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.503583 14809 ts_tablet_manager.cc:1428] T 75c581f11be84e6293a3d14966fd584f P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:21.503988 14819 ts_tablet_manager.cc:1905] T 103f78d0d9fc42308e1e23316895be33 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.504259 14628 leader_election.cc:304] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:21.504442 14809 tablet_bootstrap.cc:492] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.504459 16292 raft_consensus.cc:2798] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.504508 16292 raft_consensus.cc:491] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.504546 16292 raft_consensus.cc:3054] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.505002 14858 consensus_queue.cc:1035] T 0521973e23984355a77a321b4676b479 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:21.505360 14809 tablet_bootstrap.cc:654] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.505494 16292 raft_consensus.cc:513] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.505829 16292 leader_election.cc:290] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:21.507746 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "998a515113d04034928c04dc5798a698" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.507781 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 5b0cdca3ada044108576d904543e5602 (DEFAULT_TABLE table=default.Table_27a225477c7141f5a427f30387298506 [id=a63ee64143864ed195a55f02ed43c4be]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.507841 14422 raft_consensus.cc:3054] T 998a515113d04034928c04dc5798a698 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.507987 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b0cdca3ada044108576d904543e5602. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.508646 14422 raft_consensus.cc:2463] T 998a515113d04034928c04dc5798a698 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:21.509088 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "998a515113d04034928c04dc5798a698" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.509219 14552 raft_consensus.cc:2463] T 998a515113d04034928c04dc5798a698 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:21.509384 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "998a515113d04034928c04dc5798a698" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:21.509461 14553 raft_consensus.cc:3054] T 998a515113d04034928c04dc5798a698 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.510107 14858 raft_consensus.cc:491] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.510179 14858 raft_consensus.cc:513] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.510293 14553 raft_consensus.cc:2463] T 998a515113d04034928c04dc5798a698 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:21.510471 14858 leader_election.cc:290] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.510596 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 5b0cdca3ada044108576d904543e5602 (DEFAULT_TABLE table=default.Table_27a225477c7141f5a427f30387298506 [id=a63ee64143864ed195a55f02ed43c4be]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.510819 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b0cdca3ada044108576d904543e5602. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.510877 14628 leader_election.cc:304] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:21.510937 14819 ts_tablet_manager.cc:1918] T 103f78d0d9fc42308e1e23316895be33 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.510998 14819 log.cc:1198] T 103f78d0d9fc42308e1e23316895be33 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/103f78d0d9fc42308e1e23316895be33
I20250115 03:06:21.511015 15153 raft_consensus.cc:2798] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.511080 15153 raft_consensus.cc:695] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:21.511183 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1016cf85491d4ceda132ee57325b48a4" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.511237 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1016cf85491d4ceda132ee57325b48a4" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.511197 15153 consensus_queue.cc:237] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.511318 14687 raft_consensus.cc:2463] T 1016cf85491d4ceda132ee57325b48a4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.511337 14422 raft_consensus.cc:2463] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.511547 14819 ts_tablet_manager.cc:1939] T 103f78d0d9fc42308e1e23316895be33 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.511664 14493 leader_election.cc:304] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.511912 14858 raft_consensus.cc:2798] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.511953 14526 tablet_service.cc:1467] Processing CreateTablet for tablet 2f1809e868e840b796114b7ddd508d69 (DEFAULT_TABLE table=default.Table_89657e3d79dc42d182839fa60ce9190a [id=36d4b00bfa9a41f48b83705d33b0caea]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.512151 14526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f1809e868e840b796114b7ddd508d69. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.511961 14858 raft_consensus.cc:491] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.513022 14858 raft_consensus.cc:3054] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.513779 14858 raft_consensus.cc:513] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.513916 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 2f1809e868e840b796114b7ddd508d69 (DEFAULT_TABLE table=default.Table_89657e3d79dc42d182839fa60ce9190a [id=36d4b00bfa9a41f48b83705d33b0caea]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.514081 14858 leader_election.cc:290] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.514675 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 103f78d0d9fc42308e1e23316895be33 (table default.Table_ccd2815995fc485d827b79ae818fedae [id=522c89cadb084de68368a97199ea66db]) successfully deleted
I20250115 03:06:21.515131 14811 tablet_bootstrap.cc:492] T 38f5554b10a44c249e4c3d3825cd7952 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.515211 14811 ts_tablet_manager.cc:1397] T 38f5554b10a44c249e4c3d3825cd7952 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.084s user 0.002s sys 0.000s
I20250115 03:06:21.515554 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1016cf85491d4ceda132ee57325b48a4" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:21.515650 14687 raft_consensus.cc:3054] T 1016cf85491d4ceda132ee57325b48a4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.515615 14811 raft_consensus.cc:357] T 38f5554b10a44c249e4c3d3825cd7952 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.515702 14811 raft_consensus.cc:383] T 38f5554b10a44c249e4c3d3825cd7952 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.515730 14811 raft_consensus.cc:738] T 38f5554b10a44c249e4c3d3825cd7952 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.515836 14811 consensus_queue.cc:260] T 38f5554b10a44c249e4c3d3825cd7952 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.516088 14811 ts_tablet_manager.cc:1428] T 38f5554b10a44c249e4c3d3825cd7952 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:21.516245 14811 tablet_bootstrap.cc:492] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.516546 14687 raft_consensus.cc:2463] T 1016cf85491d4ceda132ee57325b48a4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.516963 14811 tablet_bootstrap.cc:654] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.517153 14493 leader_election.cc:304] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.517292 14858 raft_consensus.cc:2798] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.517357 14858 raft_consensus.cc:695] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:21.517459 14858 consensus_queue.cc:237] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.518664 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f1809e868e840b796114b7ddd508d69. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.519979 14166 catalog_manager.cc:5526] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:21.520540 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1016cf85491d4ceda132ee57325b48a4" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.520646 14422 raft_consensus.cc:3054] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.521646 14422 raft_consensus.cc:2463] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.523871 14656 tablet_service.cc:1467] Processing CreateTablet for tablet 5b0cdca3ada044108576d904543e5602 (DEFAULT_TABLE table=default.Table_27a225477c7141f5a427f30387298506 [id=a63ee64143864ed195a55f02ed43c4be]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.523926 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet d6cf1a0b9c81413b8f72de0cafc76963 (table default.Table_e51b0b04bb044b36be347afe90845b14 [id=17ea0c263011486f82895eb86b21fe78]) successfully deleted
I20250115 03:06:21.523975 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ad04661b1b684f879f1c0cd4f94e5458" } modify_external_catalogs: true
I20250115 03:06:21.524057 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ad04661b1b684f879f1c0cd4f94e5458" } modify_external_catalogs: true
I20250115 03:06:21.524094 14656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b0cdca3ada044108576d904543e5602. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.524113 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_de7f79ee6adb4796911bd6e9f106da1f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.524421 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_de7f79ee6adb4796911bd6e9f106da1f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.524389 14655 tablet_service.cc:1467] Processing CreateTablet for tablet 2f1809e868e840b796114b7ddd508d69 (DEFAULT_TABLE table=default.Table_89657e3d79dc42d182839fa60ce9190a [id=36d4b00bfa9a41f48b83705d33b0caea]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.524617 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f1809e868e840b796114b7ddd508d69. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.527107 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 75c581f11be84e6293a3d14966fd584f with delete_type TABLET_DATA_DELETED (replaced by tablet 5c5e30d9b2ce4537b00b216ef1f2dd1c) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.528561 14811 tablet_bootstrap.cc:492] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.528643 14811 ts_tablet_manager.cc:1397] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.002s
I20250115 03:06:21.529104 14811 raft_consensus.cc:357] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.529176 14811 raft_consensus.cc:383] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.529196 14811 raft_consensus.cc:738] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.529310 14811 consensus_queue.cc:260] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.529562 14811 ts_tablet_manager.cc:1428] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:21.529714 14811 tablet_bootstrap.cc:492] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.530109 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet bb68869eaf55477090cf12f40f76a89c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.530256 14823 tablet_replica.cc:331] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:21.530421 14823 raft_consensus.cc:2238] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.530521 14823 raft_consensus.cc:2267] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.530241 16454 tablet.cc:1721] T 15fbf8d5de904fc4b284af664f3bd60f P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:21.530819 16265 tablet.cc:1721] T 15fbf8d5de904fc4b284af664f3bd60f P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:21.530925 14823 ts_tablet_manager.cc:1905] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.532567 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet b9f67c35591f4fdeb8a926a085171b49
I20250115 03:06:21.533459 14526 tablet_service.cc:1514] Processing DeleteTablet for tablet b9f67c35591f4fdeb8a926a085171b49 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.533830 14823 ts_tablet_manager.cc:1918] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.533864 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet b9f67c35591f4fdeb8a926a085171b49 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.533900 14823 log.cc:1198] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/bb68869eaf55477090cf12f40f76a89c
I20250115 03:06:21.534088 14823 ts_tablet_manager.cc:1939] T bb68869eaf55477090cf12f40f76a89c P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:21.535225 14811 tablet_bootstrap.cc:654] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.535354 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet bb68869eaf55477090cf12f40f76a89c (table default.Table_0964201dda974eaabef3d5694c40389c [id=4a6fefd2006a4b11b121bfad61cf95ed]) successfully deleted
I20250115 03:06:21.535776 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet b9f67c35591f4fdeb8a926a085171b49 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
W20250115 03:06:21.536813 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet b9f67c35591f4fdeb8a926a085171b49 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b9f67c35591f4fdeb8a926a085171b49 already in progress: creating tablet
I20250115 03:06:21.536969 15617 tablet.cc:1721] T 15fbf8d5de904fc4b284af664f3bd60f P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:21.541849 14811 tablet_bootstrap.cc:492] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.541927 14811 ts_tablet_manager.cc:1397] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20250115 03:06:21.542294 14811 raft_consensus.cc:357] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.542385 14811 raft_consensus.cc:383] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.542425 14811 raft_consensus.cc:738] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.542550 14811 consensus_queue.cc:260] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.542805 14811 ts_tablet_manager.cc:1428] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:21.542977 14811 tablet_bootstrap.cc:492] T 4122c97f5ba445ee8c79b7d056438229 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.543871 14811 tablet_bootstrap.cc:654] T 4122c97f5ba445ee8c79b7d056438229 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.544481 14164 master_service.cc:946] ReplaceTablet: received request to replace tablet 15263a3b32be418288770ce348c082d7 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:21.544560 14853 raft_consensus.cc:491] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.544620 14853 raft_consensus.cc:513] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.544672 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2cb3931be82e48859ff5afc497e38338" } new_table_name: "default.Table_59522d805bce49cd9b837dd585b3cfcf" modify_external_catalogs: true
I20250115 03:06:21.544977 14853 leader_election.cc:290] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.545416 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8688ed8444804f65b144ec7ac48c5e73" } new_table_name: "default.Table_d9796d81c05d4f50bc62f2f73fefc5d8" modify_external_catalogs: true
I20250115 03:06:21.545584 15153 raft_consensus.cc:491] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.545740 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 3efc2fb8427f493ea933347bec34f88f
I20250115 03:06:21.545706 15153 raft_consensus.cc:513] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.546016 15153 leader_election.cc:290] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:21.546705 14819 tablet_replica.cc:331] T 75c581f11be84e6293a3d14966fd584f P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.546839 14819 raft_consensus.cc:2238] T 75c581f11be84e6293a3d14966fd584f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.546933 14819 raft_consensus.cc:2267] T 75c581f11be84e6293a3d14966fd584f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:21.546937 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet b9f67c35591f4fdeb8a926a085171b49 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b9f67c35591f4fdeb8a926a085171b49 already in progress: creating tablet
I20250115 03:06:21.547102 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4547465f3124e108dc2c4c54e2bc751" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.547194 14688 raft_consensus.cc:1270] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:21.547220 14687 raft_consensus.cc:2463] T a4547465f3124e108dc2c4c54e2bc751 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.547513 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b86e66428504217af08f667f660d304" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.547602 14493 leader_election.cc:304] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.547616 14553 raft_consensus.cc:2463] T 7b86e66428504217af08f667f660d304 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:21.547853 16189 raft_consensus.cc:2798] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.547904 16189 raft_consensus.cc:491] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.547937 16189 raft_consensus.cc:3054] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.548027 14819 ts_tablet_manager.cc:1905] T 75c581f11be84e6293a3d14966fd584f P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.548583 14858 consensus_queue.cc:1035] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:21.548760 14812 tablet_bootstrap.cc:492] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.548835 14812 ts_tablet_manager.cc:1397] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.061s user 0.000s sys 0.001s
I20250115 03:06:21.548810 16189 raft_consensus.cc:513] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.549147 16189 leader_election.cc:290] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.549270 14812 raft_consensus.cc:357] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.549362 14812 raft_consensus.cc:383] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.549389 14812 raft_consensus.cc:738] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.549490 14812 consensus_queue.cc:260] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.549556 14626 leader_election.cc:304] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.549741 16292 raft_consensus.cc:2798] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.549746 14812 ts_tablet_manager.cc:1428] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:21.549806 16292 raft_consensus.cc:491] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.549844 16292 raft_consensus.cc:3054] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.549934 14812 tablet_bootstrap.cc:492] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.550237 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4547465f3124e108dc2c4c54e2bc751" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:21.550333 14687 raft_consensus.cc:3054] T a4547465f3124e108dc2c4c54e2bc751 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.550501 14166 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_655e9d02aacf4b88ba521f0f57a5f522 [id=6ed01835736b40f8b4caeca7378770d8] alter complete (version 1)
I20250115 03:06:21.550661 16292 raft_consensus.cc:513] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.550807 14812 tablet_bootstrap.cc:654] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.550964 16292 leader_election.cc:290] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:21.551142 14687 raft_consensus.cc:2463] T a4547465f3124e108dc2c4c54e2bc751 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.551122 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4547465f3124e108dc2c4c54e2bc751" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.551221 14419 raft_consensus.cc:2463] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.551402 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4547465f3124e108dc2c4c54e2bc751" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.551492 14421 raft_consensus.cc:3054] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.551487 14493 leader_election.cc:304] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.551585 14819 ts_tablet_manager.cc:1918] T 75c581f11be84e6293a3d14966fd584f P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.551635 14819 log.cc:1198] T 75c581f11be84e6293a3d14966fd584f P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/75c581f11be84e6293a3d14966fd584f
I20250115 03:06:21.551636 14853 raft_consensus.cc:2798] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.551703 14853 raft_consensus.cc:695] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:21.551803 14819 ts_tablet_manager.cc:1939] T 75c581f11be84e6293a3d14966fd584f P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.551839 14853 consensus_queue.cc:237] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.552294 14421 raft_consensus.cc:2463] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.552721 14422 raft_consensus.cc:1270] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:21.553351 16189 consensus_queue.cc:1035] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:21.554138 14526 tablet_service.cc:1514] Processing DeleteTablet for tablet 3efc2fb8427f493ea933347bec34f88f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.554796 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b86e66428504217af08f667f660d304" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.554834 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 75c581f11be84e6293a3d14966fd584f (table default.Table_cb51431eaddb48858a61d248b688bbe7 [id=5624f7f1c61140a798718c8c85da14ee]) successfully deleted
I20250115 03:06:21.554889 14420 raft_consensus.cc:3054] T 7b86e66428504217af08f667f660d304 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.555689 14809 tablet_bootstrap.cc:492] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.555730 14420 raft_consensus.cc:2463] T 7b86e66428504217af08f667f660d304 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:21.555780 14809 ts_tablet_manager.cc:1397] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.051s user 0.001s sys 0.000s
I20250115 03:06:21.556035 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b86e66428504217af08f667f660d304" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.556133 14419 raft_consensus.cc:2371] T 7b86e66428504217af08f667f660d304 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1. Re-sending same reply.
I20250115 03:06:21.556212 14809 raft_consensus.cc:357] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.556305 14809 raft_consensus.cc:383] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.556332 14809 raft_consensus.cc:738] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.556424 14809 consensus_queue.cc:260] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.556731 14809 ts_tablet_manager.cc:1428] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:21.556895 14809 tablet_bootstrap.cc:492] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
W20250115 03:06:21.557125 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet b9f67c35591f4fdeb8a926a085171b49 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b9f67c35591f4fdeb8a926a085171b49 already in progress: creating tablet
I20250115 03:06:21.557842 14668 tablet_service.cc:1514] Processing DeleteTablet for tablet 3efc2fb8427f493ea933347bec34f88f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.558722 14823 tablet_replica.cc:331] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:21.558840 14823 raft_consensus.cc:2238] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:21.558981 14823 raft_consensus.cc:2267] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.559285 14823 ts_tablet_manager.cc:1905] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.560122 14812 tablet_bootstrap.cc:492] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.560205 14812 ts_tablet_manager.cc:1397] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20250115 03:06:21.560612 14812 raft_consensus.cc:357] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.560710 14812 raft_consensus.cc:383] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.560746 14812 raft_consensus.cc:738] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.560846 14812 consensus_queue.cc:260] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.561017 14820 tablet_replica.cc:331] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:21.561113 14812 ts_tablet_manager.cc:1428] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:21.561141 14820 raft_consensus.cc:2238] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.561244 14820 raft_consensus.cc:2267] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.561270 14812 tablet_bootstrap.cc:492] T dda418ed197e4af2aa511d3f1851663e P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.561565 14820 ts_tablet_manager.cc:1905] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.561697 14628 leader_election.cc:304] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:21.562122 15153 raft_consensus.cc:2798] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.562201 15153 raft_consensus.cc:695] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:21.562302 15153 consensus_queue.cc:237] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.563691 14820 ts_tablet_manager.cc:1918] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:21.563764 14820 log.cc:1198] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/3efc2fb8427f493ea933347bec34f88f
I20250115 03:06:21.563972 14820 ts_tablet_manager.cc:1939] T 3efc2fb8427f493ea933347bec34f88f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:21.564224 14809 tablet_bootstrap.cc:654] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.564461 16406 raft_consensus.cc:491] T a8e41e31ffae4bb092cb08d83eda16b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.564543 16406 raft_consensus.cc:513] T a8e41e31ffae4bb092cb08d83eda16b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.564870 16406 leader_election.cc:290] T a8e41e31ffae4bb092cb08d83eda16b5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.562129 14812 tablet_bootstrap.cc:654] T dda418ed197e4af2aa511d3f1851663e P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.564880 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_99c8d02547a54f8fa40ee2aebde0dbb4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.565349 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_99c8d02547a54f8fa40ee2aebde0dbb4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.565392 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e41e31ffae4bb092cb08d83eda16b5" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:21.565712 14358 leader_election.cc:343] T a8e41e31ffae4bb092cb08d83eda16b5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.566496 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e41e31ffae4bb092cb08d83eda16b5" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:21.568171 14356 leader_election.cc:343] T a8e41e31ffae4bb092cb08d83eda16b5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.568240 14356 leader_election.cc:304] T a8e41e31ffae4bb092cb08d83eda16b5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.568388 16406 raft_consensus.cc:2743] T a8e41e31ffae4bb092cb08d83eda16b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.568986 14809 tablet_bootstrap.cc:492] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.569056 14809 ts_tablet_manager.cc:1397] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20250115 03:06:21.569411 14809 raft_consensus.cc:357] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.569494 14809 raft_consensus.cc:383] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.569543 14809 raft_consensus.cc:738] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.569649 14809 consensus_queue.cc:260] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.569787 14812 tablet_bootstrap.cc:492] T dda418ed197e4af2aa511d3f1851663e P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.569854 14812 ts_tablet_manager.cc:1397] T dda418ed197e4af2aa511d3f1851663e P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:21.570266 14812 raft_consensus.cc:357] T dda418ed197e4af2aa511d3f1851663e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.570348 14812 raft_consensus.cc:383] T dda418ed197e4af2aa511d3f1851663e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.570377 14812 raft_consensus.cc:738] T dda418ed197e4af2aa511d3f1851663e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.570498 14812 consensus_queue.cc:260] T dda418ed197e4af2aa511d3f1851663e P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.570838 14812 ts_tablet_manager.cc:1428] T dda418ed197e4af2aa511d3f1851663e P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.570969 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 3efc2fb8427f493ea933347bec34f88f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.571216 14812 tablet_bootstrap.cc:492] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.571844 14819 tablet_replica.cc:331] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.571970 14819 raft_consensus.cc:2238] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.571496 16001 consensus_queue.cc:1035] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:21.572727 14819 raft_consensus.cc:2267] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.573092 14819 ts_tablet_manager.cc:1905] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.574110 14823 ts_tablet_manager.cc:1918] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:21.574179 14823 log.cc:1198] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/3efc2fb8427f493ea933347bec34f88f
I20250115 03:06:21.574402 14823 ts_tablet_manager.cc:1939] T 3efc2fb8427f493ea933347bec34f88f P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:21.576532 16406 raft_consensus.cc:491] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.576629 14819 ts_tablet_manager.cc:1918] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20250115 03:06:21.576610 16406 raft_consensus.cc:513] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.576689 14819 log.cc:1198] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/3efc2fb8427f493ea933347bec34f88f
I20250115 03:06:21.576907 14819 ts_tablet_manager.cc:1939] T 3efc2fb8427f493ea933347bec34f88f P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.577159 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c580cdf3a854df988df32a9294c83bc" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:21.577510 14356 leader_election.cc:343] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.577724 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 3efc2fb8427f493ea933347bec34f88f (table default.Table_ad04661b1b684f879f1c0cd4f94e5458 [id=364a7905f3f044f684e9c75927f7efda]) successfully deleted
I20250115 03:06:21.577839 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c580cdf3a854df988df32a9294c83bc" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.578018 14809 ts_tablet_manager.cc:1428] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:21.578106 16406 leader_election.cc:290] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.578179 14809 tablet_bootstrap.cc:492] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
W20250115 03:06:21.578311 14358 leader_election.cc:343] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.578331 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 3efc2fb8427f493ea933347bec34f88f (table default.Table_ad04661b1b684f879f1c0cd4f94e5458 [id=364a7905f3f044f684e9c75927f7efda]) successfully deleted
I20250115 03:06:21.578364 14358 leader_election.cc:304] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.578559 16406 raft_consensus.cc:2743] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.579044 14809 tablet_bootstrap.cc:654] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.579671 14811 tablet_bootstrap.cc:492] T 4122c97f5ba445ee8c79b7d056438229 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.579767 14811 ts_tablet_manager.cc:1397] T 4122c97f5ba445ee8c79b7d056438229 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.037s user 0.001s sys 0.000s
I20250115 03:06:21.580104 14809 tablet_bootstrap.cc:492] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.580175 14809 ts_tablet_manager.cc:1397] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:21.580190 14811 raft_consensus.cc:357] T 4122c97f5ba445ee8c79b7d056438229 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.580278 14811 raft_consensus.cc:383] T 4122c97f5ba445ee8c79b7d056438229 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.580308 14811 raft_consensus.cc:738] T 4122c97f5ba445ee8c79b7d056438229 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.580399 14811 consensus_queue.cc:260] T 4122c97f5ba445ee8c79b7d056438229 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.580597 14809 raft_consensus.cc:357] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.580644 14811 ts_tablet_manager.cc:1428] T 4122c97f5ba445ee8c79b7d056438229 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.580688 14809 raft_consensus.cc:383] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.580721 14809 raft_consensus.cc:738] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.580750 16454 tablet.cc:1721] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:21.580804 14811 tablet_bootstrap.cc:492] T ad0e9216bd154f4a8a37874d3d8a08c2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.580824 14809 consensus_queue.cc:260] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.580999 16265 tablet.cc:1721] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:21.581393 14809 ts_tablet_manager.cc:1428] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:21.581557 14809 tablet_bootstrap.cc:492] T 83a4f95eb9494a5295a81a8a51f58bd3 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.581763 14811 tablet_bootstrap.cc:654] T ad0e9216bd154f4a8a37874d3d8a08c2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.582376 14809 tablet_bootstrap.cc:654] T 83a4f95eb9494a5295a81a8a51f58bd3 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.583056 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 3efc2fb8427f493ea933347bec34f88f (table default.Table_ad04661b1b684f879f1c0cd4f94e5458 [id=364a7905f3f044f684e9c75927f7efda]) successfully deleted
I20250115 03:06:21.583441 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_28d056349ee04f339900b307997bca9a" } new_table_name: "default.Table_0d782d9b6b9b4020b10ab81b8541375e" modify_external_catalogs: true
I20250115 03:06:21.583652 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_256bd9b91a3e426992d4b732f03d8181"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.583990 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_256bd9b91a3e426992d4b732f03d8181 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.584439 14812 tablet_bootstrap.cc:654] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.585762 14812 tablet_bootstrap.cc:492] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.585830 14812 ts_tablet_manager.cc:1397] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
W20250115 03:06:21.586367 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 29c72f33a2034c4eae81c7dfcca27ca4 consensus state has no leader
I20250115 03:06:21.586592 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f88c5bc446c6498393a81148a97bb27f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:21.586225 14812 raft_consensus.cc:357] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.586846 14812 raft_consensus.cc:383] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.586887 14812 raft_consensus.cc:738] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
W20250115 03:06:21.586908 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f88c5bc446c6498393a81148a97bb27f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.587018 14812 consensus_queue.cc:260] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:21.587584 14167 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 695927c78bb74b669f2aeca1abf1b648 consensus state has no leader
I20250115 03:06:21.587628 14809 tablet_bootstrap.cc:492] T 83a4f95eb9494a5295a81a8a51f58bd3 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.587692 14809 ts_tablet_manager.cc:1397] T 83a4f95eb9494a5295a81a8a51f58bd3 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20250115 03:06:21.587736 14167 master_service.cc:946] ReplaceTablet: received request to replace tablet fdaf7c090ef642b8b339f61a5d3fb06e from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:21.587899 14812 ts_tablet_manager.cc:1428] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:21.588040 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e68b21fe987c4ea49523f24440585fce" } new_table_name: "default.Table_ca82711024824acda4e6b4d3cf4f0537" modify_external_catalogs: true
I20250115 03:06:21.588045 14809 raft_consensus.cc:357] T 83a4f95eb9494a5295a81a8a51f58bd3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.588125 14809 raft_consensus.cc:383] T 83a4f95eb9494a5295a81a8a51f58bd3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.588150 14809 raft_consensus.cc:738] T 83a4f95eb9494a5295a81a8a51f58bd3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.588240 14809 consensus_queue.cc:260] T 83a4f95eb9494a5295a81a8a51f58bd3 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.588455 14809 ts_tablet_manager.cc:1428] T 83a4f95eb9494a5295a81a8a51f58bd3 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.588594 14809 tablet_bootstrap.cc:492] T 238e68300f304266aaed94bc3b76a01d P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.588797 14812 tablet_bootstrap.cc:492] T b08e5fc2110e476e897240f9cf3b874d P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.589062 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_c94a10c892a343a4bf3917501b740480"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.589335 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c94a10c892a343a4bf3917501b740480 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.589443 14809 tablet_bootstrap.cc:654] T 238e68300f304266aaed94bc3b76a01d P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.589681 14812 tablet_bootstrap.cc:654] T b08e5fc2110e476e897240f9cf3b874d P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.590605 14812 tablet_bootstrap.cc:492] T b08e5fc2110e476e897240f9cf3b874d P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.590590 14853 raft_consensus.cc:491] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.590682 14812 ts_tablet_manager.cc:1397] T b08e5fc2110e476e897240f9cf3b874d P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:21.590659 14853 raft_consensus.cc:513] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.590914 14853 leader_election.cc:290] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.591074 14812 raft_consensus.cc:357] T b08e5fc2110e476e897240f9cf3b874d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.591161 14812 raft_consensus.cc:383] T b08e5fc2110e476e897240f9cf3b874d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.591190 14812 raft_consensus.cc:738] T b08e5fc2110e476e897240f9cf3b874d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.591315 14812 consensus_queue.cc:260] T b08e5fc2110e476e897240f9cf3b874d P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.591516 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12678da7ffee4a46b4690ac3e94cb19b" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.591611 14689 raft_consensus.cc:2463] T 12678da7ffee4a46b4690ac3e94cb19b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.591768 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12678da7ffee4a46b4690ac3e94cb19b" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.591848 14493 leader_election.cc:304] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.591871 14421 raft_consensus.cc:2463] T 12678da7ffee4a46b4690ac3e94cb19b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.591984 14858 raft_consensus.cc:2798] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.592020 14858 raft_consensus.cc:491] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.592051 14858 raft_consensus.cc:3054] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.592860 14858 raft_consensus.cc:513] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.593185 14858 leader_election.cc:290] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.593518 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12678da7ffee4a46b4690ac3e94cb19b" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.591609 14812 ts_tablet_manager.cc:1428] T b08e5fc2110e476e897240f9cf3b874d P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.593612 14421 raft_consensus.cc:3054] T 12678da7ffee4a46b4690ac3e94cb19b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.593737 14812 tablet_bootstrap.cc:492] T 8efa998709034fe9bb37c7c608c99e76 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.594087 14811 tablet_bootstrap.cc:492] T ad0e9216bd154f4a8a37874d3d8a08c2 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.594156 14811 ts_tablet_manager.cc:1397] T ad0e9216bd154f4a8a37874d3d8a08c2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20250115 03:06:21.594525 14421 raft_consensus.cc:2463] T 12678da7ffee4a46b4690ac3e94cb19b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.594554 14811 raft_consensus.cc:357] T ad0e9216bd154f4a8a37874d3d8a08c2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.594637 14811 raft_consensus.cc:383] T ad0e9216bd154f4a8a37874d3d8a08c2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.594666 14811 raft_consensus.cc:738] T ad0e9216bd154f4a8a37874d3d8a08c2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.594697 14812 tablet_bootstrap.cc:654] T 8efa998709034fe9bb37c7c608c99e76 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.594764 14811 consensus_queue.cc:260] T ad0e9216bd154f4a8a37874d3d8a08c2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.594858 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12678da7ffee4a46b4690ac3e94cb19b" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:21.594945 14688 raft_consensus.cc:3054] T 12678da7ffee4a46b4690ac3e94cb19b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.595036 14811 ts_tablet_manager.cc:1428] T ad0e9216bd154f4a8a37874d3d8a08c2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.595197 14811 tablet_bootstrap.cc:492] T b2629ae9d9854ae4b6e7ddddbb6f341b P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.595815 14688 raft_consensus.cc:2463] T 12678da7ffee4a46b4690ac3e94cb19b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.596052 14811 tablet_bootstrap.cc:654] T b2629ae9d9854ae4b6e7ddddbb6f341b P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.596339 15153 consensus_queue.cc:1035] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:21.596751 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b86e66428504217af08f667f660d304" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:21.596843 14553 raft_consensus.cc:3054] T 7b86e66428504217af08f667f660d304 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.597584 14493 leader_election.cc:304] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:21.597731 14553 raft_consensus.cc:2463] T 7b86e66428504217af08f667f660d304 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:21.597949 14853 raft_consensus.cc:2798] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.598021 14853 raft_consensus.cc:695] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:21.598124 14853 consensus_queue.cc:237] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.598595 14811 tablet_bootstrap.cc:492] T b2629ae9d9854ae4b6e7ddddbb6f341b P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.598665 14811 ts_tablet_manager.cc:1397] T b2629ae9d9854ae4b6e7ddddbb6f341b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:21.599025 14811 raft_consensus.cc:357] T b2629ae9d9854ae4b6e7ddddbb6f341b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.599110 14811 raft_consensus.cc:383] T b2629ae9d9854ae4b6e7ddddbb6f341b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.599139 14811 raft_consensus.cc:738] T b2629ae9d9854ae4b6e7ddddbb6f341b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.599242 14811 consensus_queue.cc:260] T b2629ae9d9854ae4b6e7ddddbb6f341b P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.599468 14811 ts_tablet_manager.cc:1428] T b2629ae9d9854ae4b6e7ddddbb6f341b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.599610 14811 tablet_bootstrap.cc:492] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.600453 14811 tablet_bootstrap.cc:654] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.601703 14811 tablet_bootstrap.cc:492] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.601802 14811 ts_tablet_manager.cc:1397] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:21.602169 14811 raft_consensus.cc:357] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.602269 14811 raft_consensus.cc:383] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.602306 14811 raft_consensus.cc:738] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.602398 14811 consensus_queue.cc:260] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.602632 14811 ts_tablet_manager.cc:1428] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.602799 14811 tablet_bootstrap.cc:492] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.603796 14811 tablet_bootstrap.cc:654] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.606225 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f8d2c23a3c9d44a7b8959e53bd207f30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:21.606565 14165 catalog_manager.cc:5526] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:21.606988 14168 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 3bc1f0f72a3b4c79847c0a91fe98ccd1 (table default.Table_b165e5d12ca845b9984c99dde5e2e06e [id=dfe0ca35843c4fc6b975dd1ab87e9a99]). Expected version 1 got 0
I20250115 03:06:21.607546 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_3e458d4484e34d18adf6fd4268b0e647"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.607820 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3e458d4484e34d18adf6fd4268b0e647 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.609261 14171 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 3bc1f0f72a3b4c79847c0a91fe98ccd1 (table default.Table_b165e5d12ca845b9984c99dde5e2e06e [id=dfe0ca35843c4fc6b975dd1ab87e9a99]). Expected version 1 got 0
I20250115 03:06:21.609349 14171 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 781f6a9148cd4bd684f24c294c0b019b (table default.Table_d64c7e1e71a3478687247bda551f3fe2 [id=b9f32421ef8a4c549dab58ffb0f71fa7]). Expected version 1 got 0
I20250115 03:06:21.609802 14171 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_fb75e1b5f16f45dc9e9fc586e358dafb" } new_table_name: "default.Table_f09cc8e8d55d43f3a103cff429858fdc" modify_external_catalogs: true
I20250115 03:06:21.610410 14526 tablet_service.cc:1467] Processing CreateTablet for tablet 5b06b08f0a864629997efc0c98e84e04 (DEFAULT_TABLE table=default.Table_ef1516de522f4c0aa275c9752c4fe452 [id=5dbbe8f1a1ff48a382567a204c68f433]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.610637 14526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b06b08f0a864629997efc0c98e84e04. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.611099 14655 tablet_service.cc:1467] Processing CreateTablet for tablet 4888035519164539bed6c62f9123675e (DEFAULT_TABLE table=default.Table_2fabe46e26c64fdf9c13c69f53a754f4 [id=4f0f8e6739a64daa932102bb7d445901]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.611188 14668 tablet_service.cc:1467] Processing CreateTablet for tablet 5b06b08f0a864629997efc0c98e84e04 (DEFAULT_TABLE table=default.Table_ef1516de522f4c0aa275c9752c4fe452 [id=5dbbe8f1a1ff48a382567a204c68f433]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.611434 14656 tablet_service.cc:1467] Processing CreateTablet for tablet 4defe9264d30422c85c4fa4244685d70 (DEFAULT_TABLE table=default.Table_ed091b2f479b4435bd71157b60d837e3 [id=a03b097327924f67911bee9c6e49d48d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.612998 14657 tablet_service.cc:1467] Processing CreateTablet for tablet a5eda11ab1a24b5dbe7b177a49671cf5 (DEFAULT_TABLE table=default.Table_ccc15a33d119445e810777ee918a6541 [id=b07546f80dae4629a963326c1afd4b4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.613293 14858 raft_consensus.cc:491] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.613359 14858 raft_consensus.cc:513] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.613631 14858 leader_election.cc:290] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:21.606547 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f8d2c23a3c9d44a7b8959e53bd207f30 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.614909 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 5b06b08f0a864629997efc0c98e84e04 (DEFAULT_TABLE table=default.Table_ef1516de522f4c0aa275c9752c4fe452 [id=5dbbe8f1a1ff48a382567a204c68f433]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.615149 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b06b08f0a864629997efc0c98e84e04. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.615453 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 4888035519164539bed6c62f9123675e (DEFAULT_TABLE table=default.Table_2fabe46e26c64fdf9c13c69f53a754f4 [id=4f0f8e6739a64daa932102bb7d445901]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.615700 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4888035519164539bed6c62f9123675e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.616820 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4888035519164539bed6c62f9123675e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.618189 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b06b08f0a864629997efc0c98e84e04. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.618500 14811 tablet_bootstrap.cc:492] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.618574 14811 ts_tablet_manager.cc:1397] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20250115 03:06:21.618800 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfeafbf380df41738ebbcc11c19a7f85" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.618902 14689 raft_consensus.cc:2463] T bfeafbf380df41738ebbcc11c19a7f85 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.619009 14811 raft_consensus.cc:357] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.619098 14811 raft_consensus.cc:383] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.619130 14811 raft_consensus.cc:738] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.619195 14493 leader_election.cc:304] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.619230 14811 consensus_queue.cc:260] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.619472 14811 ts_tablet_manager.cc:1428] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.619622 14811 tablet_bootstrap.cc:492] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.620101 14656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4defe9264d30422c85c4fa4244685d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.620370 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfeafbf380df41738ebbcc11c19a7f85" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.620491 14811 tablet_bootstrap.cc:654] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.620507 14421 raft_consensus.cc:2463] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.621007 14398 tablet_service.cc:1467] Processing CreateTablet for tablet a5eda11ab1a24b5dbe7b177a49671cf5 (DEFAULT_TABLE table=default.Table_ccc15a33d119445e810777ee918a6541 [id=b07546f80dae4629a963326c1afd4b4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.621126 14858 raft_consensus.cc:2798] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.621177 14858 raft_consensus.cc:491] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.621204 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5eda11ab1a24b5dbe7b177a49671cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.621212 14858 raft_consensus.cc:3054] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.621948 15153 consensus_queue.cc:1035] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:21.622081 14858 raft_consensus.cc:513] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.622218 14657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5eda11ab1a24b5dbe7b177a49671cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.622375 14858 leader_election.cc:290] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.622828 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfeafbf380df41738ebbcc11c19a7f85" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:21.621165 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 4defe9264d30422c85c4fa4244685d70 (DEFAULT_TABLE table=default.Table_ed091b2f479b4435bd71157b60d837e3 [id=a03b097327924f67911bee9c6e49d48d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.622901 14689 raft_consensus.cc:3054] T bfeafbf380df41738ebbcc11c19a7f85 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.623006 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4defe9264d30422c85c4fa4244685d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.623482 14689 raft_consensus.cc:2463] T bfeafbf380df41738ebbcc11c19a7f85 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.623751 14493 leader_election.cc:304] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.623900 14853 raft_consensus.cc:2798] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.623965 14853 raft_consensus.cc:695] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:21.624076 14853 consensus_queue.cc:237] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.624325 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfeafbf380df41738ebbcc11c19a7f85" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.625836 14421 raft_consensus.cc:3054] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.626191 15617 tablet.cc:1721] T 3bc1f0f72a3b4c79847c0a91fe98ccd1 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:21.626580 14421 raft_consensus.cc:2463] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.626720 15614 mvcc.cc:204] Tried to move back new op lower bound from 7114384922810118144 to 7114384921428430848. Current Snapshot: MvccSnapshot[applied={T|T < 7114384922810118144}]
I20250115 03:06:21.630288 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 4defe9264d30422c85c4fa4244685d70 (DEFAULT_TABLE table=default.Table_ed091b2f479b4435bd71157b60d837e3 [id=a03b097327924f67911bee9c6e49d48d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.630520 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4defe9264d30422c85c4fa4244685d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.631208 14526 tablet_service.cc:1467] Processing CreateTablet for tablet 4888035519164539bed6c62f9123675e (DEFAULT_TABLE table=default.Table_2fabe46e26c64fdf9c13c69f53a754f4 [id=4f0f8e6739a64daa932102bb7d445901]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.631428 14526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4888035519164539bed6c62f9123675e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.632341 14162 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 676ec8b9dc8241d4ae40488d3ce8f61e (table default.Table_0b6cd33c30014981b78899a738418cbf [id=63edd09bfaaa4d56b4314ce760c87d3d]). Expected version 1 got 0
I20250115 03:06:21.632436 14162 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 83a4f95eb9494a5295a81a8a51f58bd3 (table default.Table_3e0a77ba78e7413588e3b4c24cfd0de4 [id=c8ca3cd85578452bb7b8659c9e1a4e2b]). Expected version 1 got 0
I20250115 03:06:21.632580 14530 tablet_service.cc:1467] Processing CreateTablet for tablet a5eda11ab1a24b5dbe7b177a49671cf5 (DEFAULT_TABLE table=default.Table_ccc15a33d119445e810777ee918a6541 [id=b07546f80dae4629a963326c1afd4b4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.632768 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5eda11ab1a24b5dbe7b177a49671cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.633072 15153 raft_consensus.cc:491] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.633152 15153 raft_consensus.cc:513] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.633496 15153 leader_election.cc:290] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
W20250115 03:06:21.633677 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bf6199cdd6e940d1970dd43d8090100d consensus state has no leader
I20250115 03:06:21.633913 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_582d4ea061bb4e07a300f5f5974ac57a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.634224 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_582d4ea061bb4e07a300f5f5974ac57a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.634524 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 676ec8b9dc8241d4ae40488d3ce8f61e (table default.Table_0b6cd33c30014981b78899a738418cbf [id=63edd09bfaaa4d56b4314ce760c87d3d]). Expected version 1 got 0
I20250115 03:06:21.634615 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet dda418ed197e4af2aa511d3f1851663e (table default.Table_5c360e43ddae405b9a2843fc20a9dfb9 [id=cbab87c958a346199acdad65ab75be3c]). Expected version 3 got 0
I20250115 03:06:21.634687 14162 catalog_manager.cc:5526] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:21.634845 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 7b86e66428504217af08f667f660d304 (table default.Table_341a3cd1c8c94ff4b05fc21960a4e9b4 [id=c096ce170874405ea740b9ea829f886e]). Expected version 2 got 0
I20250115 03:06:21.634925 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet b08e5fc2110e476e897240f9cf3b874d (table default.Table_6b6164bc5f2e4395b5d31047b8ef87d9 [id=f8f2a7ff6fdc40468de6ae9ba28c1427]). Expected version 4 got 0
I20250115 03:06:21.635129 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2ffd8bc4e3924dc6a952ee6b31e2b305"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.635324 14167 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5db9d5f6cecb454a90ad931d8d165bcc consensus state has no leader
W20250115 03:06:21.635423 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ffd8bc4e3924dc6a952ee6b31e2b305 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.638361 16406 raft_consensus.cc:491] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.638444 16406 raft_consensus.cc:513] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.638785 16406 leader_election.cc:290] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.639160 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f72c9bf04a3942ceacc37fd407bff033" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:21.639518 14356 leader_election.cc:343] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.640000 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f72c9bf04a3942ceacc37fd407bff033" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:21.641130 14358 leader_election.cc:343] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.641191 14358 leader_election.cc:304] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.641562 16411 raft_consensus.cc:2743] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.641716 14809 tablet_bootstrap.cc:492] T 238e68300f304266aaed94bc3b76a01d P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.641793 14809 ts_tablet_manager.cc:1397] T 238e68300f304266aaed94bc3b76a01d P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.053s user 0.001s sys 0.000s
I20250115 03:06:21.641789 14858 raft_consensus.cc:491] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.641861 14858 raft_consensus.cc:513] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.642166 14858 leader_election.cc:290] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.642228 14809 raft_consensus.cc:357] T 238e68300f304266aaed94bc3b76a01d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.642323 14809 raft_consensus.cc:383] T 238e68300f304266aaed94bc3b76a01d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.642350 14809 raft_consensus.cc:738] T 238e68300f304266aaed94bc3b76a01d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.642439 14809 consensus_queue.cc:260] T 238e68300f304266aaed94bc3b76a01d P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.642627 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "815ef85ce5cf4ae0a56c5734837e03ba" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.642705 14809 ts_tablet_manager.cc:1428] T 238e68300f304266aaed94bc3b76a01d P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.642742 14689 raft_consensus.cc:2463] T 815ef85ce5cf4ae0a56c5734837e03ba P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.642923 14809 tablet_bootstrap.cc:492] T 3e170ced1612442ebcfdaff6d9826ea3 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.642976 14493 leader_election.cc:304] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.643250 14858 raft_consensus.cc:2798] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.643299 14858 raft_consensus.cc:491] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.643334 14858 raft_consensus.cc:3054] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.643352 16411 raft_consensus.cc:491] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.643422 16411 raft_consensus.cc:513] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.643743 16411 leader_election.cc:290] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.643956 14809 tablet_bootstrap.cc:654] T 3e170ced1612442ebcfdaff6d9826ea3 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.644084 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b39608e19bf44b3a79a8616ccdad30a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.644172 14858 raft_consensus.cc:513] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:21.644398 14358 leader_election.cc:343] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.644481 14858 leader_election.cc:290] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.644734 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b39608e19bf44b3a79a8616ccdad30a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.644901 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "815ef85ce5cf4ae0a56c5734837e03ba" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:21.645010 14689 raft_consensus.cc:3054] T 815ef85ce5cf4ae0a56c5734837e03ba P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:21.645051 14356 leader_election.cc:343] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.645112 14356 leader_election.cc:304] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.645174 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b97baa78114e4eab889110d44a2da354"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:21.645391 16411 raft_consensus.cc:2743] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:21.645479 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b97baa78114e4eab889110d44a2da354 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:21.645571 14628 leader_election.cc:343] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: d6cf1a0b9c81413b8f72de0cafc76963
I20250115 03:06:21.645884 14689 raft_consensus.cc:2463] T 815ef85ce5cf4ae0a56c5734837e03ba P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.646140 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "815ef85ce5cf4ae0a56c5734837e03ba" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.646229 14419 raft_consensus.cc:3054] T 815ef85ce5cf4ae0a56c5734837e03ba P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.646378 14493 leader_election.cc:304] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.646512 16189 raft_consensus.cc:491] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.646570 16189 raft_consensus.cc:513] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.646849 16189 leader_election.cc:290] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.646924 16189 raft_consensus.cc:2798] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.646981 16189 raft_consensus.cc:695] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:21.647056 14419 raft_consensus.cc:2463] T 815ef85ce5cf4ae0a56c5734837e03ba P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.647082 16189 consensus_queue.cc:237] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.647239 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "815ef85ce5cf4ae0a56c5734837e03ba" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.647322 14422 raft_consensus.cc:2371] T 815ef85ce5cf4ae0a56c5734837e03ba P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1. Re-sending same reply.
I20250115 03:06:21.648559 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c580cdf3a854df988df32a9294c83bc with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.648697 14809 tablet_bootstrap.cc:492] T 3e170ced1612442ebcfdaff6d9826ea3 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.648769 14809 ts_tablet_manager.cc:1397] T 3e170ced1612442ebcfdaff6d9826ea3 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250115 03:06:21.649181 14809 raft_consensus.cc:357] T 3e170ced1612442ebcfdaff6d9826ea3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.649274 14809 raft_consensus.cc:383] T 3e170ced1612442ebcfdaff6d9826ea3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.649308 14809 raft_consensus.cc:738] T 3e170ced1612442ebcfdaff6d9826ea3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.649415 14809 consensus_queue.cc:260] T 3e170ced1612442ebcfdaff6d9826ea3 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.649629 14809 ts_tablet_manager.cc:1428] T 3e170ced1612442ebcfdaff6d9826ea3 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20250115 03:06:21.650372 14626 leader_election.cc:343] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Not found: Tablet not found: d6cf1a0b9c81413b8f72de0cafc76963
I20250115 03:06:21.650434 14626 leader_election.cc:304] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:21.650691 16292 raft_consensus.cc:2743] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.652138 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 15263a3b32be418288770ce348c082d7
I20250115 03:06:21.652390 14164 catalog_manager.cc:6596] ReplaceTablet: tablet 15263a3b32be418288770ce348c082d7 deleted and replaced by tablet d1daa457581a48ceb18bf617a60a0802
I20250115 03:06:21.653314 14809 tablet_bootstrap.cc:492] T a8a5743475814401ac2c0a51cfd70207 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
W20250115 03:06:21.653344 14171 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 18310ca8645747a18cafd6d9e647f9e3 consensus state has no leader
I20250115 03:06:21.654233 14809 tablet_bootstrap.cc:654] T a8a5743475814401ac2c0a51cfd70207 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.653314 14819 tablet_replica.cc:331] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.654552 14819 raft_consensus.cc:2238] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.654636 14819 raft_consensus.cc:2267] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.655136 14819 ts_tablet_manager.cc:1905] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:21.655150 14493 leader_election.cc:343] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Not found: Tablet not found: bb68869eaf55477090cf12f40f76a89c
I20250115 03:06:21.656090 16292 consensus_queue.cc:1035] T 3b5710268643446b91d04e2f4c7f999e P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:21.656102 16001 raft_consensus.cc:491] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.656390 16001 raft_consensus.cc:513] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.656682 16001 leader_election.cc:290] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:21.656976 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet f72c9bf04a3942ceacc37fd407bff033 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.657274 14819 ts_tablet_manager.cc:1918] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.657331 14819 log.cc:1198] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/0c580cdf3a854df988df32a9294c83bc
I20250115 03:06:21.657497 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f5554b10a44c249e4c3d3825cd7952" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.657523 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 15263a3b32be418288770ce348c082d7 with delete_type TABLET_DATA_DELETED (replaced by tablet d1daa457581a48ceb18bf617a60a0802) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.657593 14655 tablet_service.cc:1514] Processing DeleteTablet for tablet 15263a3b32be418288770ce348c082d7 with delete_type TABLET_DATA_DELETED (replaced by tablet d1daa457581a48ceb18bf617a60a0802) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.657599 14555 raft_consensus.cc:2463] T 38f5554b10a44c249e4c3d3825cd7952 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:21.657891 14626 leader_election.cc:304] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
W20250115 03:06:21.658015 14493 leader_election.cc:343] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: bb68869eaf55477090cf12f40f76a89c
I20250115 03:06:21.658068 14493 leader_election.cc:304] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:21.658180 16088 raft_consensus.cc:2798] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.658229 16088 raft_consensus.cc:491] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.658260 16088 raft_consensus.cc:3054] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.658239 14858 raft_consensus.cc:2743] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.659132 16088 raft_consensus.cc:513] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.657529 14819 ts_tablet_manager.cc:1939] T 0c580cdf3a854df988df32a9294c83bc P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.659443 16088 leader_election.cc:290] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:21.659766 14819 tablet_replica.cc:331] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.659868 14819 raft_consensus.cc:2238] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.659940 14819 raft_consensus.cc:2267] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.660914 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f5554b10a44c249e4c3d3825cd7952" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:21.661032 14555 raft_consensus.cc:3054] T 38f5554b10a44c249e4c3d3825cd7952 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.661156 14819 ts_tablet_manager.cc:1905] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.661933 14555 raft_consensus.cc:2463] T 38f5554b10a44c249e4c3d3825cd7952 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:21.662210 14626 leader_election.cc:304] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.662395 16292 raft_consensus.cc:2798] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.662472 16292 raft_consensus.cc:695] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:21.662580 16292 consensus_queue.cc:237] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.662902 14819 ts_tablet_manager.cc:1918] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.662958 14819 log.cc:1198] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/f72c9bf04a3942ceacc37fd407bff033
I20250115 03:06:21.663074 16292 raft_consensus.cc:491] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.663142 14819 ts_tablet_manager.cc:1939] T f72c9bf04a3942ceacc37fd407bff033 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.663134 16292 raft_consensus.cc:513] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.663398 16292 leader_election.cc:290] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
W20250115 03:06:21.666045 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 15263a3b32be418288770ce348c082d7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 15263a3b32be418288770ce348c082d7 already in progress: creating tablet
I20250115 03:06:21.666601 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2629ae9d9854ae4b6e7ddddbb6f341b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.666697 14555 raft_consensus.cc:2463] T b2629ae9d9854ae4b6e7ddddbb6f341b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:21.666920 14626 leader_election.cc:304] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.667450 16088 raft_consensus.cc:2798] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.667511 16088 raft_consensus.cc:491] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.667547 16088 raft_consensus.cc:3054] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.668183 14811 tablet_bootstrap.cc:492] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.668258 14811 ts_tablet_manager.cc:1397] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.049s user 0.001s sys 0.001s
I20250115 03:06:21.668336 16088 raft_consensus.cc:513] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.668648 16088 leader_election.cc:290] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:21.668663 14811 raft_consensus.cc:357] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.668756 14811 raft_consensus.cc:383] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.668784 14811 raft_consensus.cc:738] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.668903 14811 consensus_queue.cc:260] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.668991 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2629ae9d9854ae4b6e7ddddbb6f341b" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:21.669062 14555 raft_consensus.cc:3054] T b2629ae9d9854ae4b6e7ddddbb6f341b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.669229 14811 ts_tablet_manager.cc:1428] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:21.669390 14811 tablet_bootstrap.cc:492] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.669867 14555 raft_consensus.cc:2463] T b2629ae9d9854ae4b6e7ddddbb6f341b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:21.670176 14626 leader_election.cc:304] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.670358 14811 tablet_bootstrap.cc:654] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.670635 16088 raft_consensus.cc:2798] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.670776 16088 raft_consensus.cc:695] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:21.670893 16088 consensus_queue.cc:237] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.672852 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 0c580cdf3a854df988df32a9294c83bc (table default.Table_db820168492749099144481f4895d059 [id=e0b2dd40d51e490c8fa2a5ca2c166f91]) successfully deleted
I20250115 03:06:21.672865 15153 raft_consensus.cc:491] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.672956 15153 raft_consensus.cc:513] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.673211 15153 leader_election.cc:290] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:21.673277 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f5554b10a44c249e4c3d3825cd7952" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.673368 14422 raft_consensus.cc:2463] T 38f5554b10a44c249e4c3d3825cd7952 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:21.673424 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet f72c9bf04a3942ceacc37fd407bff033 (table default.Table_5ddb4f54525d4439b54c2c898471fa7e [id=85a660e149c4457fa9bc97df0f6f06e1]) successfully deleted
W20250115 03:06:21.673579 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 15263a3b32be418288770ce348c082d7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 15263a3b32be418288770ce348c082d7 already in progress: creating tablet
I20250115 03:06:21.673696 14811 tablet_bootstrap.cc:492] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.673702 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f5554b10a44c249e4c3d3825cd7952" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.673776 14811 ts_tablet_manager.cc:1397] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20250115 03:06:21.673804 14420 raft_consensus.cc:3054] T 38f5554b10a44c249e4c3d3825cd7952 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.674154 14811 raft_consensus.cc:357] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.674245 14811 raft_consensus.cc:383] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.674278 14811 raft_consensus.cc:738] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.674377 14811 consensus_queue.cc:260] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.674437 14418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7043616bd6b34299862336924a6f16bf" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.674535 14418 raft_consensus.cc:2463] T 7043616bd6b34299862336924a6f16bf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:21.674626 14811 ts_tablet_manager.cc:1428] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.674620 14420 raft_consensus.cc:2463] T 38f5554b10a44c249e4c3d3825cd7952 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:21.674790 14811 tablet_bootstrap.cc:492] T 72abf6acd9d54f3b983b7098aaf2691d P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
W20250115 03:06:21.675072 14628 leader_election.cc:343] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: b2629ae9d9854ae4b6e7ddddbb6f341b
W20250115 03:06:21.675217 14628 leader_election.cc:343] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: b2629ae9d9854ae4b6e7ddddbb6f341b
I20250115 03:06:21.675309 14628 leader_election.cc:304] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:21.675649 14811 tablet_bootstrap.cc:654] T 72abf6acd9d54f3b983b7098aaf2691d P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.675640 16292 raft_consensus.cc:2798] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.675684 16292 raft_consensus.cc:491] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.675716 16292 raft_consensus.cc:3054] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.676376 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7043616bd6b34299862336924a6f16bf" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.676483 14555 raft_consensus.cc:2463] T 7043616bd6b34299862336924a6f16bf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:21.676463 16292 raft_consensus.cc:513] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.676759 16292 leader_election.cc:290] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:21.678426 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7043616bd6b34299862336924a6f16bf" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.678524 14420 raft_consensus.cc:3054] T 7043616bd6b34299862336924a6f16bf P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.679198 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7043616bd6b34299862336924a6f16bf" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:21.679286 14555 raft_consensus.cc:3054] T 7043616bd6b34299862336924a6f16bf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.679273 14420 raft_consensus.cc:2463] T 7043616bd6b34299862336924a6f16bf P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:21.679647 14811 tablet_bootstrap.cc:492] T 72abf6acd9d54f3b983b7098aaf2691d P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.679746 14811 ts_tablet_manager.cc:1397] T 72abf6acd9d54f3b983b7098aaf2691d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:21.680037 14555 raft_consensus.cc:2463] T 7043616bd6b34299862336924a6f16bf P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:21.680078 14628 leader_election.cc:304] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:21.680130 14811 raft_consensus.cc:357] T 72abf6acd9d54f3b983b7098aaf2691d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.680231 14811 raft_consensus.cc:383] T 72abf6acd9d54f3b983b7098aaf2691d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.680260 14811 raft_consensus.cc:738] T 72abf6acd9d54f3b983b7098aaf2691d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.680285 15153 raft_consensus.cc:2798] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.680362 15153 raft_consensus.cc:695] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:21.680347 14811 consensus_queue.cc:260] T 72abf6acd9d54f3b983b7098aaf2691d P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.680459 15153 consensus_queue.cc:237] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.680586 14811 ts_tablet_manager.cc:1428] T 72abf6acd9d54f3b983b7098aaf2691d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.680747 14811 tablet_bootstrap.cc:492] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.681613 14811 tablet_bootstrap.cc:654] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.684569 14170 master_service.cc:946] ReplaceTablet: received request to replace tablet 6f53ad530584487e946120a724f99cfa from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:21.684968 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f2946c2c8c56478f9108ffeda4034086" } modify_external_catalogs: true
I20250115 03:06:21.685065 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f2946c2c8c56478f9108ffeda4034086" } modify_external_catalogs: true
I20250115 03:06:21.685777 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_03db1a048e7a4aff8c957fbc41cb4675" } modify_external_catalogs: true
I20250115 03:06:21.685863 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_03db1a048e7a4aff8c957fbc41cb4675" } modify_external_catalogs: true
I20250115 03:06:21.686743 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 15263a3b32be418288770ce348c082d7 with delete_type TABLET_DATA_DELETED (replaced by tablet d1daa457581a48ceb18bf617a60a0802) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.687727 14526 tablet_service.cc:1514] Processing DeleteTablet for tablet bb68869eaf55477090cf12f40f76a89c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.688011 16411 raft_consensus.cc:491] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.688093 16411 raft_consensus.cc:513] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.688431 16411 leader_election.cc:290] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.688591 14820 tablet_replica.cc:331] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:21.688740 14820 raft_consensus.cc:2238] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.688823 14820 raft_consensus.cc:2267] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.688887 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68a04d57feea40eeb8e06cb6a2c61119" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:21.689229 14358 leader_election.cc:343] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.689358 14858 raft_consensus.cc:491] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.689440 14858 raft_consensus.cc:513] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.689764 14858 leader_election.cc:290] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.690546 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8efa998709034fe9bb37c7c608c99e76" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.690711 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8efa998709034fe9bb37c7c608c99e76" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.690821 14420 raft_consensus.cc:2463] T 8efa998709034fe9bb37c7c608c99e76 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
W20250115 03:06:21.690842 14493 leader_election.cc:343] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.691159 14493 leader_election.cc:304] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770
I20250115 03:06:21.691677 14820 ts_tablet_manager.cc:1905] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.693040 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68a04d57feea40eeb8e06cb6a2c61119" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:21.693383 14356 leader_election.cc:343] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.693451 14356 leader_election.cc:304] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.693926 14820 ts_tablet_manager.cc:1918] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.693998 14820 log.cc:1198] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/bb68869eaf55477090cf12f40f76a89c
I20250115 03:06:21.693974 16411 raft_consensus.cc:2743] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.694188 14820 ts_tablet_manager.cc:1939] T bb68869eaf55477090cf12f40f76a89c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:21.694759 14812 tablet_bootstrap.cc:492] T 8efa998709034fe9bb37c7c608c99e76 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.694751 14858 raft_consensus.cc:2798] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.694802 14858 raft_consensus.cc:491] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.694841 14858 raft_consensus.cc:3054] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.694844 14812 ts_tablet_manager.cc:1397] T 8efa998709034fe9bb37c7c608c99e76 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.101s user 0.002s sys 0.000s
I20250115 03:06:21.695206 14812 raft_consensus.cc:357] T 8efa998709034fe9bb37c7c608c99e76 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.695288 14812 raft_consensus.cc:383] T 8efa998709034fe9bb37c7c608c99e76 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.695317 14812 raft_consensus.cc:738] T 8efa998709034fe9bb37c7c608c99e76 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.695442 14812 consensus_queue.cc:260] T 8efa998709034fe9bb37c7c608c99e76 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.695595 14858 raft_consensus.cc:513] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.695703 14812 ts_tablet_manager.cc:1428] T 8efa998709034fe9bb37c7c608c99e76 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.695885 14812 tablet_bootstrap.cc:492] T 0e1ce43146ed4e4d82a7c134216e2872 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.695916 14858 leader_election.cc:290] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.696350 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8efa998709034fe9bb37c7c608c99e76" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:21.696455 14689 raft_consensus.cc:3054] T 8efa998709034fe9bb37c7c608c99e76 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:21.693599 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 15263a3b32be418288770ce348c082d7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 15263a3b32be418288770ce348c082d7 already in progress: creating tablet
I20250115 03:06:21.696810 14812 tablet_bootstrap.cc:654] T 0e1ce43146ed4e4d82a7c134216e2872 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.697299 14809 tablet_bootstrap.cc:492] T a8a5743475814401ac2c0a51cfd70207 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.697332 14689 raft_consensus.cc:2463] T 8efa998709034fe9bb37c7c608c99e76 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.697381 14809 ts_tablet_manager.cc:1397] T a8a5743475814401ac2c0a51cfd70207 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.044s user 0.002s sys 0.000s
I20250115 03:06:21.697719 14166 master_service.cc:946] ReplaceTablet: received request to replace tablet 7ac903c81a1f481ab587e6772c306a96 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:21.697783 14809 raft_consensus.cc:357] T a8a5743475814401ac2c0a51cfd70207 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.697877 14809 raft_consensus.cc:383] T a8a5743475814401ac2c0a51cfd70207 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.697907 14809 raft_consensus.cc:738] T a8a5743475814401ac2c0a51cfd70207 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.698009 14809 consensus_queue.cc:260] T a8a5743475814401ac2c0a51cfd70207 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.698263 14809 ts_tablet_manager.cc:1428] T a8a5743475814401ac2c0a51cfd70207 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.698415 14809 tablet_bootstrap.cc:492] T 194d943d173340e7a8fba69967d9b090 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.699255 14809 tablet_bootstrap.cc:654] T 194d943d173340e7a8fba69967d9b090 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.699353 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8efa998709034fe9bb37c7c608c99e76" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.699443 14421 raft_consensus.cc:3054] T 8efa998709034fe9bb37c7c608c99e76 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.700343 14421 raft_consensus.cc:2463] T 8efa998709034fe9bb37c7c608c99e76 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.700552 14493 leader_election.cc:304] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.700894 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet ada45621f8864caab9189039fae0fae2
I20250115 03:06:21.701784 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet ada45621f8864caab9189039fae0fae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.701964 14819 tablet_replica.cc:331] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.702080 14819 raft_consensus.cc:2238] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.702167 14819 raft_consensus.cc:2267] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.702294 14858 raft_consensus.cc:2798] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.702363 14858 raft_consensus.cc:695] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:21.702466 14858 consensus_queue.cc:237] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.703020 14819 ts_tablet_manager.cc:1905] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.705029 14819 ts_tablet_manager.cc:1918] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:21.705083 14819 log.cc:1198] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/ada45621f8864caab9189039fae0fae2
I20250115 03:06:21.705294 14819 ts_tablet_manager.cc:1939] T ada45621f8864caab9189039fae0fae2 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.705304 14811 tablet_bootstrap.cc:492] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.705384 14811 ts_tablet_manager.cc:1397] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.000s
I20250115 03:06:21.705794 14811 raft_consensus.cc:357] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.705883 14811 raft_consensus.cc:383] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.705914 14811 raft_consensus.cc:738] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.706012 14811 consensus_queue.cc:260] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.706257 14811 ts_tablet_manager.cc:1428] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.706414 14811 tablet_bootstrap.cc:492] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.707008 14655 tablet_service.cc:1514] Processing DeleteTablet for tablet ada45621f8864caab9189039fae0fae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.707139 14823 tablet_replica.cc:331] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:21.707247 14823 raft_consensus.cc:2238] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:21.707387 14811 tablet_bootstrap.cc:654] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.707397 14823 raft_consensus.cc:2267] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.707880 14823 ts_tablet_manager.cc:1905] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.708527 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet bb68869eaf55477090cf12f40f76a89c (table default.Table_0964201dda974eaabef3d5694c40389c [id=4a6fefd2006a4b11b121bfad61cf95ed]) successfully deleted
I20250115 03:06:21.708993 14811 tablet_bootstrap.cc:492] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.709064 14811 ts_tablet_manager.cc:1397] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:21.709313 14809 tablet_bootstrap.cc:492] T 194d943d173340e7a8fba69967d9b090 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.709381 14809 ts_tablet_manager.cc:1397] T 194d943d173340e7a8fba69967d9b090 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20250115 03:06:21.709472 14811 raft_consensus.cc:357] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.709558 14811 raft_consensus.cc:383] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.709589 14811 raft_consensus.cc:738] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.709689 14811 consensus_queue.cc:260] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.709795 14809 raft_consensus.cc:357] T 194d943d173340e7a8fba69967d9b090 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.709887 14809 raft_consensus.cc:383] T 194d943d173340e7a8fba69967d9b090 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.709918 14809 raft_consensus.cc:738] T 194d943d173340e7a8fba69967d9b090 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.709956 14823 ts_tablet_manager.cc:1918] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:21.709967 14811 ts_tablet_manager.cc:1428] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.710011 14823 log.cc:1198] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/ada45621f8864caab9189039fae0fae2
I20250115 03:06:21.710037 14809 consensus_queue.cc:260] T 194d943d173340e7a8fba69967d9b090 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.710114 14811 tablet_bootstrap.cc:492] T 4b39608e19bf44b3a79a8616ccdad30a P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.710224 14823 ts_tablet_manager.cc:1939] T ada45621f8864caab9189039fae0fae2 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:21.710292 14809 ts_tablet_manager.cc:1428] T 194d943d173340e7a8fba69967d9b090 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.710440 14809 tablet_bootstrap.cc:492] T e47012a789bd469f9691e5a4162ff512 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.710959 14811 tablet_bootstrap.cc:654] T 4b39608e19bf44b3a79a8616ccdad30a P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.711444 14809 tablet_bootstrap.cc:654] T e47012a789bd469f9691e5a4162ff512 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.711521 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ccd2815995fc485d827b79ae818fedae" } modify_external_catalogs: true
I20250115 03:06:21.711616 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ccd2815995fc485d827b79ae818fedae" } modify_external_catalogs: true
I20250115 03:06:21.711692 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet ada45621f8864caab9189039fae0fae2 (table default.Table_f2946c2c8c56478f9108ffeda4034086 [id=2c3fb664638b434397824dec2147518c]) successfully deleted
I20250115 03:06:21.711997 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet ada45621f8864caab9189039fae0fae2 (table default.Table_f2946c2c8c56478f9108ffeda4034086 [id=2c3fb664638b434397824dec2147518c]) successfully deleted
I20250115 03:06:21.712078 14526 tablet_service.cc:1514] Processing DeleteTablet for tablet ada45621f8864caab9189039fae0fae2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.712821 14820 tablet_replica.cc:331] T ada45621f8864caab9189039fae0fae2 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:21.712958 14820 raft_consensus.cc:2238] T ada45621f8864caab9189039fae0fae2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.713049 14820 raft_consensus.cc:2267] T ada45621f8864caab9189039fae0fae2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.713770 14820 ts_tablet_manager.cc:1905] T ada45621f8864caab9189039fae0fae2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.714313 14812 tablet_bootstrap.cc:492] T 0e1ce43146ed4e4d82a7c134216e2872 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.714398 14812 ts_tablet_manager.cc:1397] T 0e1ce43146ed4e4d82a7c134216e2872 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.002s
I20250115 03:06:21.714820 14812 raft_consensus.cc:357] T 0e1ce43146ed4e4d82a7c134216e2872 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.714921 14812 raft_consensus.cc:383] T 0e1ce43146ed4e4d82a7c134216e2872 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.714958 14812 raft_consensus.cc:738] T 0e1ce43146ed4e4d82a7c134216e2872 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.715065 14812 consensus_queue.cc:260] T 0e1ce43146ed4e4d82a7c134216e2872 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.715338 14812 ts_tablet_manager.cc:1428] T 0e1ce43146ed4e4d82a7c134216e2872 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:21.715530 14812 tablet_bootstrap.cc:492] T 7f9818d0493547358fd5654a01d4f456 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.715857 14820 ts_tablet_manager.cc:1918] T ada45621f8864caab9189039fae0fae2 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:21.715926 14820 log.cc:1198] T ada45621f8864caab9189039fae0fae2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/ada45621f8864caab9189039fae0fae2
I20250115 03:06:21.716164 14820 ts_tablet_manager.cc:1939] T ada45621f8864caab9189039fae0fae2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:21.716395 14812 tablet_bootstrap.cc:654] T 7f9818d0493547358fd5654a01d4f456 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.717322 14812 tablet_bootstrap.cc:492] T 7f9818d0493547358fd5654a01d4f456 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.717384 14812 ts_tablet_manager.cc:1397] T 7f9818d0493547358fd5654a01d4f456 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20250115 03:06:21.717832 14812 raft_consensus.cc:357] T 7f9818d0493547358fd5654a01d4f456 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.717912 14812 raft_consensus.cc:383] T 7f9818d0493547358fd5654a01d4f456 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.717938 14812 raft_consensus.cc:738] T 7f9818d0493547358fd5654a01d4f456 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.718806 14812 consensus_queue.cc:260] T 7f9818d0493547358fd5654a01d4f456 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.719199 14809 tablet_bootstrap.cc:492] T e47012a789bd469f9691e5a4162ff512 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.719187 14853 raft_consensus.cc:491] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.719272 14809 ts_tablet_manager.cc:1397] T e47012a789bd469f9691e5a4162ff512 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:21.719257 14853 raft_consensus.cc:513] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.719569 14853 leader_election.cc:290] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.719668 14809 raft_consensus.cc:357] T e47012a789bd469f9691e5a4162ff512 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.719767 14809 raft_consensus.cc:383] T e47012a789bd469f9691e5a4162ff512 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.719798 14809 raft_consensus.cc:738] T e47012a789bd469f9691e5a4162ff512 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.719898 14809 consensus_queue.cc:260] T e47012a789bd469f9691e5a4162ff512 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.719990 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5db9d5f6cecb454a90ad931d8d165bcc" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.720144 14809 ts_tablet_manager.cc:1428] T e47012a789bd469f9691e5a4162ff512 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.720294 14809 tablet_bootstrap.cc:492] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.720316 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5db9d5f6cecb454a90ad931d8d165bcc" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.720412 14421 raft_consensus.cc:2463] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.720460 14812 ts_tablet_manager.cc:1428] T 7f9818d0493547358fd5654a01d4f456 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:21.720621 14812 tablet_bootstrap.cc:492] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:21.720609 14493 leader_election.cc:343] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.720899 14493 leader_election.cc:304] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770
I20250115 03:06:21.721104 14853 raft_consensus.cc:2798] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.721149 14853 raft_consensus.cc:491] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.721185 14853 raft_consensus.cc:3054] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.721460 14812 tablet_bootstrap.cc:654] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.721976 14853 raft_consensus.cc:513] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.722294 14853 leader_election.cc:290] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.723014 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5db9d5f6cecb454a90ad931d8d165bcc" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:21.723956 14809 tablet_bootstrap.cc:654] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.725481 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5db9d5f6cecb454a90ad931d8d165bcc" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.725579 14421 raft_consensus.cc:3054] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.725571 14655 tablet_service.cc:1467] Processing CreateTablet for tablet 96c95bb6890d423790083f6787a4b61c (DEFAULT_TABLE table=default.Table_99c8d02547a54f8fa40ee2aebde0dbb4 [id=d9a0925889c847a6a9f98dd0c197e84b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.725782 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96c95bb6890d423790083f6787a4b61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.725781 14169 catalog_manager.cc:5526] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:21.725984 14169 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet b08e5fc2110e476e897240f9cf3b874d (table default.Table_6b6164bc5f2e4395b5d31047b8ef87d9 [id=f8f2a7ff6fdc40468de6ae9ba28c1427]). Expected version 4 got 0
I20250115 03:06:21.726060 14169 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 676ec8b9dc8241d4ae40488d3ce8f61e (table default.Table_0b6cd33c30014981b78899a738418cbf [id=63edd09bfaaa4d56b4314ce760c87d3d]). Expected version 1 got 0
I20250115 03:06:21.726152 14169 catalog_manager.cc:5526] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:21.726384 14169 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 72abf6acd9d54f3b983b7098aaf2691d (table default.Table_5e72c1c96c004205bf7304be210f76f1 [id=5ca0a4e7179649d3870fdadc05eaa427]). Expected version 1 got 0
I20250115 03:06:21.726469 14421 raft_consensus.cc:2463] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.726481 14169 catalog_manager.cc:5526] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:21.726627 14169 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet bfeafbf380df41738ebbcc11c19a7f85 (table default.Table_660872e60c104e56be9287f66d74c387 [id=03ee00da1e024ebab51b3a366d2c0cf3]). Expected version 2 got 0
I20250115 03:06:21.726725 14169 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 8efa998709034fe9bb37c7c608c99e76 (table default.Table_b6002fb5cc1248bba2eb7abbd5a5d407 [id=4c458a80b1a2490da1cf689843cae584]). Expected version 1 got 0
I20250115 03:06:21.726836 14169 catalog_manager.cc:5526] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:21.726997 14169 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet a4547465f3124e108dc2c4c54e2bc751 (table default.Table_58e0901d0ec94ef9859515f126825621 [id=50c73c26b4ed47a4961348db433b7dd4]). Expected version 3 got 0
I20250115 03:06:21.727083 14169 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet dda418ed197e4af2aa511d3f1851663e (table default.Table_5c360e43ddae405b9a2843fc20a9dfb9 [id=cbab87c958a346199acdad65ab75be3c]). Expected version 3 got 0
I20250115 03:06:21.727083 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 96c95bb6890d423790083f6787a4b61c (DEFAULT_TABLE table=default.Table_99c8d02547a54f8fa40ee2aebde0dbb4 [id=d9a0925889c847a6a9f98dd0c197e84b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.727295 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96c95bb6890d423790083f6787a4b61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.727535 14162 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_b165e5d12ca845b9984c99dde5e2e06e [id=dfe0ca35843c4fc6b975dd1ab87e9a99] alter complete (version 1)
I20250115 03:06:21.727715 14657 tablet_service.cc:1467] Processing CreateTablet for tablet 7e8379764ae84a83887d3d8270a8717f (DEFAULT_TABLE table=default.Table_de7f79ee6adb4796911bd6e9f106da1f [id=b99a743f162c409b9149ae4b30a4e5cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.727937 14657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e8379764ae84a83887d3d8270a8717f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.728050 14493 leader_election.cc:304] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
W20250115 03:06:21.728195 14493 leader_election.cc:343] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.729413 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 7e8379764ae84a83887d3d8270a8717f (DEFAULT_TABLE table=default.Table_de7f79ee6adb4796911bd6e9f106da1f [id=b99a743f162c409b9149ae4b30a4e5cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.729615 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e8379764ae84a83887d3d8270a8717f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.729844 14853 raft_consensus.cc:2798] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.729918 14853 raft_consensus.cc:695] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:21.730018 14853 consensus_queue.cc:237] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.732589 14526 tablet_service.cc:1467] Processing CreateTablet for tablet 96c95bb6890d423790083f6787a4b61c (DEFAULT_TABLE table=default.Table_99c8d02547a54f8fa40ee2aebde0dbb4 [id=d9a0925889c847a6a9f98dd0c197e84b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.732820 14526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96c95bb6890d423790083f6787a4b61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.735509 14526 tablet_service.cc:1467] Processing CreateTablet for tablet 7e8379764ae84a83887d3d8270a8717f (DEFAULT_TABLE table=default.Table_de7f79ee6adb4796911bd6e9f106da1f [id=b99a743f162c409b9149ae4b30a4e5cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.735728 14526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e8379764ae84a83887d3d8270a8717f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.737178 14809 tablet_bootstrap.cc:492] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.737246 14809 ts_tablet_manager.cc:1397] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.000s
I20250115 03:06:21.737648 14809 raft_consensus.cc:357] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.737759 14809 raft_consensus.cc:383] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.737790 14809 raft_consensus.cc:738] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.737888 14809 consensus_queue.cc:260] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.738009 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet ada45621f8864caab9189039fae0fae2 (table default.Table_f2946c2c8c56478f9108ffeda4034086 [id=2c3fb664638b434397824dec2147518c]) successfully deleted
I20250115 03:06:21.738147 14809 ts_tablet_manager.cc:1428] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.738296 14809 tablet_bootstrap.cc:492] T 6132d15b17ab4d0aaa10a7ea04cda7ac P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.739135 14809 tablet_bootstrap.cc:654] T 6132d15b17ab4d0aaa10a7ea04cda7ac P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.739221 14811 tablet_bootstrap.cc:492] T 4b39608e19bf44b3a79a8616ccdad30a P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.739290 14811 ts_tablet_manager.cc:1397] T 4b39608e19bf44b3a79a8616ccdad30a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.000s
I20250115 03:06:21.739724 14811 raft_consensus.cc:357] T 4b39608e19bf44b3a79a8616ccdad30a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.739809 14811 raft_consensus.cc:383] T 4b39608e19bf44b3a79a8616ccdad30a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.739835 14811 raft_consensus.cc:738] T 4b39608e19bf44b3a79a8616ccdad30a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.739929 14811 consensus_queue.cc:260] T 4b39608e19bf44b3a79a8616ccdad30a P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.740193 14811 ts_tablet_manager.cc:1428] T 4b39608e19bf44b3a79a8616ccdad30a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.740355 14811 tablet_bootstrap.cc:492] T 6ed47b26cdf744acb28fb657a1b41087 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.741210 14811 tablet_bootstrap.cc:654] T 6ed47b26cdf744acb28fb657a1b41087 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.742373 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_9e42df4c569c47b698c9bbb6dcf39736" } modify_external_catalogs: true
I20250115 03:06:21.742470 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_9e42df4c569c47b698c9bbb6dcf39736" } modify_external_catalogs: true
I20250115 03:06:21.743110 14811 tablet_bootstrap.cc:492] T 6ed47b26cdf744acb28fb657a1b41087 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.743178 14811 ts_tablet_manager.cc:1397] T 6ed47b26cdf744acb28fb657a1b41087 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:21.743542 14811 raft_consensus.cc:357] T 6ed47b26cdf744acb28fb657a1b41087 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.743625 14811 raft_consensus.cc:383] T 6ed47b26cdf744acb28fb657a1b41087 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.743654 14811 raft_consensus.cc:738] T 6ed47b26cdf744acb28fb657a1b41087 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.743784 14811 consensus_queue.cc:260] T 6ed47b26cdf744acb28fb657a1b41087 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.744098 14811 ts_tablet_manager.cc:1428] T 6ed47b26cdf744acb28fb657a1b41087 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.744244 14811 tablet_bootstrap.cc:492] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.745064 14811 tablet_bootstrap.cc:654] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.745903 14811 tablet_bootstrap.cc:492] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.745967 14811 ts_tablet_manager.cc:1397] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:21.746317 14811 raft_consensus.cc:357] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.746403 14811 raft_consensus.cc:383] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.746431 14811 raft_consensus.cc:738] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.746542 14811 consensus_queue.cc:260] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.746764 14811 ts_tablet_manager.cc:1428] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.746882 14809 tablet_bootstrap.cc:492] T 6132d15b17ab4d0aaa10a7ea04cda7ac P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.746896 14811 tablet_bootstrap.cc:492] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.746953 14809 ts_tablet_manager.cc:1397] T 6132d15b17ab4d0aaa10a7ea04cda7ac P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:21.747345 14809 raft_consensus.cc:357] T 6132d15b17ab4d0aaa10a7ea04cda7ac P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.747431 14809 raft_consensus.cc:383] T 6132d15b17ab4d0aaa10a7ea04cda7ac P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.747458 14809 raft_consensus.cc:738] T 6132d15b17ab4d0aaa10a7ea04cda7ac P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.747596 14809 consensus_queue.cc:260] T 6132d15b17ab4d0aaa10a7ea04cda7ac P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.747704 14811 tablet_bootstrap.cc:654] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.749748 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 00aad81f4ca3427cb683de71148cff5c
I20250115 03:06:21.749975 14809 ts_tablet_manager.cc:1428] T 6132d15b17ab4d0aaa10a7ea04cda7ac P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:21.750150 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 7ac903c81a1f481ab587e6772c306a96
I20250115 03:06:21.750349 14166 catalog_manager.cc:6596] ReplaceTablet: tablet 7ac903c81a1f481ab587e6772c306a96 deleted and replaced by tablet 607fc76ce73744c4b8ac5975515e1fa3
I20250115 03:06:21.750523 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 802a007b8d154198b291aaf9530161b9
I20250115 03:06:21.750669 14809 tablet_bootstrap.cc:492] T a3eaec2fff184aad816349fc36fa3db1 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.751683 14809 tablet_bootstrap.cc:654] T a3eaec2fff184aad816349fc36fa3db1 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.752956 14657 tablet_service.cc:1514] Processing DeleteTablet for tablet 00aad81f4ca3427cb683de71148cff5c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.753101 14655 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ac903c81a1f481ab587e6772c306a96 with delete_type TABLET_DATA_DELETED (replaced by tablet 607fc76ce73744c4b8ac5975515e1fa3) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.753197 14656 tablet_service.cc:1514] Processing DeleteTablet for tablet 802a007b8d154198b291aaf9530161b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.753463 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 00aad81f4ca3427cb683de71148cff5c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.753588 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ac903c81a1f481ab587e6772c306a96 with delete_type TABLET_DATA_DELETED (replaced by tablet 607fc76ce73744c4b8ac5975515e1fa3) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.753680 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet 802a007b8d154198b291aaf9530161b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.753880 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2e115074bf7049bfbb1239b4d9c1ca2e" } modify_external_catalogs: true
I20250115 03:06:21.753971 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2e115074bf7049bfbb1239b4d9c1ca2e" } modify_external_catalogs: true
W20250115 03:06:21.754226 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 00aad81f4ca3427cb683de71148cff5c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 00aad81f4ca3427cb683de71148cff5c already in progress: creating tablet
W20250115 03:06:21.754343 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 7ac903c81a1f481ab587e6772c306a96 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7ac903c81a1f481ab587e6772c306a96 already in progress: creating tablet
W20250115 03:06:21.754455 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet 802a007b8d154198b291aaf9530161b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 802a007b8d154198b291aaf9530161b9 already in progress: creating tablet
W20250115 03:06:21.754602 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 00aad81f4ca3427cb683de71148cff5c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 00aad81f4ca3427cb683de71148cff5c already in progress: creating tablet
W20250115 03:06:21.754714 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 7ac903c81a1f481ab587e6772c306a96 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7ac903c81a1f481ab587e6772c306a96 already in progress: creating tablet
W20250115 03:06:21.755168 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet 802a007b8d154198b291aaf9530161b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 802a007b8d154198b291aaf9530161b9 already in progress: creating tablet
I20250115 03:06:21.755326 14656 tablet_service.cc:1514] Processing DeleteTablet for tablet b2629ae9d9854ae4b6e7ddddbb6f341b with delete_type TABLET_DATA_DELETED (replaced by tablet 103f78d0d9fc42308e1e23316895be33) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.755628 14655 tablet_service.cc:1514] Processing DeleteTablet for tablet d6cf1a0b9c81413b8f72de0cafc76963 with delete_type TABLET_DATA_DELETED (replaced by tablet a9807cb8c17d4b98be44e6fd5dccc150) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.755877 14823 tablet_replica.cc:331] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:21.755950 14812 tablet_bootstrap.cc:492] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.755995 14823 raft_consensus.cc:2238] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:21.756011 14812 ts_tablet_manager.cc:1397] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.035s user 0.001s sys 0.000s
I20250115 03:06:21.756057 16189 raft_consensus.cc:491] T 6ed47b26cdf744acb28fb657a1b41087 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.756122 14823 pending_rounds.cc:62] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770: Trying to abort 1 pending ops.
I20250115 03:06:21.756127 16189 raft_consensus.cc:513] T 6ed47b26cdf744acb28fb657a1b41087 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.756148 14823 pending_rounds.cc:69] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384923325104128 op_type: NO_OP noop_request { }
I20250115 03:06:21.756199 14823 raft_consensus.cc:2883] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:21.756243 14823 raft_consensus.cc:2267] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.756449 16189 leader_election.cc:290] T 6ed47b26cdf744acb28fb657a1b41087 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.756430 14812 raft_consensus.cc:357] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.756518 14812 raft_consensus.cc:383] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.756548 14812 raft_consensus.cc:738] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.756652 14812 consensus_queue.cc:260] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.756856 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ed47b26cdf744acb28fb657a1b41087" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.756917 14812 ts_tablet_manager.cc:1428] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.757105 14812 tablet_bootstrap.cc:492] T 5db9d5f6cecb454a90ad931d8d165bcc P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:21.757218 14493 leader_election.cc:343] T 6ed47b26cdf744acb28fb657a1b41087 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.757972 14812 tablet_bootstrap.cc:654] T 5db9d5f6cecb454a90ad931d8d165bcc P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.758081 14162 master_service.cc:946] ReplaceTablet: received request to replace tablet ec0f3424ebcb442597a94ccdbb25eeb6 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:21.757105 14823 ts_tablet_manager.cc:1905] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.758769 14169 catalog_manager.cc:5526] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:21.758996 14169 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 8efa998709034fe9bb37c7c608c99e76 (table default.Table_b6002fb5cc1248bba2eb7abbd5a5d407 [id=4c458a80b1a2490da1cf689843cae584]). Expected version 1 got 0
I20250115 03:06:21.759093 14169 catalog_manager.cc:5526] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: HEALTHY } } }
I20250115 03:06:21.759682 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ac903c81a1f481ab587e6772c306a96 with delete_type TABLET_DATA_DELETED (replaced by tablet 607fc76ce73744c4b8ac5975515e1fa3) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.759812 14533 tablet_service.cc:1514] Processing DeleteTablet for tablet 802a007b8d154198b291aaf9530161b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.759884 14526 tablet_service.cc:1514] Processing DeleteTablet for tablet 00aad81f4ca3427cb683de71148cff5c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.759907 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet b2629ae9d9854ae4b6e7ddddbb6f341b with delete_type TABLET_DATA_DELETED (replaced by tablet 103f78d0d9fc42308e1e23316895be33) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.759980 14518 tablet_service.cc:1514] Processing DeleteTablet for tablet 4122c97f5ba445ee8c79b7d056438229 with delete_type TABLET_DATA_DELETED (replaced by tablet fca14a7d574842ee94a93afb85ca1afd) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.760212 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1ce43146ed4e4d82a7c134216e2872 with delete_type TABLET_DATA_DELETED (replaced by tablet 578506b0528442cf834a8e65a400c635) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.760597 14823 ts_tablet_manager.cc:1918] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:21.760649 14823 log.cc:1198] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/b2629ae9d9854ae4b6e7ddddbb6f341b
I20250115 03:06:21.760859 14823 ts_tablet_manager.cc:1939] T b2629ae9d9854ae4b6e7ddddbb6f341b P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
W20250115 03:06:21.760854 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 7ac903c81a1f481ab587e6772c306a96 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7ac903c81a1f481ab587e6772c306a96 already in progress: creating tablet
I20250115 03:06:21.761186 14820 tablet_replica.cc:331] T b2629ae9d9854ae4b6e7ddddbb6f341b P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
W20250115 03:06:21.761248 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 802a007b8d154198b291aaf9530161b9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 802a007b8d154198b291aaf9530161b9 already in progress: creating tablet
I20250115 03:06:21.761301 14820 raft_consensus.cc:2238] T b2629ae9d9854ae4b6e7ddddbb6f341b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
W20250115 03:06:21.761376 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet 00aad81f4ca3427cb683de71148cff5c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 00aad81f4ca3427cb683de71148cff5c already in progress: creating tablet
I20250115 03:06:21.761385 14820 raft_consensus.cc:2267] T b2629ae9d9854ae4b6e7ddddbb6f341b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.761824 14820 ts_tablet_manager.cc:1905] T b2629ae9d9854ae4b6e7ddddbb6f341b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:21.762244 14493 leader_election.cc:343] T 6ed47b26cdf744acb28fb657a1b41087 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 6ed47b26cdf744acb28fb657a1b41087
I20250115 03:06:21.762303 14493 leader_election.cc:304] T 6ed47b26cdf744acb28fb657a1b41087 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:21.762555 14853 raft_consensus.cc:2743] T 6ed47b26cdf744acb28fb657a1b41087 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.763692 14820 ts_tablet_manager.cc:1918] T b2629ae9d9854ae4b6e7ddddbb6f341b P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.763744 14820 log.cc:1198] T b2629ae9d9854ae4b6e7ddddbb6f341b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/b2629ae9d9854ae4b6e7ddddbb6f341b
I20250115 03:06:21.763868 14820 ts_tablet_manager.cc:1939] T b2629ae9d9854ae4b6e7ddddbb6f341b P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:21.764027 16189 raft_consensus.cc:491] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.764093 16189 raft_consensus.cc:513] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.764374 16189 leader_election.cc:290] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.764644 14823 tablet_replica.cc:331] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:21.764704 14809 tablet_bootstrap.cc:492] T a3eaec2fff184aad816349fc36fa3db1 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.764732 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet b2629ae9d9854ae4b6e7ddddbb6f341b (table default.Table_ccd2815995fc485d827b79ae818fedae [id=522c89cadb084de68368a97199ea66db]) successfully deleted
I20250115 03:06:21.764752 14823 raft_consensus.cc:2238] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.764777 14809 ts_tablet_manager.cc:1397] T a3eaec2fff184aad816349fc36fa3db1 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20250115 03:06:21.764801 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet ccf6b58c55d747da89c1c218a70ce09f
I20250115 03:06:21.764834 14823 raft_consensus.cc:2267] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:21.765323 14493 leader_election.cc:343] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 0e1ce43146ed4e4d82a7c134216e2872
I20250115 03:06:21.765314 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1ce43146ed4e4d82a7c134216e2872" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.765416 14689 raft_consensus.cc:2463] T 0e1ce43146ed4e4d82a7c134216e2872 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.765241 14809 raft_consensus.cc:357] T a3eaec2fff184aad816349fc36fa3db1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.765516 14809 raft_consensus.cc:383] T a3eaec2fff184aad816349fc36fa3db1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.765550 14809 raft_consensus.cc:738] T a3eaec2fff184aad816349fc36fa3db1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.765681 14809 consensus_queue.cc:260] T a3eaec2fff184aad816349fc36fa3db1 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.765777 14493 leader_election.cc:304] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:21.765970 16189 raft_consensus.cc:2798] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.766016 16189 raft_consensus.cc:491] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.766049 16189 raft_consensus.cc:3054] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.766861 16189 raft_consensus.cc:513] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.767150 16189 leader_election.cc:290] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.767357 14823 ts_tablet_manager.cc:1905] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.768507 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 1016cf85491d4ceda132ee57325b48a4
I20250115 03:06:21.769052 14809 ts_tablet_manager.cc:1428] T a3eaec2fff184aad816349fc36fa3db1 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:21.769135 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet ec0f3424ebcb442597a94ccdbb25eeb6
I20250115 03:06:21.769357 14162 catalog_manager.cc:6596] ReplaceTablet: tablet ec0f3424ebcb442597a94ccdbb25eeb6 deleted and replaced by tablet 0c5f5dfdf8234b93a878accd42c946f2
I20250115 03:06:21.770074 14812 tablet_bootstrap.cc:492] T 5db9d5f6cecb454a90ad931d8d165bcc P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.770148 14812 ts_tablet_manager.cc:1397] T 5db9d5f6cecb454a90ad931d8d165bcc P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20250115 03:06:21.770231 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 1016cf85491d4ceda132ee57325b48a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.770264 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet ccf6b58c55d747da89c1c218a70ce09f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.770347 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet ec0f3424ebcb442597a94ccdbb25eeb6 with delete_type TABLET_DATA_DELETED (replaced by tablet 0c5f5dfdf8234b93a878accd42c946f2) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.770457 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet ccf6b58c55d747da89c1c218a70ce09f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.770465 14518 tablet_service.cc:1514] Processing DeleteTablet for tablet 1016cf85491d4ceda132ee57325b48a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.770584 14812 raft_consensus.cc:357] T 5db9d5f6cecb454a90ad931d8d165bcc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.770473 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet ec0f3424ebcb442597a94ccdbb25eeb6 with delete_type TABLET_DATA_DELETED (replaced by tablet 0c5f5dfdf8234b93a878accd42c946f2) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.770664 14812 raft_consensus.cc:383] T 5db9d5f6cecb454a90ad931d8d165bcc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.770694 14812 raft_consensus.cc:738] T 5db9d5f6cecb454a90ad931d8d165bcc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.770745 14819 tablet_replica.cc:331] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.770798 14812 consensus_queue.cc:260] T 5db9d5f6cecb454a90ad931d8d165bcc P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.770880 14819 raft_consensus.cc:2238] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.770957 14819 raft_consensus.cc:2267] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.771032 14812 ts_tablet_manager.cc:1428] T 5db9d5f6cecb454a90ad931d8d165bcc P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.771185 14812 tablet_bootstrap.cc:492] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.771261 14819 ts_tablet_manager.cc:1905] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.772017 14820 tablet_replica.cc:331] T 4122c97f5ba445ee8c79b7d056438229 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:21.772046 14812 tablet_bootstrap.cc:654] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.772122 14823 ts_tablet_manager.cc:1918] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.772131 14820 raft_consensus.cc:2238] T 4122c97f5ba445ee8c79b7d056438229 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.772177 14823 log.cc:1198] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/d6cf1a0b9c81413b8f72de0cafc76963
I20250115 03:06:21.772212 14820 raft_consensus.cc:2267] T 4122c97f5ba445ee8c79b7d056438229 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.772354 14823 ts_tablet_manager.cc:1939] T d6cf1a0b9c81413b8f72de0cafc76963 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:21.772539 14655 tablet_service.cc:1514] Processing DeleteTablet for tablet ccf6b58c55d747da89c1c218a70ce09f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.772658 14656 tablet_service.cc:1514] Processing DeleteTablet for tablet 1016cf85491d4ceda132ee57325b48a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.772692 14820 ts_tablet_manager.cc:1905] T 4122c97f5ba445ee8c79b7d056438229 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.772747 14657 tablet_service.cc:1514] Processing DeleteTablet for tablet ec0f3424ebcb442597a94ccdbb25eeb6 with delete_type TABLET_DATA_DELETED (replaced by tablet 0c5f5dfdf8234b93a878accd42c946f2) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.772893 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet d6cf1a0b9c81413b8f72de0cafc76963 (table default.Table_e51b0b04bb044b36be347afe90845b14 [id=17ea0c263011486f82895eb86b21fe78]) successfully deleted
I20250115 03:06:21.773197 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet b2629ae9d9854ae4b6e7ddddbb6f341b (table default.Table_ccd2815995fc485d827b79ae818fedae [id=522c89cadb084de68368a97199ea66db]) successfully deleted
W20250115 03:06:21.773725 14493 leader_election.cc:343] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 0e1ce43146ed4e4d82a7c134216e2872
I20250115 03:06:21.773855 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1ce43146ed4e4d82a7c134216e2872" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:21.773943 14689 raft_consensus.cc:3054] T 0e1ce43146ed4e4d82a7c134216e2872 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.774721 14689 raft_consensus.cc:2463] T 0e1ce43146ed4e4d82a7c134216e2872 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.774936 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet e47012a789bd469f9691e5a4162ff512 (table default.Table_a838d05cbe294b9e8ec38d0c1a3f6221 [id=31ee40f75b634219ba0ae4b38e2c757e]). Expected version 1 got 0
I20250115 03:06:21.774984 14493 leader_election.cc:304] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:21.775046 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 238e68300f304266aaed94bc3b76a01d (table default.Table_941290b0882541e185286447fca362b9 [id=a2689bcea23843c9bb9e1f08e335e58f]). Expected version 2 got 0
I20250115 03:06:21.775171 16189 raft_consensus.cc:2798] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.775239 16189 raft_consensus.cc:695] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:21.775332 16189 consensus_queue.cc:237] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.776785 14819 ts_tablet_manager.cc:1918] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.776844 14819 log.cc:1198] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/1016cf85491d4ceda132ee57325b48a4
I20250115 03:06:21.776728 14162 catalog_manager.cc:5526] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:21.776916 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 5db9d5f6cecb454a90ad931d8d165bcc (table default.Table_ca82711024824acda4e6b4d3cf4f0537 [id=07019baad4a74fd0b468bbd9b60d54c8]). Expected version 2 got 0
I20250115 03:06:21.776997 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e170ced1612442ebcfdaff6d9826ea3 with delete_type TABLET_DATA_DELETED (replaced by tablet 8e4410ce33f0464eadead04e8116f773) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.777117 14162 catalog_manager.cc:5526] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:21.777263 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 8efa998709034fe9bb37c7c608c99e76 (table default.Table_b6002fb5cc1248bba2eb7abbd5a5d407 [id=4c458a80b1a2490da1cf689843cae584]). Expected version 1 got 0
I20250115 03:06:21.778132 14823 tablet_replica.cc:331] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:21.778241 14823 raft_consensus.cc:2238] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.778337 14823 raft_consensus.cc:2267] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.778657 14823 ts_tablet_manager.cc:1905] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.778894 14819 ts_tablet_manager.cc:1939] T 1016cf85491d4ceda132ee57325b48a4 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.779439 14819 tablet_replica.cc:331] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.779539 14819 raft_consensus.cc:2238] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:21.779688 14819 raft_consensus.cc:2267] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.780015 14819 ts_tablet_manager.cc:1905] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.781145 14823 ts_tablet_manager.cc:1918] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20250115 03:06:21.781206 14823 log.cc:1198] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/ccf6b58c55d747da89c1c218a70ce09f
I20250115 03:06:21.781420 14823 ts_tablet_manager.cc:1939] T ccf6b58c55d747da89c1c218a70ce09f P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:21.781981 14823 tablet_replica.cc:331] T 1016cf85491d4ceda132ee57325b48a4 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:21.781965 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet ccf6b58c55d747da89c1c218a70ce09f (table default.Table_9e42df4c569c47b698c9bbb6dcf39736 [id=e441d6f5acea4837accbfb8e4856f82d]) successfully deleted
I20250115 03:06:21.783550 14812 tablet_bootstrap.cc:492] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.789047 14812 ts_tablet_manager.cc:1397] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20250115 03:06:21.789081 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 1016cf85491d4ceda132ee57325b48a4 (table default.Table_2e115074bf7049bfbb1239b4d9c1ca2e [id=6cf622de00cb490bb794837223aa7bcb]) successfully deleted
I20250115 03:06:21.782183 14819 ts_tablet_manager.cc:1918] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20250115 03:06:21.789577 14819 log.cc:1198] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/ccf6b58c55d747da89c1c218a70ce09f
I20250115 03:06:21.789563 14812 raft_consensus.cc:357] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.789651 14812 raft_consensus.cc:383] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.789680 14812 raft_consensus.cc:738] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.789785 14812 consensus_queue.cc:260] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.789830 14819 ts_tablet_manager.cc:1939] T ccf6b58c55d747da89c1c218a70ce09f P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.790230 14812 ts_tablet_manager.cc:1428] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.790386 14819 tablet_replica.cc:331] T 3e170ced1612442ebcfdaff6d9826ea3 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.790483 14819 raft_consensus.cc:2238] T 3e170ced1612442ebcfdaff6d9826ea3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.790555 14819 raft_consensus.cc:2267] T 3e170ced1612442ebcfdaff6d9826ea3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.790930 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet a3eaec2fff184aad816349fc36fa3db1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.782327 14809 tablet_bootstrap.cc:492] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.791905 14809 tablet_bootstrap.cc:654] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.792577 14812 tablet_bootstrap.cc:492] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.792594 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 6132d15b17ab4d0aaa10a7ea04cda7ac with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.792696 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet de14b1d525cc42808500279ae6b60e80 with delete_type TABLET_DATA_DELETED (replaced by tablet daea48ecc69d441ab66fcb40ea2e7a66) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.783911 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet d891020ed0bb4fc49155bd9f2f6ed35c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.785296 14820 ts_tablet_manager.cc:1918] T 4122c97f5ba445ee8c79b7d056438229 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.786516 14518 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ed47b26cdf744acb28fb657a1b41087 with delete_type TABLET_DATA_DELETED (replaced by tablet fe8d94c8fecc4077ba052390810c7430) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.787497 14811 tablet_bootstrap.cc:492] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.793644 14811 ts_tablet_manager.cc:1397] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.000s
I20250115 03:06:21.793906 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet ccf6b58c55d747da89c1c218a70ce09f (table default.Table_9e42df4c569c47b698c9bbb6dcf39736 [id=e441d6f5acea4837accbfb8e4856f82d]) successfully deleted
W20250115 03:06:21.794049 14158 catalog_manager.cc:4852] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): delete failed for tablet ec0f3424ebcb442597a94ccdbb25eeb6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ec0f3424ebcb442597a94ccdbb25eeb6 already in progress: creating tablet
I20250115 03:06:21.794111 14811 raft_consensus.cc:357] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.794210 14811 raft_consensus.cc:383] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.794242 14811 raft_consensus.cc:738] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.794337 14811 consensus_queue.cc:260] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.794620 14811 ts_tablet_manager.cc:1428] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.794801 14811 tablet_bootstrap.cc:492] T c2afa048b3fd4362b50c8a6da306c25a P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.794796 14819 ts_tablet_manager.cc:1905] T 3e170ced1612442ebcfdaff6d9826ea3 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.795667 14811 tablet_bootstrap.cc:654] T c2afa048b3fd4362b50c8a6da306c25a P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.793581 14820 log.cc:1198] T 4122c97f5ba445ee8c79b7d056438229 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/4122c97f5ba445ee8c79b7d056438229
I20250115 03:06:21.798003 14820 ts_tablet_manager.cc:1939] T 4122c97f5ba445ee8c79b7d056438229 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:21.783699 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_39c6ad12156a45948a2b77375993296c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:21.798171 14657 tablet_service.cc:1514] Processing DeleteTablet for tablet b2629ae9d9854ae4b6e7ddddbb6f341b with delete_type TABLET_DATA_DELETED (replaced by tablet 103f78d0d9fc42308e1e23316895be33) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.798298 14656 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1ce43146ed4e4d82a7c134216e2872 with delete_type TABLET_DATA_DELETED (replaced by tablet 578506b0528442cf834a8e65a400c635) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.798921 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 4122c97f5ba445ee8c79b7d056438229 (table default.Table_b169e386c2cc424da59a39503cec548f [id=2b97c2f5b5284cd8a6311247712cc447]) successfully deleted
I20250115 03:06:21.801182 14170 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 6bd4e4200f9b489ea5dd2154aa6f382d (table default.Table_83b1fd83225a4454b29b7c61bc4639d4 [id=299ac71cfb98480f873eb659419e522c]). Expected version 1 got 0
I20250115 03:06:21.801247 16001 consensus_queue.cc:1035] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:21.802587 14820 tablet_replica.cc:331] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:21.803186 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 72abf6acd9d54f3b983b7098aaf2691d (table default.Table_5e72c1c96c004205bf7304be210f76f1 [id=5ca0a4e7179649d3870fdadc05eaa427]). Expected version 1 got 0
I20250115 03:06:21.803418 14820 raft_consensus.cc:2238] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:21.803613 14820 pending_rounds.cc:62] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1: Trying to abort 1 pending ops.
I20250115 03:06:21.803691 14820 pending_rounds.cc:69] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384923752943616 op_type: NO_OP noop_request { }
I20250115 03:06:21.803818 14820 raft_consensus.cc:2883] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:21.804080 14820 raft_consensus.cc:2267] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.804507 14162 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 5db9d5f6cecb454a90ad931d8d165bcc (table default.Table_ca82711024824acda4e6b4d3cf4f0537 [id=07019baad4a74fd0b468bbd9b60d54c8]). Expected version 2 got 0
I20250115 03:06:21.804834 14656 tablet_service.cc:1467] Processing CreateTablet for tablet 224fbd2c521f4de99fb53d961736a5b5 (DEFAULT_TABLE table=default.Table_b97baa78114e4eab889110d44a2da354 [id=31384247aec54caca7efb52f0aeb1c70]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.805097 14656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 224fbd2c521f4de99fb53d961736a5b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.805743 14167 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 0f20bb66f36647e0811d0865b8f414f7 (table default.Table_0326e94494d0447585233c0c907113fd [id=f48964597a90470299a755217c4dbea0]). Expected version 1 got 0
I20250115 03:06:21.804252 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2868f144930b418199ddf15146d94c13" } new_table_name: "default.Table_0fb3ec72aa0d4826b4e7da55d76d3f55" modify_external_catalogs: true
I20250115 03:06:21.806272 14518 tablet_service.cc:1467] Processing CreateTablet for tablet 224fbd2c521f4de99fb53d961736a5b5 (DEFAULT_TABLE table=default.Table_b97baa78114e4eab889110d44a2da354 [id=31384247aec54caca7efb52f0aeb1c70]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.806490 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 224fbd2c521f4de99fb53d961736a5b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.804399 14170 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_cb51431eaddb48858a61d248b688bbe7" } new_table_name: "default.Table_881a5d0cb8b34d9aa05abd6ea71149df" modify_external_catalogs: true
W20250115 03:06:21.803905 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_39c6ad12156a45948a2b77375993296c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.808646 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 360c9811a4a845cc914e944067c536da (DEFAULT_TABLE table=default.Table_f8d2c23a3c9d44a7b8959e53bd207f30 [id=33314d6a8b624fc8a7aacd4e14180920]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.808868 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 360c9811a4a845cc914e944067c536da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.809664 14811 tablet_bootstrap.cc:492] T c2afa048b3fd4362b50c8a6da306c25a P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.809743 14811 ts_tablet_manager.cc:1397] T c2afa048b3fd4362b50c8a6da306c25a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20250115 03:06:21.810159 14811 raft_consensus.cc:357] T c2afa048b3fd4362b50c8a6da306c25a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.810256 14811 raft_consensus.cc:383] T c2afa048b3fd4362b50c8a6da306c25a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.810290 14811 raft_consensus.cc:738] T c2afa048b3fd4362b50c8a6da306c25a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.810391 14811 consensus_queue.cc:260] T c2afa048b3fd4362b50c8a6da306c25a P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.810667 14811 ts_tablet_manager.cc:1428] T c2afa048b3fd4362b50c8a6da306c25a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.810851 14811 tablet_bootstrap.cc:492] T a8e41e31ffae4bb092cb08d83eda16b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.812058 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 224fbd2c521f4de99fb53d961736a5b5 (DEFAULT_TABLE table=default.Table_b97baa78114e4eab889110d44a2da354 [id=31384247aec54caca7efb52f0aeb1c70]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.812317 14811 tablet_bootstrap.cc:654] T a8e41e31ffae4bb092cb08d83eda16b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.812382 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 360c9811a4a845cc914e944067c536da (DEFAULT_TABLE table=default.Table_f8d2c23a3c9d44a7b8959e53bd207f30 [id=33314d6a8b624fc8a7aacd4e14180920]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.812840 14656 tablet_service.cc:1467] Processing CreateTablet for tablet 360c9811a4a845cc914e944067c536da (DEFAULT_TABLE table=default.Table_f8d2c23a3c9d44a7b8959e53bd207f30 [id=33314d6a8b624fc8a7aacd4e14180920]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.813082 14656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 360c9811a4a845cc914e944067c536da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.813388 14820 ts_tablet_manager.cc:1905] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.814716 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 360c9811a4a845cc914e944067c536da. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.814738 14819 ts_tablet_manager.cc:1918] T 3e170ced1612442ebcfdaff6d9826ea3 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:21.814803 14819 log.cc:1198] T 3e170ced1612442ebcfdaff6d9826ea3 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/3e170ced1612442ebcfdaff6d9826ea3
I20250115 03:06:21.815025 14819 ts_tablet_manager.cc:1939] T 3e170ced1612442ebcfdaff6d9826ea3 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.815290 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 224fbd2c521f4de99fb53d961736a5b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.815382 14809 tablet_bootstrap.cc:492] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.815469 14809 ts_tablet_manager.cc:1397] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
I20250115 03:06:21.815935 14809 raft_consensus.cc:357] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.816047 14809 raft_consensus.cc:383] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.816087 14809 raft_consensus.cc:738] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.816200 14809 consensus_queue.cc:260] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.816478 14809 ts_tablet_manager.cc:1428] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.816638 14809 tablet_bootstrap.cc:492] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.817525 14809 tablet_bootstrap.cc:654] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.819326 16406 raft_consensus.cc:491] T 83a4f95eb9494a5295a81a8a51f58bd3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.819415 14420 raft_consensus.cc:1270] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:21.819399 16406 raft_consensus.cc:513] T 83a4f95eb9494a5295a81a8a51f58bd3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.819713 16406 leader_election.cc:290] T 83a4f95eb9494a5295a81a8a51f58bd3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:21.819801 14170 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5c5e30d9b2ce4537b00b216ef1f2dd1c consensus state has no leader
I20250115 03:06:21.819911 14853 consensus_queue.cc:1035] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:21.820153 14853 raft_consensus.cc:491] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.820220 14853 raft_consensus.cc:513] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.820490 14853 leader_election.cc:290] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.821194 14418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b08e5fc2110e476e897240f9cf3b874d" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.821295 14418 raft_consensus.cc:2463] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.821457 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83a4f95eb9494a5295a81a8a51f58bd3" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:21.822507 14356 leader_election.cc:343] T 83a4f95eb9494a5295a81a8a51f58bd3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.822770 14518 tablet_service.cc:1467] Processing CreateTablet for tablet 26af38b28c7b4e828ba7866c06103d19 (DEFAULT_TABLE table=default.Table_f88c5bc446c6498393a81148a97bb27f [id=28732377a6524cc995f1aefb88c3dbf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.822927 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet c2afa048b3fd4362b50c8a6da306c25a with delete_type TABLET_DATA_DELETED (replaced by tablet 6501c82435154d6e8f2c12b7c82da296) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.823081 14493 leader_election.cc:304] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:21.823288 16189 raft_consensus.cc:2798] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.823334 16189 raft_consensus.cc:491] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.823369 16189 raft_consensus.cc:3054] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.825177 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 1fd24fcb53864222868fe4ac7d2160dd (DEFAULT_TABLE table=default.Table_256bd9b91a3e426992d4b732f03d8181 [id=0a3e135e69c848ad8063c0044098e480]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.826023 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 26af38b28c7b4e828ba7866c06103d19 (DEFAULT_TABLE table=default.Table_f88c5bc446c6498393a81148a97bb27f [id=28732377a6524cc995f1aefb88c3dbf9]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:21.826285 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ffc493e8444147018bdf50382dc95b63 consensus state has no leader
I20250115 03:06:21.826706 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_6887c1062a6f4180a342c6353806be41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:21.827423 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 3e170ced1612442ebcfdaff6d9826ea3 (table default.Table_77d4ac27a2c44250a2dcc62121f202c7 [id=6af8f6f562ac47b484eeb4d67ac6c7df]) successfully deleted
I20250115 03:06:21.827363 14819 tablet_replica.cc:331] T a3eaec2fff184aad816349fc36fa3db1 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.827755 14819 raft_consensus.cc:2238] T a3eaec2fff184aad816349fc36fa3db1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.827767 14809 tablet_bootstrap.cc:492] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.827883 14819 raft_consensus.cc:2267] T a3eaec2fff184aad816349fc36fa3db1 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.827899 14809 ts_tablet_manager.cc:1397] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20250115 03:06:21.827960 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 1fd24fcb53864222868fe4ac7d2160dd (DEFAULT_TABLE table=default.Table_256bd9b91a3e426992d4b732f03d8181 [id=0a3e135e69c848ad8063c0044098e480]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.828158 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fd24fcb53864222868fe4ac7d2160dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.828316 14809 raft_consensus.cc:357] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.828400 14809 raft_consensus.cc:383] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.828429 14809 raft_consensus.cc:738] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.828531 14809 consensus_queue.cc:260] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.828562 14397 tablet_service.cc:1467] Processing CreateTablet for tablet 430a19d510994e999cb736c8455ea494 (DEFAULT_TABLE table=default.Table_582d4ea061bb4e07a300f5f5974ac57a [id=27fec636627145a3979b336a9b11c379]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.828760 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 430a19d510994e999cb736c8455ea494. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.829062 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 430a19d510994e999cb736c8455ea494 (DEFAULT_TABLE table=default.Table_582d4ea061bb4e07a300f5f5974ac57a [id=27fec636627145a3979b336a9b11c379]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.829488 14059 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu
/tmp/dist-test-taskE2XgYg/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.186.254:39901
--webserver_interface=127.13.186.254
--webserver_port=37481
--builtin_ntp_servers=127.13.186.212:37407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.186.254:39901,127.13.186.253:35411,127.13.186.252:41855
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20250115 03:06:21.828774 14809 ts_tablet_manager.cc:1428] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.829818 14809 tablet_bootstrap.cc:492] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.829869 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26af38b28c7b4e828ba7866c06103d19. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.830133 14819 ts_tablet_manager.cc:1905] T a3eaec2fff184aad816349fc36fa3db1 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.831897 14526 tablet_service.cc:1467] Processing CreateTablet for tablet fe448a09acdc4f418b759ab1ad40b626 (DEFAULT_TABLE table=default.Table_3e458d4484e34d18adf6fd4268b0e647 [id=f9e299a1eaef446b80f7ef9bf30adb8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.832180 14809 tablet_bootstrap.cc:654] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.833600 14533 tablet_service.cc:1467] Processing CreateTablet for tablet c9ed00b2835b46f78b3393fa40b13bcf (DEFAULT_TABLE table=default.Table_2ffd8bc4e3924dc6a952ee6b31e2b305 [id=5143bc1ce41743a0a1831ed98c4b87bd]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:21.833765 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6887c1062a6f4180a342c6353806be41 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.834394 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26af38b28c7b4e828ba7866c06103d19. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.834538 16189 raft_consensus.cc:513] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.834694 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fd24fcb53864222868fe4ac7d2160dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.834795 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9ed00b2835b46f78b3393fa40b13bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.834892 16189 leader_election.cc:290] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.834920 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 430a19d510994e999cb736c8455ea494. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.835033 14526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe448a09acdc4f418b759ab1ad40b626. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.838151 14820 ts_tablet_manager.cc:1918] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:21.838219 14820 log.cc:1198] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/0e1ce43146ed4e4d82a7c134216e2872
I20250115 03:06:21.838445 14820 ts_tablet_manager.cc:1939] T 0e1ce43146ed4e4d82a7c134216e2872 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:21.839107 14397 tablet_service.cc:1467] Processing CreateTablet for tablet e9725e15494e4a988db499414b94f84a (DEFAULT_TABLE table=default.Table_c94a10c892a343a4bf3917501b740480 [id=da4dae59def44b45836e087d42312315]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.839288 14418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b08e5fc2110e476e897240f9cf3b874d" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.839393 14418 raft_consensus.cc:3054] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.840245 14858 raft_consensus.cc:491] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.840327 14858 raft_consensus.cc:513] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.840490 14537 tablet_service.cc:1467] Processing CreateTablet for tablet e9725e15494e4a988db499414b94f84a (DEFAULT_TABLE table=default.Table_c94a10c892a343a4bf3917501b740480 [id=da4dae59def44b45836e087d42312315]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.840648 14858 leader_election.cc:290] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.840675 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9725e15494e4a988db499414b94f84a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.841105 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda418ed197e4af2aa511d3f1851663e" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.841204 14419 raft_consensus.cc:2463] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.841547 14493 leader_election.cc:304] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:21.841635 14396 tablet_service.cc:1467] Processing CreateTablet for tablet fe448a09acdc4f418b759ab1ad40b626 (DEFAULT_TABLE table=default.Table_3e458d4484e34d18adf6fd4268b0e647 [id=f9e299a1eaef446b80f7ef9bf30adb8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.841782 14858 raft_consensus.cc:2798] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.841832 14858 raft_consensus.cc:491] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.841866 14858 raft_consensus.cc:3054] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.842548 14391 tablet_service.cc:1467] Processing CreateTablet for tablet d1daa457581a48ceb18bf617a60a0802 (DEFAULT_TABLE table=default.Table_70287f2d5a024cc88244993789eb747b [id=88146f846aa74ecfb3977f01ea45df2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.842907 14518 tablet_service.cc:1467] Processing CreateTablet for tablet d1daa457581a48ceb18bf617a60a0802 (DEFAULT_TABLE table=default.Table_70287f2d5a024cc88244993789eb747b [id=88146f846aa74ecfb3977f01ea45df2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.843111 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1daa457581a48ceb18bf617a60a0802. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.839303 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9725e15494e4a988db499414b94f84a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.844064 16406 raft_consensus.cc:491] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.844134 16406 raft_consensus.cc:513] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.844430 16406 leader_election.cc:290] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.844941 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b5fd91a0ea64b3b85b7571e5daa08b2" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:21.845280 14356 leader_election.cc:343] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.846761 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_47430d0013a7450aa28c3ef58ff9ca75"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.847280 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_47430d0013a7450aa28c3ef58ff9ca75 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.847453 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_64d33ab8807d44a996a1f0fcc2deb7e2" } new_table_name: "default.Table_6bde8c0c2f004edda4a091f8a81f98a4" modify_external_catalogs: true
I20250115 03:06:21.847654 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_12e083a49c774f2eab3c48fb2dd2cd0f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.847918 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_12e083a49c774f2eab3c48fb2dd2cd0f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.849345 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5577048ebf95456d9cbe74d11b84f131" } new_table_name: "default.Table_5a9c75c8867b4381bb170cecd8dd6acf" modify_external_catalogs: true
I20250115 03:06:21.849576 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f53f2ed27f714b12ab52472c670b506e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:21.849857 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f53f2ed27f714b12ab52472c670b506e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.849997 14398 tablet_service.cc:1467] Processing CreateTablet for tablet c9ed00b2835b46f78b3393fa40b13bcf (DEFAULT_TABLE table=default.Table_2ffd8bc4e3924dc6a952ee6b31e2b305 [id=5143bc1ce41743a0a1831ed98c4b87bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.816255 14812 tablet_bootstrap.cc:654] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.820313 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83a4f95eb9494a5295a81a8a51f58bd3" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.820784 14688 raft_consensus.cc:1270] T f2ca06c402cd46dc8311c8a47da92846 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:21.821907 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b08e5fc2110e476e897240f9cf3b874d" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.850921 14687 raft_consensus.cc:2463] T b08e5fc2110e476e897240f9cf3b874d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
W20250115 03:06:21.850946 14358 leader_election.cc:343] T 83a4f95eb9494a5295a81a8a51f58bd3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.822865 14657 tablet_service.cc:1467] Processing CreateTablet for tablet 1fd24fcb53864222868fe4ac7d2160dd (DEFAULT_TABLE table=default.Table_256bd9b91a3e426992d4b732f03d8181 [id=0a3e135e69c848ad8063c0044098e480]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.851008 14358 leader_election.cc:304] T 83a4f95eb9494a5295a81a8a51f58bd3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.851154 14657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fd24fcb53864222868fe4ac7d2160dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.851188 16406 raft_consensus.cc:2743] T 83a4f95eb9494a5295a81a8a51f58bd3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.827513 14655 tablet_service.cc:1467] Processing CreateTablet for tablet 430a19d510994e999cb736c8455ea494 (DEFAULT_TABLE table=default.Table_582d4ea061bb4e07a300f5f5974ac57a [id=27fec636627145a3979b336a9b11c379]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.853310 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 430a19d510994e999cb736c8455ea494. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.831624 14668 tablet_service.cc:1467] Processing CreateTablet for tablet fe448a09acdc4f418b759ab1ad40b626 (DEFAULT_TABLE table=default.Table_3e458d4484e34d18adf6fd4268b0e647 [id=f9e299a1eaef446b80f7ef9bf30adb8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.855629 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe448a09acdc4f418b759ab1ad40b626. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.832764 14671 tablet_service.cc:1467] Processing CreateTablet for tablet c9ed00b2835b46f78b3393fa40b13bcf (DEFAULT_TABLE table=default.Table_2ffd8bc4e3924dc6a952ee6b31e2b305 [id=5143bc1ce41743a0a1831ed98c4b87bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.857475 14809 tablet_bootstrap.cc:492] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.857550 14809 ts_tablet_manager.cc:1397] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.000s
I20250115 03:06:21.857627 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9ed00b2835b46f78b3393fa40b13bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.858001 14809 raft_consensus.cc:357] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.858088 14809 raft_consensus.cc:383] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.858114 14809 raft_consensus.cc:738] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.858206 14809 consensus_queue.cc:260] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
W20250115 03:06:21.858393 14167 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a198e74fd7db4247a673ccb834e7b150 consensus state has no leader
I20250115 03:06:21.858479 14809 ts_tablet_manager.cc:1428] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.858656 14809 tablet_bootstrap.cc:492] T fe8d94c8fecc4077ba052390810c7430 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.858929 14171 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet f2ca06c402cd46dc8311c8a47da92846 (table default.Table_802f667f1ad246688cbdcffcb3001021 [id=e929ffaba567498c815393836b264a12]). Expected version 1 got 0
I20250115 03:06:21.838738 14653 tablet_service.cc:1467] Processing CreateTablet for tablet e9725e15494e4a988db499414b94f84a (DEFAULT_TABLE table=default.Table_c94a10c892a343a4bf3917501b740480 [id=da4dae59def44b45836e087d42312315]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.859490 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9725e15494e4a988db499414b94f84a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.859544 14809 tablet_bootstrap.cc:654] T fe8d94c8fecc4077ba052390810c7430 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.860468 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 78cc96de8b414ac19285338ccb1024e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.839126 16088 consensus_queue.cc:1035] T 0f20bb66f36647e0811d0865b8f414f7 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:21.862614 14811 tablet_bootstrap.cc:492] T a8e41e31ffae4bb092cb08d83eda16b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.862690 14811 ts_tablet_manager.cc:1397] T a8e41e31ffae4bb092cb08d83eda16b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.052s user 0.000s sys 0.002s
I20250115 03:06:21.863142 14811 raft_consensus.cc:357] T a8e41e31ffae4bb092cb08d83eda16b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.863246 14811 raft_consensus.cc:383] T a8e41e31ffae4bb092cb08d83eda16b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.863278 14811 raft_consensus.cc:738] T a8e41e31ffae4bb092cb08d83eda16b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.863394 14811 consensus_queue.cc:260] T a8e41e31ffae4bb092cb08d83eda16b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.863644 14811 ts_tablet_manager.cc:1428] T a8e41e31ffae4bb092cb08d83eda16b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:21.863833 14811 tablet_bootstrap.cc:492] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.842657 15153 raft_consensus.cc:491] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.864024 15153 raft_consensus.cc:513] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.864352 15153 leader_election.cc:290] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:21.864717 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad0e9216bd154f4a8a37874d3d8a08c2" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.864774 14811 tablet_bootstrap.cc:654] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.864818 14552 raft_consensus.cc:2463] T ad0e9216bd154f4a8a37874d3d8a08c2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:21.865096 14626 leader_election.cc:304] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.843089 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda418ed197e4af2aa511d3f1851663e" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.865314 14690 raft_consensus.cc:2463] T dda418ed197e4af2aa511d3f1851663e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.843291 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b08e5fc2110e476e897240f9cf3b874d" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:21.866079 14692 raft_consensus.cc:3054] T b08e5fc2110e476e897240f9cf3b874d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.866683 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad0e9216bd154f4a8a37874d3d8a08c2" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.866788 14419 raft_consensus.cc:2463] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
W20250115 03:06:21.867023 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a8e41e31ffae4bb092cb08d83eda16b5 consensus state has no leader
I20250115 03:06:21.845543 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b5fd91a0ea64b3b85b7571e5daa08b2" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.846840 14659 tablet_service.cc:1467] Processing CreateTablet for tablet d1daa457581a48ceb18bf617a60a0802 (DEFAULT_TABLE table=default.Table_70287f2d5a024cc88244993789eb747b [id=88146f846aa74ecfb3977f01ea45df2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.867563 14809 tablet_bootstrap.cc:492] T fe8d94c8fecc4077ba052390810c7430 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.867575 14659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1daa457581a48ceb18bf617a60a0802. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:21.867609 14358 leader_election.cc:343] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.867637 14809 ts_tablet_manager.cc:1397] T fe8d94c8fecc4077ba052390810c7430 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:21.867663 14358 leader_election.cc:304] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.867861 16406 raft_consensus.cc:2743] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.868024 14809 raft_consensus.cc:357] T fe8d94c8fecc4077ba052390810c7430 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.868108 14809 raft_consensus.cc:383] T fe8d94c8fecc4077ba052390810c7430 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.868136 14809 raft_consensus.cc:738] T fe8d94c8fecc4077ba052390810c7430 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.868186 15153 raft_consensus.cc:2798] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.868228 15153 raft_consensus.cc:491] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.868264 15153 raft_consensus.cc:3054] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.868225 14809 consensus_queue.cc:260] T fe8d94c8fecc4077ba052390810c7430 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.868487 14809 ts_tablet_manager.cc:1428] T fe8d94c8fecc4077ba052390810c7430 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.868638 14809 tablet_bootstrap.cc:492] T d5e4743e016e49ed8ba26b1ac96b1470 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.869099 16189 consensus_queue.cc:1035] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:21.869617 14809 tablet_bootstrap.cc:654] T d5e4743e016e49ed8ba26b1ac96b1470 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.869824 16411 raft_consensus.cc:491] T 194d943d173340e7a8fba69967d9b090 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.869890 16411 raft_consensus.cc:513] T 194d943d173340e7a8fba69967d9b090 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.870147 16189 raft_consensus.cc:491] T c2afa048b3fd4362b50c8a6da306c25a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.870237 16411 leader_election.cc:290] T 194d943d173340e7a8fba69967d9b090 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.870229 16189 raft_consensus.cc:513] T c2afa048b3fd4362b50c8a6da306c25a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.870551 16189 leader_election.cc:290] T c2afa048b3fd4362b50c8a6da306c25a P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.870609 14811 tablet_bootstrap.cc:492] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.870682 14811 ts_tablet_manager.cc:1397] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20250115 03:06:21.870697 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "194d943d173340e7a8fba69967d9b090" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.870990 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "194d943d173340e7a8fba69967d9b090" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.871091 14811 raft_consensus.cc:357] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.871167 14811 raft_consensus.cc:383] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.871196 14811 raft_consensus.cc:738] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.871313 14811 consensus_queue.cc:260] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:21.871531 14358 leader_election.cc:343] T 194d943d173340e7a8fba69967d9b090 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.871574 14811 ts_tablet_manager.cc:1428] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:21.871717 14811 tablet_bootstrap.cc:492] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
W20250115 03:06:21.872136 14356 leader_election.cc:343] T 194d943d173340e7a8fba69967d9b090 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.872206 14356 leader_election.cc:304] T 194d943d173340e7a8fba69967d9b090 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.872468 16411 raft_consensus.cc:2743] T 194d943d173340e7a8fba69967d9b090 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.872685 14811 tablet_bootstrap.cc:654] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.873847 14811 tablet_bootstrap.cc:492] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.874661 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2afa048b3fd4362b50c8a6da306c25a" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.875092 14811 ts_tablet_manager.cc:1397] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
W20250115 03:06:21.874107 14493 leader_election.cc:343] T c2afa048b3fd4362b50c8a6da306c25a P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: c2afa048b3fd4362b50c8a6da306c25a
I20250115 03:06:21.876506 14811 raft_consensus.cc:357] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.876667 14811 raft_consensus.cc:383] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.876747 14811 raft_consensus.cc:738] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.876906 14811 consensus_queue.cc:260] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.877409 14811 ts_tablet_manager.cc:1428] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
W20250115 03:06:21.878273 14493 leader_election.cc:343] T c2afa048b3fd4362b50c8a6da306c25a P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.878343 14493 leader_election.cc:304] T c2afa048b3fd4362b50c8a6da306c25a P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:21.878542 16189 raft_consensus.cc:2743] T c2afa048b3fd4362b50c8a6da306c25a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.878953 14811 tablet_bootstrap.cc:492] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.879714 16189 raft_consensus.cc:491] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.879789 16189 raft_consensus.cc:513] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.879845 14811 tablet_bootstrap.cc:654] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.880055 16189 leader_election.cc:290] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.880771 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bd4e4200f9b489ea5dd2154aa6f382d" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:21.881255 14493 leader_election.cc:343] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.881628 14671 tablet_service.cc:1467] Processing CreateTablet for tablet 3ef0024957054c08afd3d4b8887ae6a8 (DEFAULT_TABLE table=default.Table_6887c1062a6f4180a342c6353806be41 [id=c90882dd49694795beb5fce44469c566]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.881847 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ef0024957054c08afd3d4b8887ae6a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.881937 14809 tablet_bootstrap.cc:492] T d5e4743e016e49ed8ba26b1ac96b1470 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.881958 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bd4e4200f9b489ea5dd2154aa6f382d" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.882005 14809 ts_tablet_manager.cc:1397] T d5e4743e016e49ed8ba26b1ac96b1470 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20250115 03:06:21.882062 14421 raft_consensus.cc:2463] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.882400 14809 raft_consensus.cc:357] T d5e4743e016e49ed8ba26b1ac96b1470 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.882486 14809 raft_consensus.cc:383] T d5e4743e016e49ed8ba26b1ac96b1470 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.882514 14809 raft_consensus.cc:738] T d5e4743e016e49ed8ba26b1ac96b1470 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.882616 14809 consensus_queue.cc:260] T d5e4743e016e49ed8ba26b1ac96b1470 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.882895 14809 ts_tablet_manager.cc:1428] T d5e4743e016e49ed8ba26b1ac96b1470 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.882874 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 3ef0024957054c08afd3d4b8887ae6a8 (DEFAULT_TABLE table=default.Table_6887c1062a6f4180a342c6353806be41 [id=c90882dd49694795beb5fce44469c566]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.883054 14390 tablet_service.cc:1467] Processing CreateTablet for tablet 677820c9b1b448c480ae24821904ac7a (DEFAULT_TABLE table=default.Table_39c6ad12156a45948a2b77375993296c [id=ff002e7616ef46a6a890b8c627e5c332]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.883095 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 607fc76ce73744c4b8ac5975515e1fa3 (DEFAULT_TABLE table=default.Table_e69034f6c27e4ba8bccd3bfa8d47428e [id=bbe51385424040a1a7ef7720483939fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.883145 14395 tablet_service.cc:1467] Processing CreateTablet for tablet 0c5f5dfdf8234b93a878accd42c946f2 (DEFAULT_TABLE table=default.Table_c906e8e84e6a4543aa7ed60a65e7a042 [id=c80c26973339459e826e9ecc70343fe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.883054 14809 tablet_bootstrap.cc:492] T f2b8bd783b614bf9b5526de197c51df3 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.883752 14493 leader_election.cc:304] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770
I20250115 03:06:21.883952 16189 raft_consensus.cc:2798] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.883996 16189 raft_consensus.cc:491] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.884042 16189 raft_consensus.cc:3054] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.884001 14526 tablet_service.cc:1467] Processing CreateTablet for tablet 0c5f5dfdf8234b93a878accd42c946f2 (DEFAULT_TABLE table=default.Table_c906e8e84e6a4543aa7ed60a65e7a042 [id=c80c26973339459e826e9ecc70343fe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.884209 14809 tablet_bootstrap.cc:654] T f2b8bd783b614bf9b5526de197c51df3 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.884001 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 607fc76ce73744c4b8ac5975515e1fa3 (DEFAULT_TABLE table=default.Table_e69034f6c27e4ba8bccd3bfa8d47428e [id=bbe51385424040a1a7ef7720483939fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.884521 14820 tablet_replica.cc:331] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:21.884631 14820 raft_consensus.cc:2238] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.884701 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 607fc76ce73744c4b8ac5975515e1fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.884729 14820 raft_consensus.cc:2267] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.884819 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 3ef0024957054c08afd3d4b8887ae6a8 (DEFAULT_TABLE table=default.Table_6887c1062a6f4180a342c6353806be41 [id=c90882dd49694795beb5fce44469c566]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.885051 14820 ts_tablet_manager.cc:1905] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.885119 14530 tablet_service.cc:1467] Processing CreateTablet for tablet 677820c9b1b448c480ae24821904ac7a (DEFAULT_TABLE table=default.Table_39c6ad12156a45948a2b77375993296c [id=ff002e7616ef46a6a890b8c627e5c332]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.885352 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 0e1ce43146ed4e4d82a7c134216e2872 (table default.Table_3984fd35e6814fd4901d3894b30d7a91 [id=5775da4995774af39bfc049e48eaf2f0]) successfully deleted
I20250115 03:06:21.886907 14169 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d91eb70cd94b4ca389f3d7aa903da7e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:21.886991 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_14cc3f3be6bc492ca9ba0e1a879c8a0e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:21.886909 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_75817b46a6414888988ff7373ea11b27" } new_table_name: "default.Table_a0e9bf99191f41d6a55b5b32049090e8" modify_external_catalogs: true
I20250115 03:06:21.888161 14526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c5f5dfdf8234b93a878accd42c946f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.888429 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ef0024957054c08afd3d4b8887ae6a8. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:21.887253 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_14cc3f3be6bc492ca9ba0e1a879c8a0e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.890280 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 677820c9b1b448c480ae24821904ac7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.890364 14668 tablet_service.cc:1467] Processing CreateTablet for tablet 677820c9b1b448c480ae24821904ac7a (DEFAULT_TABLE table=default.Table_39c6ad12156a45948a2b77375993296c [id=ff002e7616ef46a6a890b8c627e5c332]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.890569 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 677820c9b1b448c480ae24821904ac7a. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:21.887253 14169 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d91eb70cd94b4ca389f3d7aa903da7e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:21.895422 14163 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9476eae94d664f5eb94865db2c95dca6 consensus state has no leader
I20250115 03:06:21.895567 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_13922b2e0cc243669f2b64039b083650"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:21.895977 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_6496be150f7f4fddbc5cd00cf9f03544"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:21.896369 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b3ed368ac6d8449d841b1f7522cd7815"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:21.850335 14656 tablet_service.cc:1467] Processing CreateTablet for tablet 26af38b28c7b4e828ba7866c06103d19 (DEFAULT_TABLE table=default.Table_f88c5bc446c6498393a81148a97bb27f [id=28732377a6524cc995f1aefb88c3dbf9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.897289 14656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26af38b28c7b4e828ba7866c06103d19. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:21.897332 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6496be150f7f4fddbc5cd00cf9f03544 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.898247 14811 tablet_bootstrap.cc:492] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.898324 14811 ts_tablet_manager.cc:1397] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.002s
I20250115 03:06:21.898509 14655 tablet_service.cc:1467] Processing CreateTablet for tablet 0c5f5dfdf8234b93a878accd42c946f2 (DEFAULT_TABLE table=default.Table_c906e8e84e6a4543aa7ed60a65e7a042 [id=c80c26973339459e826e9ecc70343fe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.898741 14811 raft_consensus.cc:357] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.898833 14811 raft_consensus.cc:383] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.898862 14811 raft_consensus.cc:738] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.898962 14811 consensus_queue.cc:260] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.899217 14811 ts_tablet_manager.cc:1428] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:21.899374 14811 tablet_bootstrap.cc:492] T b61cc4b11251486a8089ca9bf44ac91e P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.899434 14657 tablet_service.cc:1467] Processing CreateTablet for tablet 607fc76ce73744c4b8ac5975515e1fa3 (DEFAULT_TABLE table=default.Table_e69034f6c27e4ba8bccd3bfa8d47428e [id=bbe51385424040a1a7ef7720483939fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.900270 14811 tablet_bootstrap.cc:654] T b61cc4b11251486a8089ca9bf44ac91e P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.905225 14169 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 0f20bb66f36647e0811d0865b8f414f7 (table default.Table_0326e94494d0447585233c0c907113fd [id=f48964597a90470299a755217c4dbea0]). Expected version 1 got 0
I20250115 03:06:21.905339 14169 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet f2ca06c402cd46dc8311c8a47da92846 (table default.Table_802f667f1ad246688cbdcffcb3001021 [id=e929ffaba567498c815393836b264a12]). Expected version 1 got 0
W20250115 03:06:21.907357 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b3ed368ac6d8449d841b1f7522cd7815 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.907989 14391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1daa457581a48ceb18bf617a60a0802. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.908007 14819 ts_tablet_manager.cc:1918] T a3eaec2fff184aad816349fc36fa3db1 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20250115 03:06:21.908028 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_13922b2e0cc243669f2b64039b083650 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:21.908072 14819 log.cc:1198] T a3eaec2fff184aad816349fc36fa3db1 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/a3eaec2fff184aad816349fc36fa3db1
I20250115 03:06:21.908272 14819 ts_tablet_manager.cc:1939] T a3eaec2fff184aad816349fc36fa3db1 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:21.908417 14395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c5f5dfdf8234b93a878accd42c946f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.908705 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 677820c9b1b448c480ae24821904ac7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.908993 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ef0024957054c08afd3d4b8887ae6a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.909083 14692 raft_consensus.cc:2463] T b08e5fc2110e476e897240f9cf3b874d P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.909255 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 607fc76ce73744c4b8ac5975515e1fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.909570 14858 raft_consensus.cc:513] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.909919 14858 leader_election.cc:290] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.912101 16189 raft_consensus.cc:513] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.912422 16189 leader_election.cc:290] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.912658 14418 raft_consensus.cc:2463] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:21.919163 16001 raft_consensus.cc:491] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.919242 16001 raft_consensus.cc:513] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.919544 16001 leader_election.cc:290] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:21.920264 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b39608e19bf44b3a79a8616ccdad30a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.920372 14552 raft_consensus.cc:2463] T 4b39608e19bf44b3a79a8616ccdad30a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:21.920722 14418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b39608e19bf44b3a79a8616ccdad30a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.920827 14418 raft_consensus.cc:2463] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:21.921201 14628 leader_election.cc:304] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:21.921514 16292 raft_consensus.cc:2798] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.921558 16292 raft_consensus.cc:491] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.921589 16292 raft_consensus.cc:3054] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.922472 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe448a09acdc4f418b759ab1ad40b626. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.923660 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d91eb70cd94b4ca389f3d7aa903da7e9" } modify_external_catalogs: true
I20250115 03:06:21.923776 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d91eb70cd94b4ca389f3d7aa903da7e9" } modify_external_catalogs: true
I20250115 03:06:21.925381 14493 leader_election.cc:304] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:21.925494 14418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda418ed197e4af2aa511d3f1851663e" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.925602 14418 raft_consensus.cc:3054] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.925818 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet 341b24d9b63f4761b2013035023fcd98 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:21.926426 16189 raft_consensus.cc:2798] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:21.926506 16189 raft_consensus.cc:695] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:21.926622 16189 consensus_queue.cc:237] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.927424 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bd4e4200f9b489ea5dd2154aa6f382d" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:21.927510 14421 raft_consensus.cc:3054] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.928567 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9ed00b2835b46f78b3393fa40b13bcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.928773 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bd4e4200f9b489ea5dd2154aa6f382d" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:21.929150 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dda418ed197e4af2aa511d3f1851663e" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:21.929236 14692 raft_consensus.cc:3054] T dda418ed197e4af2aa511d3f1851663e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.930997 16411 raft_consensus.cc:491] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.931075 16411 raft_consensus.cc:513] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.931339 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c5f5dfdf8234b93a878accd42c946f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.931420 16411 leader_election.cc:290] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.931885 14819 tablet_replica.cc:331] T 6132d15b17ab4d0aaa10a7ea04cda7ac P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:21.931979 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "781f6a9148cd4bd684f24c294c0b019b" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.932013 14819 raft_consensus.cc:2238] T 6132d15b17ab4d0aaa10a7ea04cda7ac P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.932098 14819 raft_consensus.cc:2267] T 6132d15b17ab4d0aaa10a7ea04cda7ac P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.932255 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "781f6a9148cd4bd684f24c294c0b019b" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:21.932299 14356 leader_election.cc:343] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:21.932569 14358 leader_election.cc:343] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.932592 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet a3eaec2fff184aad816349fc36fa3db1 (table default.Table_f3b64ddeafb3400583229c6698c2a5e5 [id=538723971ef44b719ff8bf684fdc5676]) successfully deleted
I20250115 03:06:21.932633 14358 leader_election.cc:304] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.932758 14657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 607fc76ce73744c4b8ac5975515e1fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:21.932807 16411 raft_consensus.cc:2743] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:21.934146 14493 leader_election.cc:343] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.934391 14819 ts_tablet_manager.cc:1905] T 6132d15b17ab4d0aaa10a7ea04cda7ac P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.936159 14823 raft_consensus.cc:2238] T 1016cf85491d4ceda132ee57325b48a4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:21.936278 14823 raft_consensus.cc:2267] T 1016cf85491d4ceda132ee57325b48a4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:21.936599 14823 ts_tablet_manager.cc:1905] T 1016cf85491d4ceda132ee57325b48a4 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:21.939379 14162 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet 67889f5c311749269b6ebe500a24366c
I20250115 03:06:21.939692 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet d5e4743e016e49ed8ba26b1ac96b1470 (table default.Table_013cb9366ef94a75a90ef4dfed00ff41 [id=6ab343bee1254fb19a3024ebafa44464]). Expected version 2 got 0
I20250115 03:06:21.939790 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 0f20bb66f36647e0811d0865b8f414f7 (table default.Table_0326e94494d0447585233c0c907113fd [id=f48964597a90470299a755217c4dbea0]). Expected version 1 got 0
I20250115 03:06:21.940644 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet 78b5e2e16442407880b75d256e6b2614 with delete_type TABLET_DATA_DELETED (replaced by tablet ec1a8d0326f84ca0a87851010c1e68b7) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.940693 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet fe8d94c8fecc4077ba052390810c7430 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.942077 14659 tablet_service.cc:1467] Processing CreateTablet for tablet 67ff648cc30b4f07a944005f53494e8c (DEFAULT_TABLE table=default.Table_f53f2ed27f714b12ab52472c670b506e [id=5d505c588806496f979567857f9f56df]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.942341 16265 tablet.cc:1721] T f2ca06c402cd46dc8311c8a47da92846 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:21.943264 16454 tablet.cc:1721] T f2ca06c402cd46dc8311c8a47da92846 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:21.943295 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet a8e41e31ffae4bb092cb08d83eda16b5 (table default.Table_5a9c75c8867b4381bb170cecd8dd6acf [id=a530442772dc4b12b23992cee12fea85]). Expected version 1 got 0
I20250115 03:06:21.943395 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 0b5fd91a0ea64b3b85b7571e5daa08b2 (table default.Table_1f92c58a90374663a4c6368b14f71d43 [id=526370a40a0d4f7aa96c8348412d8b26]). Expected version 2 got 0
I20250115 03:06:21.943450 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 5a2894e4736b4191963a61d908bc673c (table default.Table_df79ed42f26f4b68b197877c445b9665 [id=bb6453f2ab97404cbd3ba32a87976e52]). Expected version 1 got 0
I20250115 03:06:21.943501 14162 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 0f20bb66f36647e0811d0865b8f414f7 (table default.Table_0326e94494d0447585233c0c907113fd [id=f48964597a90470299a755217c4dbea0]). Expected version 1 got 0
I20250115 03:06:21.945413 14812 tablet_bootstrap.cc:492] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:21.945504 14812 ts_tablet_manager.cc:1397] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.154s user 0.000s sys 0.002s
I20250115 03:06:21.945919 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 341b24d9b63f4761b2013035023fcd98
I20250115 03:06:21.945976 14812 raft_consensus.cc:357] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.946115 14163 catalog_manager.cc:6596] ReplaceTablet: tablet 341b24d9b63f4761b2013035023fcd98 deleted and replaced by tablet 306e4240c66647bc85d862252518f499
I20250115 03:06:21.945973 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 26fa8ff146834d32b97943e14f170e67 (DEFAULT_TABLE table=default.Table_47430d0013a7450aa28c3ef58ff9ca75 [id=1cb583178ccc40bfb51340dc54fc91f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.946079 14812 raft_consensus.cc:383] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.946220 14812 raft_consensus.cc:738] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.946331 14812 consensus_queue.cc:260] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.946678 14812 ts_tablet_manager.cc:1428] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:21.946794 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 341b24d9b63f4761b2013035023fcd98 with delete_type TABLET_DATA_DELETED (replaced by tablet 306e4240c66647bc85d862252518f499) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.946853 14812 tablet_bootstrap.cc:492] T 6ed47b26cdf744acb28fb657a1b41087 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:21.946898 14670 tablet_service.cc:1467] Processing CreateTablet for tablet f118a169019d4a32aa86b4fe9314f747 (DEFAULT_TABLE table=default.Table_12e083a49c774f2eab3c48fb2dd2cd0f [id=4e8899376d264906ab9f623b6ab235dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.947537 14163 master_service.cc:946] ReplaceTablet: received request to replace tablet 15fbf8d5de904fc4b284af664f3bd60f from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:21.947768 14812 tablet_bootstrap.cc:654] T 6ed47b26cdf744acb28fb657a1b41087 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.950091 15617 tablet.cc:1721] T f2ca06c402cd46dc8311c8a47da92846 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:21.950188 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 67ff648cc30b4f07a944005f53494e8c (DEFAULT_TABLE table=default.Table_f53f2ed27f714b12ab52472c670b506e [id=5d505c588806496f979567857f9f56df]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.950608 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 341b24d9b63f4761b2013035023fcd98 with delete_type TABLET_DATA_DELETED (replaced by tablet 306e4240c66647bc85d862252518f499) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.950620 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 26fa8ff146834d32b97943e14f170e67 (DEFAULT_TABLE table=default.Table_47430d0013a7450aa28c3ef58ff9ca75 [id=1cb583178ccc40bfb51340dc54fc91f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.950620 14392 tablet_service.cc:1467] Processing CreateTablet for tablet f118a169019d4a32aa86b4fe9314f747 (DEFAULT_TABLE table=default.Table_12e083a49c774f2eab3c48fb2dd2cd0f [id=4e8899376d264906ab9f623b6ab235dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.952772 16411 raft_consensus.cc:491] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.952840 16411 raft_consensus.cc:513] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.953143 16411 leader_election.cc:290] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.953534 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de14b1d525cc42808500279ae6b60e80" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:21.953859 14356 leader_election.cc:343] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.954425 14858 raft_consensus.cc:491] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.954416 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de14b1d525cc42808500279ae6b60e80" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.954444 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 15fbf8d5de904fc4b284af664f3bd60f
I20250115 03:06:21.954495 14858 raft_consensus.cc:513] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.954669 14163 catalog_manager.cc:6596] ReplaceTablet: tablet 15fbf8d5de904fc4b284af664f3bd60f deleted and replaced by tablet cd07196be4b54a90865aecea55493aa1
I20250115 03:06:21.954780 14858 leader_election.cc:290] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
W20250115 03:06:21.955039 14358 leader_election.cc:343] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.955101 14358 leader_election.cc:304] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.955287 14533 tablet_service.cc:1467] Processing CreateTablet for tablet 67ff648cc30b4f07a944005f53494e8c (DEFAULT_TABLE table=default.Table_f53f2ed27f714b12ab52472c670b506e [id=5d505c588806496f979567857f9f56df]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.955341 16411 raft_consensus.cc:2743] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.955467 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet 341b24d9b63f4761b2013035023fcd98 with delete_type TABLET_DATA_DELETED (replaced by tablet 306e4240c66647bc85d862252518f499) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.955595 14527 tablet_service.cc:1467] Processing CreateTablet for tablet f118a169019d4a32aa86b4fe9314f747 (DEFAULT_TABLE table=default.Table_12e083a49c774f2eab3c48fb2dd2cd0f [id=4e8899376d264906ab9f623b6ab235dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.956385 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 26fa8ff146834d32b97943e14f170e67 (DEFAULT_TABLE table=default.Table_47430d0013a7450aa28c3ef58ff9ca75 [id=1cb583178ccc40bfb51340dc54fc91f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.957234 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet 15fbf8d5de904fc4b284af664f3bd60f with delete_type TABLET_DATA_DELETED (replaced by tablet cd07196be4b54a90865aecea55493aa1) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.957120 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d4d372b6bd40428090e1280d674a8b20" } modify_external_catalogs: true
I20250115 03:06:21.957383 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d4d372b6bd40428090e1280d674a8b20" } modify_external_catalogs: true
I20250115 03:06:21.957741 16001 raft_consensus.cc:491] T 5db9d5f6cecb454a90ad931d8d165bcc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.957783 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a2894e4736b4191963a61d908bc673c" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.957818 16001 raft_consensus.cc:513] T 5db9d5f6cecb454a90ad931d8d165bcc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:21.958117 16001 leader_election.cc:290] T 5db9d5f6cecb454a90ad931d8d165bcc P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
W20250115 03:06:21.958423 14493 leader_election.cc:343] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.958544 14691 raft_consensus.cc:3054] T 5db9d5f6cecb454a90ad931d8d165bcc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.958927 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a2894e4736b4191963a61d908bc673c" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.959035 14419 raft_consensus.cc:2463] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:21.959105 14420 raft_consensus.cc:1270] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:21.959208 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 15fbf8d5de904fc4b284af664f3bd60f with delete_type TABLET_DATA_DELETED (replaced by tablet cd07196be4b54a90865aecea55493aa1) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.959193 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5db9d5f6cecb454a90ad931d8d165bcc" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:21.959363 14422 raft_consensus.cc:2388] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c0dae7aff4b44c699275b95ba639770 in current term 1: Already voted for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in this term.
I20250115 03:06:21.960013 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5db9d5f6cecb454a90ad931d8d165bcc" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:21.960053 14518 tablet_service.cc:1514] Processing DeleteTablet for tablet 15fbf8d5de904fc4b284af664f3bd60f with delete_type TABLET_DATA_DELETED (replaced by tablet cd07196be4b54a90865aecea55493aa1) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.960613 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_39adda20d0304f8f8785171a07e25531" } modify_external_catalogs: true
I20250115 03:06:21.961087 16411 raft_consensus.cc:491] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:21.961163 16411 raft_consensus.cc:513] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.961460 16411 leader_election.cc:290] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:21.961774 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aadcec8aca2b4ece915707c17f60b8b5" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:21.962077 14356 leader_election.cc:343] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.962375 14626 leader_election.cc:304] T 5db9d5f6cecb454a90ad931d8d165bcc P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:21.962813 14493 leader_election.cc:304] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770
I20250115 03:06:21.962949 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aadcec8aca2b4ece915707c17f60b8b5" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:21.963112 14853 raft_consensus.cc:2798] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:21.963163 14853 raft_consensus.cc:491] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:21.963196 14853 raft_consensus.cc:3054] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:21.963420 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_39adda20d0304f8f8785171a07e25531" } modify_external_catalogs: true
I20250115 03:06:21.964126 14858 consensus_queue.cc:1035] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:21.964519 14358 leader_election.cc:343] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:21.964515 14660 tablet_service.cc:1467] Processing CreateTablet for tablet ee9ef489671c454993d67632766ab1d7 (DEFAULT_TABLE table=default.Table_13922b2e0cc243669f2b64039b083650 [id=255684695cf04bd8ac1ed0cf676b6115]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.964583 14358 leader_election.cc:304] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:21.964860 16411 raft_consensus.cc:2743] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:21.964942 14658 tablet_service.cc:1467] Processing CreateTablet for tablet 22d43b546ec34e60b497d1ecca8b8a1e (DEFAULT_TABLE table=default.Table_14cc3f3be6bc492ca9ba0e1a879c8a0e [id=96da43cbbb6b4d2a9904ef96b58f6b9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.965317 14401 tablet_service.cc:1467] Processing CreateTablet for tablet ee9ef489671c454993d67632766ab1d7 (DEFAULT_TABLE table=default.Table_13922b2e0cc243669f2b64039b083650 [id=255684695cf04bd8ac1ed0cf676b6115]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.965806 14654 tablet_service.cc:1467] Processing CreateTablet for tablet 2d80f412b8824b65ab20154d46538165 (DEFAULT_TABLE table=default.Table_6496be150f7f4fddbc5cd00cf9f03544 [id=d982968727b842979459ab4b2dafd8aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.965834 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 22d43b546ec34e60b497d1ecca8b8a1e (DEFAULT_TABLE table=default.Table_14cc3f3be6bc492ca9ba0e1a879c8a0e [id=96da43cbbb6b4d2a9904ef96b58f6b9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.966120 14389 tablet_service.cc:1467] Processing CreateTablet for tablet 2d80f412b8824b65ab20154d46538165 (DEFAULT_TABLE table=default.Table_6496be150f7f4fddbc5cd00cf9f03544 [id=d982968727b842979459ab4b2dafd8aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.966580 14672 tablet_service.cc:1467] Processing CreateTablet for tablet db486cc5a4f84ab6ac6d2c6aa02eca3d (DEFAULT_TABLE table=default.Table_b3ed368ac6d8449d841b1f7522cd7815 [id=7c4c87b2f9054dacac52cd117c02ae10]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.967583 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 306e4240c66647bc85d862252518f499 (DEFAULT_TABLE table=default.Table_2d473b82a1764ef6afd586a7bf8e17dc [id=369c42ec23ec4a27a71a54be23a410a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.968047 14388 tablet_service.cc:1467] Processing CreateTablet for tablet db486cc5a4f84ab6ac6d2c6aa02eca3d (DEFAULT_TABLE table=default.Table_b3ed368ac6d8449d841b1f7522cd7815 [id=7c4c87b2f9054dacac52cd117c02ae10]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.969122 14387 tablet_service.cc:1467] Processing CreateTablet for tablet 306e4240c66647bc85d862252518f499 (DEFAULT_TABLE table=default.Table_2d473b82a1764ef6afd586a7bf8e17dc [id=369c42ec23ec4a27a71a54be23a410a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.971318 14534 tablet_service.cc:1467] Processing CreateTablet for tablet 2d80f412b8824b65ab20154d46538165 (DEFAULT_TABLE table=default.Table_6496be150f7f4fddbc5cd00cf9f03544 [id=d982968727b842979459ab4b2dafd8aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.971534 14518 tablet_service.cc:1467] Processing CreateTablet for tablet ee9ef489671c454993d67632766ab1d7 (DEFAULT_TABLE table=default.Table_13922b2e0cc243669f2b64039b083650 [id=255684695cf04bd8ac1ed0cf676b6115]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.971624 14536 tablet_service.cc:1467] Processing CreateTablet for tablet db486cc5a4f84ab6ac6d2c6aa02eca3d (DEFAULT_TABLE table=default.Table_b3ed368ac6d8449d841b1f7522cd7815 [id=7c4c87b2f9054dacac52cd117c02ae10]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.971318 14521 tablet_service.cc:1467] Processing CreateTablet for tablet 22d43b546ec34e60b497d1ecca8b8a1e (DEFAULT_TABLE table=default.Table_14cc3f3be6bc492ca9ba0e1a879c8a0e [id=96da43cbbb6b4d2a9904ef96b58f6b9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.972118 14169 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_dfcccd3390fa4814a0a04bb1ff5e43fa" } modify_external_catalogs: true
I20250115 03:06:21.972165 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_13922b2e0cc243669f2b64039b083650" } modify_external_catalogs: true
I20250115 03:06:21.972225 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_13922b2e0cc243669f2b64039b083650" } modify_external_catalogs: true
I20250115 03:06:21.972224 14169 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_dfcccd3390fa4814a0a04bb1ff5e43fa" } modify_external_catalogs: true
I20250115 03:06:21.975348 14167 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5db9d5f6cecb454a90ad931d8d165bcc (table default.Table_ca82711024824acda4e6b4d3cf4f0537 [id=07019baad4a74fd0b468bbd9b60d54c8]). Expected version 2 got 0
I20250115 03:06:21.975665 14170 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet a8e41e31ffae4bb092cb08d83eda16b5 (table default.Table_5a9c75c8867b4381bb170cecd8dd6acf [id=a530442772dc4b12b23992cee12fea85]). Expected version 1 got 0
I20250115 03:06:21.977721 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 306e4240c66647bc85d862252518f499 (DEFAULT_TABLE table=default.Table_2d473b82a1764ef6afd586a7bf8e17dc [id=369c42ec23ec4a27a71a54be23a410a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:21.982337 14809 tablet_bootstrap.cc:492] T f2b8bd783b614bf9b5526de197c51df3 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:21.982333 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet cc328a01837a4c21abc38c7d85557e15
I20250115 03:06:21.982419 14809 ts_tablet_manager.cc:1397] T f2b8bd783b614bf9b5526de197c51df3 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.099s user 0.002s sys 0.000s
I20250115 03:06:21.982476 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet ee9ef489671c454993d67632766ab1d7
I20250115 03:06:21.982734 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 0521973e23984355a77a321b4676b479
I20250115 03:06:21.982797 14169 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 7c503b29cb2843338fb1870a89aee443
I20250115 03:06:21.982838 14809 raft_consensus.cc:357] T f2b8bd783b614bf9b5526de197c51df3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.982928 14809 raft_consensus.cc:383] T f2b8bd783b614bf9b5526de197c51df3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.982952 14809 raft_consensus.cc:738] T f2b8bd783b614bf9b5526de197c51df3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.983052 14809 consensus_queue.cc:260] T f2b8bd783b614bf9b5526de197c51df3 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.983321 14809 ts_tablet_manager.cc:1428] T f2b8bd783b614bf9b5526de197c51df3 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.983487 14809 tablet_bootstrap.cc:492] T cef4442bfcfd4ad4b343a2785c3a9a74 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:21.984444 14809 tablet_bootstrap.cc:654] T cef4442bfcfd4ad4b343a2785c3a9a74 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.985702 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet 6004f4adca8e47ddb91f7e39c4cf9089 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:21.986068 14524 tablet_service.cc:1514] Processing DeleteTablet for tablet ee9ef489671c454993d67632766ab1d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.986204 14519 tablet_service.cc:1514] Processing DeleteTablet for tablet 0521973e23984355a77a321b4676b479 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.986306 14523 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c503b29cb2843338fb1870a89aee443 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.986441 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet cc328a01837a4c21abc38c7d85557e15 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:21.987212 14661 tablet_service.cc:1514] Processing DeleteTablet for tablet ee9ef489671c454993d67632766ab1d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.987347 14667 tablet_service.cc:1514] Processing DeleteTablet for tablet cc328a01837a4c21abc38c7d85557e15 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.987442 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 0521973e23984355a77a321b4676b479 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.987545 14666 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c503b29cb2843338fb1870a89aee443 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:21.987902 14386 tablet_service.cc:1514] Processing DeleteTablet for tablet cc328a01837a4c21abc38c7d85557e15 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.987947 14384 tablet_service.cc:1514] Processing DeleteTablet for tablet ee9ef489671c454993d67632766ab1d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.988091 14385 tablet_service.cc:1514] Processing DeleteTablet for tablet 0521973e23984355a77a321b4676b479 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.988498 14386 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c503b29cb2843338fb1870a89aee443 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:21.990701 14811 tablet_bootstrap.cc:492] T b61cc4b11251486a8089ca9bf44ac91e P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:21.990793 14811 ts_tablet_manager.cc:1397] T b61cc4b11251486a8089ca9bf44ac91e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.091s user 0.001s sys 0.000s
I20250115 03:06:21.991228 14811 raft_consensus.cc:357] T b61cc4b11251486a8089ca9bf44ac91e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.991357 14811 raft_consensus.cc:383] T b61cc4b11251486a8089ca9bf44ac91e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:21.991393 14811 raft_consensus.cc:738] T b61cc4b11251486a8089ca9bf44ac91e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:21.991513 14811 consensus_queue.cc:260] T b61cc4b11251486a8089ca9bf44ac91e P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:21.991803 14811 ts_tablet_manager.cc:1428] T b61cc4b11251486a8089ca9bf44ac91e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:21.992000 14811 tablet_bootstrap.cc:492] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:21.992897 14811 tablet_bootstrap.cc:654] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:21.994836 14858 consensus_queue.cc:1035] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.000954 16088 raft_consensus.cc:491] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.001046 16088 raft_consensus.cc:513] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.001385 16088 leader_election.cc:290] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.001652 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72abf6acd9d54f3b983b7098aaf2691d" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.001775 14422 raft_consensus.cc:2463] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.002040 14628 leader_election.cc:304] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.002233 16088 raft_consensus.cc:2798] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.002280 16088 raft_consensus.cc:491] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.002314 16088 raft_consensus.cc:3054] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.005507 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72abf6acd9d54f3b983b7098aaf2691d" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.005622 14552 raft_consensus.cc:2463] T 72abf6acd9d54f3b983b7098aaf2691d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.005797 14820 ts_tablet_manager.cc:1918] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20250115 03:06:22.005863 14820 log.cc:1198] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/ccf6b58c55d747da89c1c218a70ce09f
I20250115 03:06:22.006084 14820 ts_tablet_manager.cc:1939] T ccf6b58c55d747da89c1c218a70ce09f P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:22.006174 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67ff648cc30b4f07a944005f53494e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.006482 14388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db486cc5a4f84ab6ac6d2c6aa02eca3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.006625 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee9ef489671c454993d67632766ab1d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.006994 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26fa8ff146834d32b97943e14f170e67. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.007195 16555 tablet.cc:1721] T 5db9d5f6cecb454a90ad931d8d165bcc P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:22.007265 14392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f118a169019d4a32aa86b4fe9314f747. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.008118 16411 raft_consensus.cc:491] T e47012a789bd469f9691e5a4162ff512 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.008198 16411 raft_consensus.cc:513] T e47012a789bd469f9691e5a4162ff512 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.008558 16411 leader_election.cc:290] T e47012a789bd469f9691e5a4162ff512 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.008639 14812 tablet_bootstrap.cc:492] T 6ed47b26cdf744acb28fb657a1b41087 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.008718 14812 ts_tablet_manager.cc:1397] T 6ed47b26cdf744acb28fb657a1b41087 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.062s user 0.001s sys 0.000s
I20250115 03:06:22.008899 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e47012a789bd469f9691e5a4162ff512" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.009194 14812 raft_consensus.cc:357] T 6ed47b26cdf744acb28fb657a1b41087 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:22.009241 14358 leader_election.cc:343] T e47012a789bd469f9691e5a4162ff512 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.009286 14812 raft_consensus.cc:383] T 6ed47b26cdf744acb28fb657a1b41087 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.009315 14812 raft_consensus.cc:738] T 6ed47b26cdf744acb28fb657a1b41087 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.009418 14812 consensus_queue.cc:260] T 6ed47b26cdf744acb28fb657a1b41087 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.009560 16292 raft_consensus.cc:513] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.009609 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e47012a789bd469f9691e5a4162ff512" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.009744 14812 ts_tablet_manager.cc:1428] T 6ed47b26cdf744acb28fb657a1b41087 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.009852 16292 leader_election.cc:290] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.009933 14812 tablet_bootstrap.cc:492] T 6bd4e4200f9b489ea5dd2154aa6f382d P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:22.009930 14356 leader_election.cc:343] T e47012a789bd469f9691e5a4162ff512 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.009989 14356 leader_election.cc:304] T e47012a789bd469f9691e5a4162ff512 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.010097 16411 raft_consensus.cc:2743] T e47012a789bd469f9691e5a4162ff512 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.010324 14418 raft_consensus.cc:2463] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:22.010599 14659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67ff648cc30b4f07a944005f53494e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.010706 14493 leader_election.cc:304] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.010804 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f118a169019d4a32aa86b4fe9314f747. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.010886 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 306e4240c66647bc85d862252518f499. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.010875 14853 raft_consensus.cc:513] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.011092 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d80f412b8824b65ab20154d46538165. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.011124 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db486cc5a4f84ab6ac6d2c6aa02eca3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.011214 14853 leader_election.cc:290] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.011327 14853 raft_consensus.cc:2798] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:22.011376 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f118a169019d4a32aa86b4fe9314f747. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.011380 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db486cc5a4f84ab6ac6d2c6aa02eca3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.011474 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22d43b546ec34e60b497d1ecca8b8a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.011634 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22d43b546ec34e60b497d1ecca8b8a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.011752 14389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d80f412b8824b65ab20154d46538165. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.011914 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 306e4240c66647bc85d862252518f499. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.011996 14387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 306e4240c66647bc85d862252518f499. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.012032 16554 tablet.cc:1721] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:22.011387 14853 raft_consensus.cc:695] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:22.012709 14853 consensus_queue.cc:237] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.014031 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67ff648cc30b4f07a944005f53494e8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.014489 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26fa8ff146834d32b97943e14f170e67. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.014876 14812 tablet_bootstrap.cc:654] T 6bd4e4200f9b489ea5dd2154aa6f382d P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.015483 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee9ef489671c454993d67632766ab1d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.015754 14658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22d43b546ec34e60b497d1ecca8b8a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.016052 14654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d80f412b8824b65ab20154d46538165. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.019335 14809 tablet_bootstrap.cc:492] T cef4442bfcfd4ad4b343a2785c3a9a74 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.019407 14809 ts_tablet_manager.cc:1397] T cef4442bfcfd4ad4b343a2785c3a9a74 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.036s user 0.001s sys 0.000s
I20250115 03:06:22.019838 14809 raft_consensus.cc:357] T cef4442bfcfd4ad4b343a2785c3a9a74 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.019932 14809 raft_consensus.cc:383] T cef4442bfcfd4ad4b343a2785c3a9a74 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.019959 14809 raft_consensus.cc:738] T cef4442bfcfd4ad4b343a2785c3a9a74 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.020064 14809 consensus_queue.cc:260] T cef4442bfcfd4ad4b343a2785c3a9a74 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.020232 14418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b39608e19bf44b3a79a8616ccdad30a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:22.020325 14418 raft_consensus.cc:3054] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.020541 14518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee9ef489671c454993d67632766ab1d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.020709 14811 tablet_bootstrap.cc:492] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.020781 14811 ts_tablet_manager.cc:1397] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.029s user 0.000s sys 0.001s
I20250115 03:06:22.021188 14811 raft_consensus.cc:357] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.021258 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26fa8ff146834d32b97943e14f170e67. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.021286 14811 raft_consensus.cc:383] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.021320 14811 raft_consensus.cc:738] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.021410 14811 consensus_queue.cc:260] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.021670 14811 ts_tablet_manager.cc:1428] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:22.021831 14811 tablet_bootstrap.cc:492] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.022568 14811 tablet_bootstrap.cc:654] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.023288 14169 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ef1516de522f4c0aa275c9752c4fe452" } new_table_name: "default.Table_2e11c8e596be45598a86f0439162b164" modify_external_catalogs: true
I20250115 03:06:22.024681 16292 consensus_queue.cc:1035] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.027101 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b39608e19bf44b3a79a8616ccdad30a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:22.027207 14552 raft_consensus.cc:3054] T 4b39608e19bf44b3a79a8616ccdad30a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.027849 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_20607cbd2ddf454bb006aa12bb3eca45" } new_table_name: "default.Table_4479fcf9dae14e2b82d8400944206205" modify_external_catalogs: true
I20250115 03:06:22.028052 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_962c421edc794976b03cf360d8e14638"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.028383 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_962c421edc794976b03cf360d8e14638 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.029496 16292 consensus_queue.cc:1035] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.030377 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a2894e4736b4191963a61d908bc673c" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:22.030867 16189 raft_consensus.cc:491] T b61cc4b11251486a8089ca9bf44ac91e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.030942 16189 raft_consensus.cc:513] T b61cc4b11251486a8089ca9bf44ac91e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.031193 16189 leader_election.cc:290] T b61cc4b11251486a8089ca9bf44ac91e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
W20250115 03:06:22.031448 14493 leader_election.cc:343] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.031813 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b61cc4b11251486a8089ca9bf44ac91e" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.032128 14493 leader_election.cc:343] T b61cc4b11251486a8089ca9bf44ac91e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.032639 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a2894e4736b4191963a61d908bc673c" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:22.032748 14420 raft_consensus.cc:3054] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.020330 14809 ts_tablet_manager.cc:1428] T cef4442bfcfd4ad4b343a2785c3a9a74 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.034397 14809 tablet_bootstrap.cc:492] T 87b234185fc64505b097f2ae7d52d813 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.034755 16292 consensus_queue.cc:1035] T 7b86e66428504217af08f667f660d304 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.035315 14809 tablet_bootstrap.cc:654] T 87b234185fc64505b097f2ae7d52d813 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.039304 14166 catalog_manager.cc:5526] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:22.039502 14166 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet b08e5fc2110e476e897240f9cf3b874d (table default.Table_6b6164bc5f2e4395b5d31047b8ef87d9 [id=f8f2a7ff6fdc40468de6ae9ba28c1427]). Expected version 4 got 0
I20250115 03:06:22.039911 14167 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_802f667f1ad246688cbdcffcb3001021 [id=e929ffaba567498c815393836b264a12] alter complete (version 1)
I20250115 03:06:22.041337 14395 tablet_service.cc:1467] Processing CreateTablet for tablet cd07196be4b54a90865aecea55493aa1 (DEFAULT_TABLE table=default.Table_655e9d02aacf4b88ba521f0f57a5f522 [id=6ed01835736b40f8b4caeca7378770d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.041337 14666 tablet_service.cc:1467] Processing CreateTablet for tablet cd07196be4b54a90865aecea55493aa1 (DEFAULT_TABLE table=default.Table_655e9d02aacf4b88ba521f0f57a5f522 [id=6ed01835736b40f8b4caeca7378770d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.042019 14809 tablet_bootstrap.cc:492] T 87b234185fc64505b097f2ae7d52d813 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.042104 14809 ts_tablet_manager.cc:1397] T 87b234185fc64505b097f2ae7d52d813 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20250115 03:06:22.042843 14809 raft_consensus.cc:357] T 87b234185fc64505b097f2ae7d52d813 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.043077 14809 raft_consensus.cc:383] T 87b234185fc64505b097f2ae7d52d813 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.043151 14809 raft_consensus.cc:738] T 87b234185fc64505b097f2ae7d52d813 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.043335 14809 consensus_queue.cc:260] T 87b234185fc64505b097f2ae7d52d813 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.043844 14809 ts_tablet_manager.cc:1428] T 87b234185fc64505b097f2ae7d52d813 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:22.044056 14809 tablet_bootstrap.cc:492] T 041a30d6ee3247df883d00b5896b0eb0 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.045267 14809 tablet_bootstrap.cc:654] T 041a30d6ee3247df883d00b5896b0eb0 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:22.045360 14169 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5b06b08f0a864629997efc0c98e84e04 consensus state has no leader
I20250115 03:06:22.047330 14169 master_service.cc:946] ReplaceTablet: received request to replace tablet 2a9144a303eb4db08840b2f2749c52d6 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:22.051179 16411 raft_consensus.cc:491] T cef4442bfcfd4ad4b343a2785c3a9a74 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.051251 16411 raft_consensus.cc:513] T cef4442bfcfd4ad4b343a2785c3a9a74 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.051573 16411 leader_election.cc:290] T cef4442bfcfd4ad4b343a2785c3a9a74 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.051574 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b7bf51cb55344d7083f4fda2cfa2e1fd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.051906 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b7bf51cb55344d7083f4fda2cfa2e1fd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.052197 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef4442bfcfd4ad4b343a2785c3a9a74" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.052220 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef4442bfcfd4ad4b343a2785c3a9a74" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.052531 14170 master_service.cc:946] ReplaceTablet: received request to replace tablet 0580a8f7b2984162b9275d74334b1905 from {username='slave'} at 127.0.0.1:60432
W20250115 03:06:22.053860 14358 leader_election.cc:343] T cef4442bfcfd4ad4b343a2785c3a9a74 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.054091 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet f2b8bd783b614bf9b5526de197c51df3 (table default.Table_4fbcf95abbb546dfb26e14668b42465f [id=5ebda2c8e8c14092a6e894b5502e4763]). Expected version 1 got 0
I20250115 03:06:22.054199 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 87b234185fc64505b097f2ae7d52d813 (table default.Table_9f9940dec82b49098561c9a2e90b3475 [id=bd521742ddaa43d18ef24465d5ba313b]). Expected version 1 got 0
I20250115 03:06:22.054273 14169 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 7b86e66428504217af08f667f660d304 (table default.Table_341a3cd1c8c94ff4b05fc21960a4e9b4 [id=c096ce170874405ea740b9ea829f886e]). Expected version 2 got 0
I20250115 03:06:22.055593 16411 raft_consensus.cc:491] T fe8d94c8fecc4077ba052390810c7430 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.055667 16411 raft_consensus.cc:513] T fe8d94c8fecc4077ba052390810c7430 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:22.055884 14162 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c8c73857d8184dc8a535b562b966ff4f consensus state has no leader
I20250115 03:06:22.055922 16411 leader_election.cc:290] T fe8d94c8fecc4077ba052390810c7430 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.057503 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe8d94c8fecc4077ba052390810c7430" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.057835 14358 leader_election.cc:343] T fe8d94c8fecc4077ba052390810c7430 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:22.058808 14356 leader_election.cc:343] T cef4442bfcfd4ad4b343a2785c3a9a74 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.058867 14356 leader_election.cc:304] T cef4442bfcfd4ad4b343a2785c3a9a74 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.059047 16411 raft_consensus.cc:2743] T cef4442bfcfd4ad4b343a2785c3a9a74 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.060179 14537 tablet_service.cc:1467] Processing CreateTablet for tablet cd07196be4b54a90865aecea55493aa1 (DEFAULT_TABLE table=default.Table_655e9d02aacf4b88ba521f0f57a5f522 [id=6ed01835736b40f8b4caeca7378770d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.062489 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe8d94c8fecc4077ba052390810c7430" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:22.062722 14493 leader_election.cc:343] T b61cc4b11251486a8089ca9bf44ac91e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: b61cc4b11251486a8089ca9bf44ac91e
I20250115 03:06:22.062791 14493 leader_election.cc:304] T b61cc4b11251486a8089ca9bf44ac91e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:22.062906 16411 raft_consensus.cc:491] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.062922 16189 raft_consensus.cc:2743] T b61cc4b11251486a8089ca9bf44ac91e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.062971 16411 raft_consensus.cc:513] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.063241 16411 leader_election.cc:290] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:22.063350 14493 consensus_peers.cc:487] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 -> Peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Couldn't send request to peer e8df7247ee9c43e0aa18216da2661f75. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1016cf85491d4ceda132ee57325b48a4. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:22.063658 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cc96de8b414ac19285338ccb1024e0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.064030 14358 leader_election.cc:343] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.064225 16189 raft_consensus.cc:491] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.064296 16189 raft_consensus.cc:513] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.064602 16189 leader_election.cc:290] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.064891 16189 consensus_queue.cc:1035] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.065182 16411 raft_consensus.cc:491] T a8a5743475814401ac2c0a51cfd70207 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.065265 16411 raft_consensus.cc:513] T a8a5743475814401ac2c0a51cfd70207 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.065527 16411 leader_election.cc:290] T a8a5743475814401ac2c0a51cfd70207 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.065589 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9818d0493547358fd5654a01d4f456" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.065685 14687 raft_consensus.cc:2463] T 7f9818d0493547358fd5654a01d4f456 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.065987 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a5743475814401ac2c0a51cfd70207" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.067272 14493 leader_election.cc:304] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:22.067503 16189 raft_consensus.cc:2798] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.067545 16189 raft_consensus.cc:491] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.067576 16189 raft_consensus.cc:3054] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:22.068717 14356 leader_election.cc:343] T fe8d94c8fecc4077ba052390810c7430 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.068780 14356 leader_election.cc:304] T fe8d94c8fecc4077ba052390810c7430 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.069208 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a5743475814401ac2c0a51cfd70207" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.069348 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cc96de8b414ac19285338ccb1024e0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.069506 14417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9818d0493547358fd5654a01d4f456" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.069607 14417 raft_consensus.cc:2463] T 7f9818d0493547358fd5654a01d4f456 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
W20250115 03:06:22.070163 14356 leader_election.cc:343] T a8a5743475814401ac2c0a51cfd70207 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:22.070282 14356 leader_election.cc:343] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.070326 14356 leader_election.cc:304] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.070441 16406 raft_consensus.cc:2743] T fe8d94c8fecc4077ba052390810c7430 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.070549 16406 raft_consensus.cc:2743] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.071144 16570 tablet.cc:1721] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20250115 03:06:22.071455 14358 leader_election.cc:343] T a8a5743475814401ac2c0a51cfd70207 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.071525 14358 leader_election.cc:304] T a8a5743475814401ac2c0a51cfd70207 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.072130 16411 raft_consensus.cc:2743] T a8a5743475814401ac2c0a51cfd70207 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.074795 16571 tablet.cc:1721] T a4547465f3124e108dc2c4c54e2bc751 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:22.076903 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0654e6fa86284deab15640918a465fbc" } new_table_name: "default.Table_2eec7fe42b894a729d365bc75c060236" modify_external_catalogs: true
I20250115 03:06:22.077083 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_36648b7bea684eecb64fad0ae41e7b53"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.077405 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_36648b7bea684eecb64fad0ae41e7b53 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.078539 16568 consensus_queue.cc:1035] T 998a515113d04034928c04dc5798a698 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.078680 14165 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet a4547465f3124e108dc2c4c54e2bc751 (table default.Table_58e0901d0ec94ef9859515f126825621 [id=50c73c26b4ed47a4961348db433b7dd4]). Expected version 3 got 0
I20250115 03:06:22.079254 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d120fbdbf5d746d287477050fdc9959c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.079530 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d120fbdbf5d746d287477050fdc9959c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.080339 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a0c249d69b904390952a780b10e1369b" } new_table_name: "default.Table_77761cbb0c804fa4a8d4298c4db47f76" modify_external_catalogs: true
I20250115 03:06:22.081141 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_c88e706cbc9d4c91bbb662eede011de6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.081422 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c88e706cbc9d4c91bbb662eede011de6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.083120 14663 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ed47b26cdf744acb28fb657a1b41087 with delete_type TABLET_DATA_DELETED (replaced by tablet fe8d94c8fecc4077ba052390810c7430) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:22.084142 14853 consensus_queue.cc:1035] T a4547465f3124e108dc2c4c54e2bc751 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.084385 14853 consensus_queue.cc:1035] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.093632 14812 tablet_bootstrap.cc:492] T 6bd4e4200f9b489ea5dd2154aa6f382d P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.093710 14812 ts_tablet_manager.cc:1397] T 6bd4e4200f9b489ea5dd2154aa6f382d P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.084s user 0.002s sys 0.000s
I20250115 03:06:22.094127 14812 raft_consensus.cc:357] T 6bd4e4200f9b489ea5dd2154aa6f382d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.094169 14171 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 7b86e66428504217af08f667f660d304 (table default.Table_341a3cd1c8c94ff4b05fc21960a4e9b4 [id=c096ce170874405ea740b9ea829f886e]). Expected version 2 got 0
I20250115 03:06:22.094216 14812 raft_consensus.cc:383] T 6bd4e4200f9b489ea5dd2154aa6f382d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.094244 14812 raft_consensus.cc:738] T 6bd4e4200f9b489ea5dd2154aa6f382d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.094339 14812 consensus_queue.cc:260] T 6bd4e4200f9b489ea5dd2154aa6f382d P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.094619 14812 ts_tablet_manager.cc:1428] T 6bd4e4200f9b489ea5dd2154aa6f382d P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.094770 14812 tablet_bootstrap.cc:492] T d891020ed0bb4fc49155bd9f2f6ed35c P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.095634 14812 tablet_bootstrap.cc:654] T d891020ed0bb4fc49155bd9f2f6ed35c P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.097762 14171 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_54bdb95d2f9244c2a2e3559a36b1f6e1" } new_table_name: "default.Table_1d176a5c544a42e0b270d132e5f9889e" modify_external_catalogs: true
I20250115 03:06:22.099889 14811 tablet_bootstrap.cc:492] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.099972 14811 ts_tablet_manager.cc:1397] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.078s user 0.001s sys 0.001s
I20250115 03:06:22.100349 14811 raft_consensus.cc:357] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.100445 14811 raft_consensus.cc:383] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.100471 14811 raft_consensus.cc:738] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.100567 14811 consensus_queue.cc:260] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.100805 14811 ts_tablet_manager.cc:1428] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:22.101024 14811 tablet_bootstrap.cc:492] T 398b93f1db0741a9b54837bca43b1bad P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.101935 14811 tablet_bootstrap.cc:654] T 398b93f1db0741a9b54837bca43b1bad P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.102572 14809 tablet_bootstrap.cc:492] T 041a30d6ee3247df883d00b5896b0eb0 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.102663 14809 ts_tablet_manager.cc:1397] T 041a30d6ee3247df883d00b5896b0eb0 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.059s user 0.002s sys 0.000s
I20250115 03:06:22.103104 14809 raft_consensus.cc:357] T 041a30d6ee3247df883d00b5896b0eb0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.103217 14809 raft_consensus.cc:383] T 041a30d6ee3247df883d00b5896b0eb0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.103250 14809 raft_consensus.cc:738] T 041a30d6ee3247df883d00b5896b0eb0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.103361 14809 consensus_queue.cc:260] T 041a30d6ee3247df883d00b5896b0eb0 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.103623 14809 ts_tablet_manager.cc:1428] T 041a30d6ee3247df883d00b5896b0eb0 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:22.103789 14809 tablet_bootstrap.cc:492] T 2e7cf4fc3be8446292b0c91141eeadcc P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.104523 14809 tablet_bootstrap.cc:654] T 2e7cf4fc3be8446292b0c91141eeadcc P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.105217 14552 raft_consensus.cc:2463] T 4b39608e19bf44b3a79a8616ccdad30a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:22.107088 16189 raft_consensus.cc:513] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.107378 16189 leader_election.cc:290] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.107944 16406 raft_consensus.cc:491] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.108083 16406 raft_consensus.cc:513] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.107925 16575 tablet.cc:1721] T a4547465f3124e108dc2c4c54e2bc751 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20250115 03:06:22.108371 16406 leader_election.cc:290] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.109191 14812 tablet_bootstrap.cc:492] T d891020ed0bb4fc49155bd9f2f6ed35c P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.109267 14812 ts_tablet_manager.cc:1397] T d891020ed0bb4fc49155bd9f2f6ed35c P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20250115 03:06:22.109711 14812 raft_consensus.cc:357] T d891020ed0bb4fc49155bd9f2f6ed35c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.109822 14812 raft_consensus.cc:383] T d891020ed0bb4fc49155bd9f2f6ed35c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.109854 14812 raft_consensus.cc:738] T d891020ed0bb4fc49155bd9f2f6ed35c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.109957 14812 consensus_queue.cc:260] T d891020ed0bb4fc49155bd9f2f6ed35c P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.110222 14812 ts_tablet_manager.cc:1428] T d891020ed0bb4fc49155bd9f2f6ed35c P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.110417 14812 tablet_bootstrap.cc:492] T c2afa048b3fd4362b50c8a6da306c25a P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.111346 14812 tablet_bootstrap.cc:654] T c2afa048b3fd4362b50c8a6da306c25a P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.112227 14626 leader_election.cc:304] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:22.112514 16557 raft_consensus.cc:2798] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:22.112593 16557 raft_consensus.cc:695] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:22.112576 16567 consensus_queue.cc:1035] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.112704 16557 consensus_queue.cc:237] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.113315 16292 consensus_queue.cc:1035] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.117874 15153 raft_consensus.cc:513] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.118213 15153 leader_election.cc:290] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.120126 14417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad0e9216bd154f4a8a37874d3d8a08c2" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:22.120229 14417 raft_consensus.cc:3054] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.120358 14812 tablet_bootstrap.cc:492] T c2afa048b3fd4362b50c8a6da306c25a P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.120436 14812 ts_tablet_manager.cc:1397] T c2afa048b3fd4362b50c8a6da306c25a P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20250115 03:06:22.120977 14812 raft_consensus.cc:357] T c2afa048b3fd4362b50c8a6da306c25a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.121062 14812 raft_consensus.cc:383] T c2afa048b3fd4362b50c8a6da306c25a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.121090 14812 raft_consensus.cc:738] T c2afa048b3fd4362b50c8a6da306c25a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.121189 14812 consensus_queue.cc:260] T c2afa048b3fd4362b50c8a6da306c25a P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.121412 14421 raft_consensus.cc:2463] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:22.121436 14812 ts_tablet_manager.cc:1428] T c2afa048b3fd4362b50c8a6da306c25a P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.121601 14812 tablet_bootstrap.cc:492] T fca14a7d574842ee94a93afb85ca1afd P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.122484 14812 tablet_bootstrap.cc:654] T fca14a7d574842ee94a93afb85ca1afd P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.125270 14812 tablet_bootstrap.cc:492] T fca14a7d574842ee94a93afb85ca1afd P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.125344 14812 ts_tablet_manager.cc:1397] T fca14a7d574842ee94a93afb85ca1afd P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250115 03:06:22.125746 14812 raft_consensus.cc:357] T fca14a7d574842ee94a93afb85ca1afd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.125851 14812 raft_consensus.cc:383] T fca14a7d574842ee94a93afb85ca1afd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.125882 14812 raft_consensus.cc:738] T fca14a7d574842ee94a93afb85ca1afd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.125985 14812 consensus_queue.cc:260] T fca14a7d574842ee94a93afb85ca1afd P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.126242 14812 ts_tablet_manager.cc:1428] T fca14a7d574842ee94a93afb85ca1afd P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.126385 14812 tablet_bootstrap.cc:492] T 5a2894e4736b4191963a61d908bc673c P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.127457 14812 tablet_bootstrap.cc:654] T 5a2894e4736b4191963a61d908bc673c P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.128178 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78b5e2e16442407880b75d256e6b2614" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.128505 14358 leader_election.cc:343] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.129578 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad0e9216bd154f4a8a37874d3d8a08c2" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:22.129674 14557 raft_consensus.cc:3054] T ad0e9216bd154f4a8a37874d3d8a08c2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.129760 15153 consensus_queue.cc:1035] T 7043616bd6b34299862336924a6f16bf P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.132141 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78b5e2e16442407880b75d256e6b2614" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:22.133596 14356 leader_election.cc:343] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.133630 14811 tablet_bootstrap.cc:492] T 398b93f1db0741a9b54837bca43b1bad P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.133670 14356 leader_election.cc:304] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.133708 14811 ts_tablet_manager.cc:1397] T 398b93f1db0741a9b54837bca43b1bad P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.000s
I20250115 03:06:22.133874 16411 raft_consensus.cc:2743] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.134156 14811 raft_consensus.cc:357] T 398b93f1db0741a9b54837bca43b1bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.134245 14811 raft_consensus.cc:383] T 398b93f1db0741a9b54837bca43b1bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.134280 14811 raft_consensus.cc:738] T 398b93f1db0741a9b54837bca43b1bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.134371 14811 consensus_queue.cc:260] T 398b93f1db0741a9b54837bca43b1bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.134644 14811 ts_tablet_manager.cc:1428] T 398b93f1db0741a9b54837bca43b1bad P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.134810 14811 tablet_bootstrap.cc:492] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.135696 14811 tablet_bootstrap.cc:654] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.137065 14809 tablet_bootstrap.cc:492] T 2e7cf4fc3be8446292b0c91141eeadcc P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.137140 14809 ts_tablet_manager.cc:1397] T 2e7cf4fc3be8446292b0c91141eeadcc P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.000s
I20250115 03:06:22.137568 14809 raft_consensus.cc:357] T 2e7cf4fc3be8446292b0c91141eeadcc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.137662 14809 raft_consensus.cc:383] T 2e7cf4fc3be8446292b0c91141eeadcc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.137696 14809 raft_consensus.cc:738] T 2e7cf4fc3be8446292b0c91141eeadcc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.137807 14809 consensus_queue.cc:260] T 2e7cf4fc3be8446292b0c91141eeadcc P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.138052 14809 ts_tablet_manager.cc:1428] T 2e7cf4fc3be8446292b0c91141eeadcc P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.138203 14809 tablet_bootstrap.cc:492] T ef0994b1e51b4c688354998730e7b743 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.138891 14809 tablet_bootstrap.cc:654] T ef0994b1e51b4c688354998730e7b743 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.139590 16406 raft_consensus.cc:491] T 238e68300f304266aaed94bc3b76a01d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.139663 16406 raft_consensus.cc:513] T 238e68300f304266aaed94bc3b76a01d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.139950 16406 leader_election.cc:290] T 238e68300f304266aaed94bc3b76a01d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.140295 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "238e68300f304266aaed94bc3b76a01d" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.140990 14358 leader_election.cc:343] T 238e68300f304266aaed94bc3b76a01d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.142230 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "238e68300f304266aaed94bc3b76a01d" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.142514 16411 raft_consensus.cc:491] T f2b8bd783b614bf9b5526de197c51df3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.142585 16411 raft_consensus.cc:513] T f2b8bd783b614bf9b5526de197c51df3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.142849 16411 leader_election.cc:290] T f2b8bd783b614bf9b5526de197c51df3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.143303 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2b8bd783b614bf9b5526de197c51df3" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.143339 14356 leader_election.cc:343] T 238e68300f304266aaed94bc3b76a01d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.143451 14356 leader_election.cc:304] T 238e68300f304266aaed94bc3b76a01d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
W20250115 03:06:22.143718 14358 leader_election.cc:343] T f2b8bd783b614bf9b5526de197c51df3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.143867 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2b8bd783b614bf9b5526de197c51df3" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.144163 16411 raft_consensus.cc:2743] T 238e68300f304266aaed94bc3b76a01d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:22.144323 14356 leader_election.cc:343] T f2b8bd783b614bf9b5526de197c51df3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.144371 14356 leader_election.cc:304] T f2b8bd783b614bf9b5526de197c51df3 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.144524 16411 raft_consensus.cc:2743] T f2b8bd783b614bf9b5526de197c51df3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.144775 14809 tablet_bootstrap.cc:492] T ef0994b1e51b4c688354998730e7b743 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.144845 14809 ts_tablet_manager.cc:1397] T ef0994b1e51b4c688354998730e7b743 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:22.145231 14809 raft_consensus.cc:357] T ef0994b1e51b4c688354998730e7b743 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.145313 14809 raft_consensus.cc:383] T ef0994b1e51b4c688354998730e7b743 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.145339 14809 raft_consensus.cc:738] T ef0994b1e51b4c688354998730e7b743 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.145748 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9818d0493547358fd5654a01d4f456" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:22.145836 14687 raft_consensus.cc:3054] T 7f9818d0493547358fd5654a01d4f456 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.146943 14811 tablet_bootstrap.cc:492] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.147105 14809 consensus_queue.cc:260] T ef0994b1e51b4c688354998730e7b743 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.147411 14809 ts_tablet_manager.cc:1428] T ef0994b1e51b4c688354998730e7b743 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:22.148671 14811 ts_tablet_manager.cc:1397] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20250115 03:06:22.148370 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f9818d0493547358fd5654a01d4f456" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:22.148875 14422 raft_consensus.cc:3054] T 7f9818d0493547358fd5654a01d4f456 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.149663 14809 tablet_bootstrap.cc:492] T 836541bcc82d4644b4c5eb2f3f25c030 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.151067 14809 tablet_bootstrap.cc:654] T 836541bcc82d4644b4c5eb2f3f25c030 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.151250 14811 raft_consensus.cc:357] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.151420 14811 raft_consensus.cc:383] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.151494 14811 raft_consensus.cc:738] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.151645 14811 consensus_queue.cc:260] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.152026 14811 ts_tablet_manager.cc:1428] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:22.152334 14811 tablet_bootstrap.cc:492] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.154131 14811 tablet_bootstrap.cc:654] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.147657 14493 leader_election.cc:304] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770
I20250115 03:06:22.154852 16189 raft_consensus.cc:2798] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:22.154928 16189 raft_consensus.cc:695] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:22.155040 16189 consensus_queue.cc:237] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:22.156198 14493 consensus_peers.cc:487] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 -> Peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Couldn't send request to peer 5c0dae7aff4b44c699275b95ba639770. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20250115 03:06:22.157333 14853 consensus_queue.cc:1035] T 815ef85ce5cf4ae0a56c5734837e03ba P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.157776 14853 consensus_queue.cc:1035] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.158241 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 0580a8f7b2984162b9275d74334b1905
I20250115 03:06:22.158548 14170 catalog_manager.cc:6596] ReplaceTablet: tablet 0580a8f7b2984162b9275d74334b1905 deleted and replaced by tablet a52345cdcd434b1b92f7bc62b90a5a46
I20250115 03:06:22.158926 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 0580a8f7b2984162b9275d74334b1905 with delete_type TABLET_DATA_DELETED (replaced by tablet a52345cdcd434b1b92f7bc62b90a5a46) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.159116 14812 tablet_bootstrap.cc:492] T 5a2894e4736b4191963a61d908bc673c P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.159202 14812 ts_tablet_manager.cc:1397] T 5a2894e4736b4191963a61d908bc673c P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.000s
I20250115 03:06:22.159266 14654 tablet_service.cc:1514] Processing DeleteTablet for tablet 0580a8f7b2984162b9275d74334b1905 with delete_type TABLET_DATA_DELETED (replaced by tablet a52345cdcd434b1b92f7bc62b90a5a46) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:22.159742 14812 raft_consensus.cc:357] T 5a2894e4736b4191963a61d908bc673c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.159842 14812 raft_consensus.cc:383] T 5a2894e4736b4191963a61d908bc673c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.159874 14812 raft_consensus.cc:738] T 5a2894e4736b4191963a61d908bc673c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.159976 14812 consensus_queue.cc:260] T 5a2894e4736b4191963a61d908bc673c P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.160220 14812 ts_tablet_manager.cc:1428] T 5a2894e4736b4191963a61d908bc673c P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.160382 14812 tablet_bootstrap.cc:492] T ca117aca95f640be880b640c85e18cd4 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.160553 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet 0580a8f7b2984162b9275d74334b1905 with delete_type TABLET_DATA_DELETED (replaced by tablet a52345cdcd434b1b92f7bc62b90a5a46) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:22.161302 14812 tablet_bootstrap.cc:654] T ca117aca95f640be880b640c85e18cd4 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.162133 16189 consensus_queue.cc:1035] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.165975 14170 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_7269ada5c0804361a76bdaaad1c12b0c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.166316 14170 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7269ada5c0804361a76bdaaad1c12b0c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.167928 16189 consensus_queue.cc:1035] T 12678da7ffee4a46b4690ac3e94cb19b P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.171530 16558 consensus_queue.cc:1035] T 38f5554b10a44c249e4c3d3825cd7952 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.172343 14688 raft_consensus.cc:1270] T b08e5fc2110e476e897240f9cf3b874d P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:22.173137 14812 tablet_bootstrap.cc:492] T ca117aca95f640be880b640c85e18cd4 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.173228 14812 ts_tablet_manager.cc:1397] T ca117aca95f640be880b640c85e18cd4 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.002s
I20250115 03:06:22.173628 14812 raft_consensus.cc:357] T ca117aca95f640be880b640c85e18cd4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.173713 14812 raft_consensus.cc:383] T ca117aca95f640be880b640c85e18cd4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.173738 14812 raft_consensus.cc:738] T ca117aca95f640be880b640c85e18cd4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.173837 14812 consensus_queue.cc:260] T ca117aca95f640be880b640c85e18cd4 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.173826 16189 consensus_queue.cc:1035] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.174072 14812 ts_tablet_manager.cc:1428] T ca117aca95f640be880b640c85e18cd4 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:22.174081 14858 raft_consensus.cc:491] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.174149 14858 raft_consensus.cc:513] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.174218 14812 tablet_bootstrap.cc:492] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.174422 14858 leader_election.cc:290] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.175081 14812 tablet_bootstrap.cc:654] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.175549 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d891020ed0bb4fc49155bd9f2f6ed35c" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.175665 14689 raft_consensus.cc:2463] T d891020ed0bb4fc49155bd9f2f6ed35c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.175691 14415 raft_consensus.cc:1270] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:22.177114 14858 consensus_queue.cc:1035] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.177791 14823 ts_tablet_manager.cc:1918] T 1016cf85491d4ceda132ee57325b48a4 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:22.177847 14823 log.cc:1198] T 1016cf85491d4ceda132ee57325b48a4 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/1016cf85491d4ceda132ee57325b48a4
I20250115 03:06:22.178015 14823 ts_tablet_manager.cc:1939] T 1016cf85491d4ceda132ee57325b48a4 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
W20250115 03:06:22.177124 14493 leader_election.cc:343] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: d891020ed0bb4fc49155bd9f2f6ed35c
I20250115 03:06:22.179895 14820 tablet_replica.cc:331] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
I20250115 03:06:22.180042 14820 raft_consensus.cc:2238] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:22.180194 14820 pending_rounds.cc:62] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Trying to abort 1 pending ops.
I20250115 03:06:22.180218 14820 pending_rounds.cc:69] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384922696773632 op_type: NO_OP noop_request { }
I20250115 03:06:22.180258 14820 raft_consensus.cc:2883] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:22.180298 14820 raft_consensus.cc:2267] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:22.180742 14820 ts_tablet_manager.cc:1905] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:22.181352 14395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd07196be4b54a90865aecea55493aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.182571 14691 raft_consensus.cc:1270] T 5db9d5f6cecb454a90ad931d8d165bcc P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:22.182777 16001 raft_consensus.cc:2743] T 5db9d5f6cecb454a90ad931d8d165bcc P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.183935 14557 raft_consensus.cc:2463] T ad0e9216bd154f4a8a37874d3d8a08c2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:22.184224 14626 leader_election.cc:304] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:22.184993 16001 raft_consensus.cc:2798] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:22.185086 16001 raft_consensus.cc:695] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:22.185200 16001 consensus_queue.cc:237] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.188143 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd07196be4b54a90865aecea55493aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.189644 14812 tablet_bootstrap.cc:492] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.189718 14812 ts_tablet_manager.cc:1397] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20250115 03:06:22.190018 14418 raft_consensus.cc:2463] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:22.190141 14812 raft_consensus.cc:357] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.190235 14812 raft_consensus.cc:383] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.190265 14812 raft_consensus.cc:738] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.190349 14812 consensus_queue.cc:260] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.190593 14812 ts_tablet_manager.cc:1428] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:22.190750 14812 tablet_bootstrap.cc:492] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.191766 14812 tablet_bootstrap.cc:654] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.192593 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet ccf6b58c55d747da89c1c218a70ce09f (table default.Table_9e42df4c569c47b698c9bbb6dcf39736 [id=e441d6f5acea4837accbfb8e4856f82d]) successfully deleted
I20250115 03:06:22.192823 14493 leader_election.cc:304] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:22.193202 16189 raft_consensus.cc:2798] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.193259 16189 raft_consensus.cc:491] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.193296 16189 raft_consensus.cc:3054] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.199172 14853 consensus_queue.cc:1035] T 5db9d5f6cecb454a90ad931d8d165bcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.201661 16454 tablet.cc:1721] T 5db9d5f6cecb454a90ad931d8d165bcc P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:22.203377 14858 consensus_queue.cc:1035] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.208576 14858 consensus_queue.cc:1035] T 8efa998709034fe9bb37c7c608c99e76 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.209650 16554 tablet.cc:1721] T bfeafbf380df41738ebbcc11c19a7f85 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:22.211460 16555 tablet.cc:1721] T dcdb5282e41c43dbb235ae78465d19f9 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:22.212985 16265 tablet.cc:1721] T b08e5fc2110e476e897240f9cf3b874d P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:22.214255 15617 tablet.cc:1721] T bfeafbf380df41738ebbcc11c19a7f85 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:22.226310 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_c87c8d70caff4b60905ff5c5ae70f4ab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.226650 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c87c8d70caff4b60905ff5c5ae70f4ab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.227651 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_fbb74332af2948bcb49ebad772550bc5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.227955 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fbb74332af2948bcb49ebad772550bc5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.229100 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8f491acf518347ee9b94a67183cfde02" } modify_external_catalogs: true
I20250115 03:06:22.229179 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8f491acf518347ee9b94a67183cfde02" } modify_external_catalogs: true
I20250115 03:06:22.230038 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_58a8eab6cfd443b284d979891ef43040"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:22.230149 16608 tablet.cc:1721] T dcdb5282e41c43dbb235ae78465d19f9 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
W20250115 03:06:22.230320 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_58a8eab6cfd443b284d979891ef43040 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.230504 16606 tablet.cc:1721] T b08e5fc2110e476e897240f9cf3b874d P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:22.233002 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_8bd7e1f9b8fe4c60aea31b8b7bcf4aa8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.233279 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8bd7e1f9b8fe4c60aea31b8b7bcf4aa8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.233762 14853 raft_consensus.cc:491] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.233839 14853 raft_consensus.cc:513] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.234148 14853 leader_election.cc:290] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.234535 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca117aca95f640be880b640c85e18cd4" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.234607 14812 tablet_bootstrap.cc:492] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.234640 14688 raft_consensus.cc:2463] T ca117aca95f640be880b640c85e18cd4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.234674 14812 ts_tablet_manager.cc:1397] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.044s user 0.001s sys 0.000s
I20250115 03:06:22.235082 14812 raft_consensus.cc:357] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.235147 14493 leader_election.cc:304] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:22.235177 14812 raft_consensus.cc:383] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.235205 14812 raft_consensus.cc:738] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.235308 14812 consensus_queue.cc:260] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.235478 14853 raft_consensus.cc:2798] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.235527 14853 raft_consensus.cc:491] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.235563 14853 raft_consensus.cc:3054] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.235584 14812 ts_tablet_manager.cc:1428] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.235750 14812 tablet_bootstrap.cc:492] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:22.235805 14493 leader_election.cc:343] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: ca117aca95f640be880b640c85e18cd4
I20250115 03:06:22.236502 14812 tablet_bootstrap.cc:654] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.237128 16001 raft_consensus.cc:491] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.237211 16001 raft_consensus.cc:513] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.237514 16001 leader_election.cc:290] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.239084 16001 raft_consensus.cc:491] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.239158 16001 raft_consensus.cc:513] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.239449 16001 leader_election.cc:290] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:22.240020 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b5fd91a0ea64b3b85b7571e5daa08b2" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.240087 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b5fd91a0ea64b3b85b7571e5daa08b2" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.240147 14421 raft_consensus.cc:2463] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.240193 14556 raft_consensus.cc:2463] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.240386 14628 leader_election.cc:304] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.240284 14396 tablet_service.cc:1467] Processing CreateTablet for tablet ee1266087cbf44e591798dbd84782db4 (DEFAULT_TABLE table=default.Table_962c421edc794976b03cf360d8e14638 [id=c6bfae1af35c4788acd78658f4736999]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.240475 14668 tablet_service.cc:1467] Processing CreateTablet for tablet ee1266087cbf44e591798dbd84782db4 (DEFAULT_TABLE table=default.Table_962c421edc794976b03cf360d8e14638 [id=c6bfae1af35c4788acd78658f4736999]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.240597 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e41e31ffae4bb092cb08d83eda16b5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.240701 14555 raft_consensus.cc:2463] T a8e41e31ffae4bb092cb08d83eda16b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.240732 16558 raft_consensus.cc:2798] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.240778 16558 raft_consensus.cc:491] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.240810 16558 raft_consensus.cc:3054] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.240890 14418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e41e31ffae4bb092cb08d83eda16b5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.241004 14418 raft_consensus.cc:2463] T a8e41e31ffae4bb092cb08d83eda16b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.241101 14626 leader_election.cc:304] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:22.241432 16001 raft_consensus.cc:2798] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.241477 16001 raft_consensus.cc:491] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.241509 16001 raft_consensus.cc:3054] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.241760 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 7b86e66428504217af08f667f660d304 (table default.Table_341a3cd1c8c94ff4b05fc21960a4e9b4 [id=c096ce170874405ea740b9ea829f886e]). Expected version 2 got 0
I20250115 03:06:22.241894 14167 catalog_manager.cc:5526] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:22.242043 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet dda418ed197e4af2aa511d3f1851663e (table default.Table_5c360e43ddae405b9a2843fc20a9dfb9 [id=cbab87c958a346199acdad65ab75be3c]). Expected version 3 got 0
I20250115 03:06:22.243388 14533 tablet_service.cc:1467] Processing CreateTablet for tablet ee1266087cbf44e591798dbd84782db4 (DEFAULT_TABLE table=default.Table_962c421edc794976b03cf360d8e14638 [id=c6bfae1af35c4788acd78658f4736999]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.243925 14812 tablet_bootstrap.cc:492] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.243997 14812 ts_tablet_manager.cc:1397] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20250115 03:06:22.244323 14812 raft_consensus.cc:357] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.244400 14812 raft_consensus.cc:383] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.244426 14812 raft_consensus.cc:738] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.244544 14812 consensus_queue.cc:260] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.244782 14812 ts_tablet_manager.cc:1428] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.244948 14812 tablet_bootstrap.cc:492] T f47da0e523d14388898a4be9c37bb809 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.245643 14170 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a0e9bf99191f41d6a55b5b32049090e8" } modify_external_catalogs: true
I20250115 03:06:22.245750 14170 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a0e9bf99191f41d6a55b5b32049090e8" } modify_external_catalogs: true
I20250115 03:06:22.246143 14812 tablet_bootstrap.cc:654] T f47da0e523d14388898a4be9c37bb809 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.247656 14162 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_652f8e06e005491381404a11759aefb1" } modify_external_catalogs: true
I20250115 03:06:22.247762 14162 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_652f8e06e005491381404a11759aefb1" } modify_external_catalogs: true
I20250115 03:06:22.248188 16607 tablet.cc:1721] T dcdb5282e41c43dbb235ae78465d19f9 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:22.248432 14809 tablet_bootstrap.cc:492] T 836541bcc82d4644b4c5eb2f3f25c030 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.248507 14809 ts_tablet_manager.cc:1397] T 836541bcc82d4644b4c5eb2f3f25c030 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.099s user 0.002s sys 0.000s
I20250115 03:06:22.251606 14809 raft_consensus.cc:357] T 836541bcc82d4644b4c5eb2f3f25c030 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.251727 14809 raft_consensus.cc:383] T 836541bcc82d4644b4c5eb2f3f25c030 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.251758 14809 raft_consensus.cc:738] T 836541bcc82d4644b4c5eb2f3f25c030 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.251863 14809 consensus_queue.cc:260] T 836541bcc82d4644b4c5eb2f3f25c030 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.252113 14809 ts_tablet_manager.cc:1428] T 836541bcc82d4644b4c5eb2f3f25c030 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:22.252259 14809 tablet_bootstrap.cc:492] T 1202288f2442477badeb35667dd08dab P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.253237 14809 tablet_bootstrap.cc:654] T 1202288f2442477badeb35667dd08dab P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.254045 14811 tablet_bootstrap.cc:492] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.254127 14811 ts_tablet_manager.cc:1397] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.102s user 0.001s sys 0.000s
I20250115 03:06:22.254526 14811 raft_consensus.cc:357] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.254621 14811 raft_consensus.cc:383] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.254652 14811 raft_consensus.cc:738] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.254770 14811 consensus_queue.cc:260] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.255041 14811 ts_tablet_manager.cc:1428] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.255210 14811 tablet_bootstrap.cc:492] T 75c581f11be84e6293a3d14966fd584f P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.255246 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ab72071c412848939b54fcc72d5974fa" } modify_external_catalogs: true
I20250115 03:06:22.255335 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_ab72071c412848939b54fcc72d5974fa" } modify_external_catalogs: true
I20250115 03:06:22.256099 14811 tablet_bootstrap.cc:654] T 75c581f11be84e6293a3d14966fd584f P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.256181 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4bde77af131940a099eabda16fa2ccb2" } modify_external_catalogs: true
I20250115 03:06:22.256271 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4bde77af131940a099eabda16fa2ccb2" } modify_external_catalogs: true
I20250115 03:06:22.257066 14858 raft_consensus.cc:491] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.257143 14858 raft_consensus.cc:513] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.257423 14858 leader_election.cc:290] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.258173 14412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68a04d57feea40eeb8e06cb6a2c61119" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.258278 14412 raft_consensus.cc:2463] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.258533 14493 leader_election.cc:304] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.258540 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68a04d57feea40eeb8e06cb6a2c61119" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.258879 14493 leader_election.cc:343] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.259109 14858 raft_consensus.cc:2798] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.259162 14858 raft_consensus.cc:491] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.259196 14858 raft_consensus.cc:3054] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.261224 16406 raft_consensus.cc:491] T 87b234185fc64505b097f2ae7d52d813 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.261296 16406 raft_consensus.cc:513] T 87b234185fc64505b097f2ae7d52d813 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.261583 16406 leader_election.cc:290] T 87b234185fc64505b097f2ae7d52d813 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.262012 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b234185fc64505b097f2ae7d52d813" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.262264 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b234185fc64505b097f2ae7d52d813" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.262595 14358 leader_election.cc:343] T 87b234185fc64505b097f2ae7d52d813 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.262869 14165 master_service.cc:946] ReplaceTablet: received request to replace tablet fdaf7c090ef642b8b339f61a5d3fb06e from {username='slave'} at 127.0.0.1:60432
W20250115 03:06:22.263603 14356 leader_election.cc:343] T 87b234185fc64505b097f2ae7d52d813 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.263666 14356 leader_election.cc:304] T 87b234185fc64505b097f2ae7d52d813 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.263787 16406 raft_consensus.cc:2743] T 87b234185fc64505b097f2ae7d52d813 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.267544 14165 master_service.cc:946] ReplaceTablet: received request to replace tablet 745344db22da4102bad39c2f121548a1 from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:22.267630 16613 tablet.cc:1721] T b08e5fc2110e476e897240f9cf3b874d P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:22.268409 15153 raft_consensus.cc:491] T 6bd4e4200f9b489ea5dd2154aa6f382d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.268494 15153 raft_consensus.cc:513] T 6bd4e4200f9b489ea5dd2154aa6f382d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:22.268533 14171 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dcc1c999a9d34006a9bd4ec2bec02bdd consensus state has no leader
I20250115 03:06:22.268790 15153 leader_election.cc:290] T 6bd4e4200f9b489ea5dd2154aa6f382d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.268779 14171 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_5c360e43ddae405b9a2843fc20a9dfb9" } new_table_name: "default.Table_f92234da7e9c4042a36f7acbd9c276d5" modify_external_catalogs: true
I20250115 03:06:22.269883 14412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bd4e4200f9b489ea5dd2154aa6f382d" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.270005 14412 raft_consensus.cc:2388] T 6bd4e4200f9b489ea5dd2154aa6f382d P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c0dae7aff4b44c699275b95ba639770 in current term 1: Already voted for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in this term.
I20250115 03:06:22.273747 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6bd4e4200f9b489ea5dd2154aa6f382d" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.274989 14626 leader_election.cc:304] T 6bd4e4200f9b489ea5dd2154aa6f382d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:22.275359 15153 raft_consensus.cc:3054] T 6bd4e4200f9b489ea5dd2154aa6f382d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.275406 14169 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_ca82711024824acda4e6b4d3cf4f0537 [id=07019baad4a74fd0b468bbd9b60d54c8] alter complete (version 2)
I20250115 03:06:22.275449 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet e7fb6422afb44ec88db538b74ce74175
I20250115 03:06:22.275851 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2c71ee9d4f4d414997b2c7424816b1a8" } new_table_name: "default.Table_a600f64377344f308e32d18c3d865eba" modify_external_catalogs: true
I20250115 03:06:22.276975 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet e7fb6422afb44ec88db538b74ce74175 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.277587 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet cef4442bfcfd4ad4b343a2785c3a9a74 with delete_type TABLET_DATA_DELETED (replaced by tablet c164db2cf23149258bdc14e68def00b3) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.277762 14170 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 9476eae94d664f5eb94865db2c95dca6
I20250115 03:06:22.277781 16088 raft_consensus.cc:513] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.278087 16088 leader_election.cc:290] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.278266 14654 tablet_service.cc:1514] Processing DeleteTablet for tablet e7fb6422afb44ec88db538b74ce74175 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:22.278676 14166 catalog_manager.cc:5618] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215) does not have the latest schema for tablet 6bd4e4200f9b489ea5dd2154aa6f382d (table default.Table_83b1fd83225a4454b29b7c61bc4639d4 [id=299ac71cfb98480f873eb659419e522c]). Expected version 1 got 0
I20250115 03:06:22.278837 14162 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 7f9818d0493547358fd5654a01d4f456
I20250115 03:06:22.279615 14420 raft_consensus.cc:2463] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:22.280437 14401 tablet_service.cc:1514] Processing DeleteTablet for tablet 9476eae94d664f5eb94865db2c95dca6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.280429 14671 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f9818d0493547358fd5654a01d4f456 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:22.280603 14493 leader_election.cc:304] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770
I20250115 03:06:22.281344 14809 tablet_bootstrap.cc:492] T 1202288f2442477badeb35667dd08dab P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.281424 14809 ts_tablet_manager.cc:1397] T 1202288f2442477badeb35667dd08dab P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20250115 03:06:22.281916 14809 raft_consensus.cc:357] T 1202288f2442477badeb35667dd08dab P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.282001 14809 raft_consensus.cc:383] T 1202288f2442477badeb35667dd08dab P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.282025 14809 raft_consensus.cc:738] T 1202288f2442477badeb35667dd08dab P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.282114 14809 consensus_queue.cc:260] T 1202288f2442477badeb35667dd08dab P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.282367 14809 ts_tablet_manager.cc:1428] T 1202288f2442477badeb35667dd08dab P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.282538 14809 tablet_bootstrap.cc:492] T b3334f58b90b4947af52150546ccaa28 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.283480 14809 tablet_bootstrap.cc:654] T b3334f58b90b4947af52150546ccaa28 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.284364 16406 raft_consensus.cc:491] T 836541bcc82d4644b4c5eb2f3f25c030 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.284430 16406 raft_consensus.cc:513] T 836541bcc82d4644b4c5eb2f3f25c030 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.284718 16406 leader_election.cc:290] T 836541bcc82d4644b4c5eb2f3f25c030 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.286217 14811 tablet_bootstrap.cc:492] T 75c581f11be84e6293a3d14966fd584f P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.286298 14811 ts_tablet_manager.cc:1397] T 75c581f11be84e6293a3d14966fd584f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.031s user 0.001s sys 0.000s
I20250115 03:06:22.286372 14394 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f9818d0493547358fd5654a01d4f456 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.286731 14811 raft_consensus.cc:357] T 75c581f11be84e6293a3d14966fd584f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.286823 14811 raft_consensus.cc:383] T 75c581f11be84e6293a3d14966fd584f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.286849 14811 raft_consensus.cc:738] T 75c581f11be84e6293a3d14966fd584f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.286909 16571 tablet.cc:1721] T bfeafbf380df41738ebbcc11c19a7f85 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20250115 03:06:22.286943 14811 consensus_queue.cc:260] T 75c581f11be84e6293a3d14966fd584f P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.287263 14811 ts_tablet_manager.cc:1428] T 75c581f11be84e6293a3d14966fd584f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.287442 14811 tablet_bootstrap.cc:492] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.288534 14811 tablet_bootstrap.cc:654] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.290982 14812 tablet_bootstrap.cc:492] T f47da0e523d14388898a4be9c37bb809 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.291054 14812 ts_tablet_manager.cc:1397] T f47da0e523d14388898a4be9c37bb809 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.046s user 0.001s sys 0.000s
I20250115 03:06:22.291422 14812 raft_consensus.cc:357] T f47da0e523d14388898a4be9c37bb809 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.291512 14812 raft_consensus.cc:383] T f47da0e523d14388898a4be9c37bb809 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.291535 14812 raft_consensus.cc:738] T f47da0e523d14388898a4be9c37bb809 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.291626 14812 consensus_queue.cc:260] T f47da0e523d14388898a4be9c37bb809 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.291893 14812 ts_tablet_manager.cc:1428] T f47da0e523d14388898a4be9c37bb809 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:22.292048 14812 tablet_bootstrap.cc:492] T 68a04d57feea40eeb8e06cb6a2c61119 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.292796 14812 tablet_bootstrap.cc:654] T 68a04d57feea40eeb8e06cb6a2c61119 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.293155 16189 raft_consensus.cc:513] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.293488 16189 leader_election.cc:290] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.293749 16189 raft_consensus.cc:2798] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:22.293817 16189 raft_consensus.cc:695] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:22.293932 14533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee1266087cbf44e591798dbd84782db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.294092 14819 ts_tablet_manager.cc:1918] T 6132d15b17ab4d0aaa10a7ea04cda7ac P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:22.294148 14819 log.cc:1198] T 6132d15b17ab4d0aaa10a7ea04cda7ac P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/6132d15b17ab4d0aaa10a7ea04cda7ac
I20250115 03:06:22.294346 14819 ts_tablet_manager.cc:1939] T 6132d15b17ab4d0aaa10a7ea04cda7ac P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:22.296133 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72abf6acd9d54f3b983b7098aaf2691d" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:22.296233 14420 raft_consensus.cc:3054] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.297045 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee1266087cbf44e591798dbd84782db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.297773 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0de3c93a1d034957a27236437dc2fbca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:22.293922 16189 consensus_queue.cc:237] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
W20250115 03:06:22.298019 14493 leader_election.cc:343] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: d891020ed0bb4fc49155bd9f2f6ed35c
W20250115 03:06:22.298139 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0de3c93a1d034957a27236437dc2fbca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.298990 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d891020ed0bb4fc49155bd9f2f6ed35c" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:22.299086 14690 raft_consensus.cc:3054] T d891020ed0bb4fc49155bd9f2f6ed35c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.299799 14654 tablet_service.cc:1514] Processing DeleteTablet for tablet 9476eae94d664f5eb94865db2c95dca6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:22.299978 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836541bcc82d4644b4c5eb2f3f25c030" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.300282 14358 leader_election.cc:343] T 836541bcc82d4644b4c5eb2f3f25c030 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.300436 14535 tablet_service.cc:1514] Processing DeleteTablet for tablet e7fb6422afb44ec88db538b74ce74175 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:22.300632 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72abf6acd9d54f3b983b7098aaf2691d" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:22.300738 14552 raft_consensus.cc:3054] T 72abf6acd9d54f3b983b7098aaf2691d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.300853 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd07196be4b54a90865aecea55493aa1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.301002 14536 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f9818d0493547358fd5654a01d4f456 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:22.301498 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet 9476eae94d664f5eb94865db2c95dca6 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:22.304191 14811 tablet_bootstrap.cc:492] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.304275 14811 ts_tablet_manager.cc:1397] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20250115 03:06:22.304687 14811 raft_consensus.cc:357] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.304785 14811 raft_consensus.cc:383] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.304812 14811 raft_consensus.cc:738] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.304914 14811 consensus_queue.cc:260] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.305194 14811 ts_tablet_manager.cc:1428] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:22.305331 14811 tablet_bootstrap.cc:492] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.306269 14811 tablet_bootstrap.cc:654] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.307776 14812 tablet_bootstrap.cc:492] T 68a04d57feea40eeb8e06cb6a2c61119 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.307860 14812 ts_tablet_manager.cc:1397] T 68a04d57feea40eeb8e06cb6a2c61119 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20250115 03:06:22.308286 14812 raft_consensus.cc:357] T 68a04d57feea40eeb8e06cb6a2c61119 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.308382 14812 raft_consensus.cc:383] T 68a04d57feea40eeb8e06cb6a2c61119 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.308410 14812 raft_consensus.cc:738] T 68a04d57feea40eeb8e06cb6a2c61119 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.308518 14812 consensus_queue.cc:260] T 68a04d57feea40eeb8e06cb6a2c61119 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.308789 14812 ts_tablet_manager.cc:1428] T 68a04d57feea40eeb8e06cb6a2c61119 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.308981 14812 tablet_bootstrap.cc:492] T 103f78d0d9fc42308e1e23316895be33 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:22.309026 14163 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 16f3790cbec44f15ba1ccf84568c2296 consensus state has no leader
I20250115 03:06:22.309913 14812 tablet_bootstrap.cc:654] T 103f78d0d9fc42308e1e23316895be33 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.310295 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet ec1a8d0326f84ca0a87851010c1e68b7
I20250115 03:06:22.311055 14809 tablet_bootstrap.cc:492] T b3334f58b90b4947af52150546ccaa28 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.311154 14809 ts_tablet_manager.cc:1397] T b3334f58b90b4947af52150546ccaa28 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.000s
I20250115 03:06:22.311378 14388 tablet_service.cc:1514] Processing DeleteTablet for tablet ec1a8d0326f84ca0a87851010c1e68b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.311589 14809 raft_consensus.cc:357] T b3334f58b90b4947af52150546ccaa28 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.311679 14809 raft_consensus.cc:383] T b3334f58b90b4947af52150546ccaa28 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.311717 14809 raft_consensus.cc:738] T b3334f58b90b4947af52150546ccaa28 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.311821 14809 consensus_queue.cc:260] T b3334f58b90b4947af52150546ccaa28 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.312062 14809 ts_tablet_manager.cc:1428] T b3334f58b90b4947af52150546ccaa28 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.312237 14809 tablet_bootstrap.cc:492] T 60997f625d074960ad75aef5ba14b9c2 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.313083 16406 raft_consensus.cc:491] T d5e4743e016e49ed8ba26b1ac96b1470 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.313154 16406 raft_consensus.cc:513] T d5e4743e016e49ed8ba26b1ac96b1470 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.313472 16406 leader_election.cc:290] T d5e4743e016e49ed8ba26b1ac96b1470 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.314030 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e4743e016e49ed8ba26b1ac96b1470" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:22.314544 14356 leader_election.cc:343] T d5e4743e016e49ed8ba26b1ac96b1470 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.314538 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e4743e016e49ed8ba26b1ac96b1470" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.314898 14358 leader_election.cc:343] T d5e4743e016e49ed8ba26b1ac96b1470 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.314956 14358 leader_election.cc:304] T d5e4743e016e49ed8ba26b1ac96b1470 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.315500 16406 raft_consensus.cc:2743] T d5e4743e016e49ed8ba26b1ac96b1470 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.316749 14812 tablet_bootstrap.cc:492] T 103f78d0d9fc42308e1e23316895be33 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.316828 14812 ts_tablet_manager.cc:1397] T 103f78d0d9fc42308e1e23316895be33 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20250115 03:06:22.317202 14654 tablet_service.cc:1514] Processing DeleteTablet for tablet ec1a8d0326f84ca0a87851010c1e68b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:22.318221 14521 tablet_service.cc:1514] Processing DeleteTablet for tablet ec1a8d0326f84ca0a87851010c1e68b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:22.319470 16557 raft_consensus.cc:491] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.319540 16557 raft_consensus.cc:513] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.319864 16557 leader_election.cc:290] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.321488 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836541bcc82d4644b4c5eb2f3f25c030" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:22.321610 14628 leader_election.cc:343] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 398b93f1db0741a9b54837bca43b1bad
W20250115 03:06:22.321846 14356 leader_election.cc:343] T 836541bcc82d4644b4c5eb2f3f25c030 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.321908 14356 leader_election.cc:304] T 836541bcc82d4644b4c5eb2f3f25c030 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.321910 14162 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 7b86e66428504217af08f667f660d304 (table default.Table_341a3cd1c8c94ff4b05fc21960a4e9b4 [id=c096ce170874405ea740b9ea829f886e]). Expected version 2 got 0
I20250115 03:06:22.322124 14162 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 836541bcc82d4644b4c5eb2f3f25c030 (table default.Table_e7b6ace2c7c04dbfa9e87c6a4697a243 [id=5159afd6260c4871a21e6b2442795e9b]). Expected version 1 got 0
I20250115 03:06:22.322238 14162 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 8efa998709034fe9bb37c7c608c99e76 (table default.Table_b6002fb5cc1248bba2eb7abbd5a5d407 [id=4c458a80b1a2490da1cf689843cae584]). Expected version 1 got 0
I20250115 03:06:22.324263 14654 tablet_service.cc:1467] Processing CreateTablet for tablet a3db354a510143e08f07b267aec1dd5e (DEFAULT_TABLE table=default.Table_d120fbdbf5d746d287477050fdc9959c [id=644cda8ff2c4403db58208428d11d1be]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.324975 16406 raft_consensus.cc:2743] T 836541bcc82d4644b4c5eb2f3f25c030 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.325017 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "398b93f1db0741a9b54837bca43b1bad" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.325121 14553 raft_consensus.cc:2463] T 398b93f1db0741a9b54837bca43b1bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.325181 14812 raft_consensus.cc:357] T 103f78d0d9fc42308e1e23316895be33 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.325343 14812 raft_consensus.cc:383] T 103f78d0d9fc42308e1e23316895be33 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.325407 14812 raft_consensus.cc:738] T 103f78d0d9fc42308e1e23316895be33 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.325582 14812 consensus_queue.cc:260] T 103f78d0d9fc42308e1e23316895be33 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.326165 14812 ts_tablet_manager.cc:1428] T 103f78d0d9fc42308e1e23316895be33 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.009s user 0.000s sys 0.001s
I20250115 03:06:22.326201 14388 tablet_service.cc:1467] Processing CreateTablet for tablet a3db354a510143e08f07b267aec1dd5e (DEFAULT_TABLE table=default.Table_d120fbdbf5d746d287477050fdc9959c [id=644cda8ff2c4403db58208428d11d1be]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.326359 14812 tablet_bootstrap.cc:492] T 75c581f11be84e6293a3d14966fd584f P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.326637 14521 tablet_service.cc:1467] Processing CreateTablet for tablet a3db354a510143e08f07b267aec1dd5e (DEFAULT_TABLE table=default.Table_d120fbdbf5d746d287477050fdc9959c [id=644cda8ff2c4403db58208428d11d1be]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.326774 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 707b29d359ff496a948d3a73cc7eb583 (DEFAULT_TABLE table=default.Table_c88e706cbc9d4c91bbb662eede011de6 [id=438161f3be1242889b15d5d0eaf8a2ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.326968 14671 tablet_service.cc:1467] Processing CreateTablet for tablet 707b29d359ff496a948d3a73cc7eb583 (DEFAULT_TABLE table=default.Table_c88e706cbc9d4c91bbb662eede011de6 [id=438161f3be1242889b15d5d0eaf8a2ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.325621 14626 leader_election.cc:304] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:22.327909 16557 raft_consensus.cc:2798] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.327965 16557 raft_consensus.cc:491] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.328011 16557 raft_consensus.cc:3054] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.328469 14812 tablet_bootstrap.cc:654] T 75c581f11be84e6293a3d14966fd584f P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.329187 14809 tablet_bootstrap.cc:654] T 60997f625d074960ad75aef5ba14b9c2 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.329581 14536 tablet_service.cc:1467] Processing CreateTablet for tablet 707b29d359ff496a948d3a73cc7eb583 (DEFAULT_TABLE table=default.Table_c88e706cbc9d4c91bbb662eede011de6 [id=438161f3be1242889b15d5d0eaf8a2ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.330078 14394 tablet_service.cc:1467] Processing CreateTablet for tablet 824fe68c383d4483991841a9f4c1b4ab (DEFAULT_TABLE table=default.Table_36648b7bea684eecb64fad0ae41e7b53 [id=0f6a5506404243d0a16d7c643e25c730]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.330377 14401 tablet_service.cc:1467] Processing CreateTablet for tablet a52345cdcd434b1b92f7bc62b90a5a46 (DEFAULT_TABLE table=default.Table_ef9433c3090f4deea9bf840e58e460e5 [id=0790b08615854d3688f940aba20b6520]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.330451 14162 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_6b6164bc5f2e4395b5d31047b8ef87d9 [id=f8f2a7ff6fdc40468de6ae9ba28c1427] alter complete (version 4)
I20250115 03:06:22.330727 14663 tablet_service.cc:1467] Processing CreateTablet for tablet 824fe68c383d4483991841a9f4c1b4ab (DEFAULT_TABLE table=default.Table_36648b7bea684eecb64fad0ae41e7b53 [id=0f6a5506404243d0a16d7c643e25c730]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.331024 14667 tablet_service.cc:1467] Processing CreateTablet for tablet a52345cdcd434b1b92f7bc62b90a5a46 (DEFAULT_TABLE table=default.Table_ef9433c3090f4deea9bf840e58e460e5 [id=0790b08615854d3688f940aba20b6520]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.331326 14661 tablet_service.cc:1467] Processing CreateTablet for tablet fcf46748ff3f43a1b74ab505b4c1e9e1 (DEFAULT_TABLE table=default.Table_b7bf51cb55344d7083f4fda2cfa2e1fd [id=9934f2543427487a86ba56c0d24e2ce9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.331756 14393 tablet_service.cc:1467] Processing CreateTablet for tablet fcf46748ff3f43a1b74ab505b4c1e9e1 (DEFAULT_TABLE table=default.Table_b7bf51cb55344d7083f4fda2cfa2e1fd [id=9934f2543427487a86ba56c0d24e2ce9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.333477 14809 tablet_bootstrap.cc:492] T 60997f625d074960ad75aef5ba14b9c2 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.333560 14809 ts_tablet_manager.cc:1397] T 60997f625d074960ad75aef5ba14b9c2 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.000s
I20250115 03:06:22.333988 14809 raft_consensus.cc:357] T 60997f625d074960ad75aef5ba14b9c2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.334090 14809 raft_consensus.cc:383] T 60997f625d074960ad75aef5ba14b9c2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.334123 14809 raft_consensus.cc:738] T 60997f625d074960ad75aef5ba14b9c2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.334254 14809 consensus_queue.cc:260] T 60997f625d074960ad75aef5ba14b9c2 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.334611 14809 ts_tablet_manager.cc:1428] T 60997f625d074960ad75aef5ba14b9c2 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.334772 14809 tablet_bootstrap.cc:492] T bf6199cdd6e940d1970dd43d8090100d P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.335568 14162 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_660872e60c104e56be9287f66d74c387 [id=03ee00da1e024ebab51b3a366d2c0cf3] alter complete (version 2)
I20250115 03:06:22.335641 14809 tablet_bootstrap.cc:654] T bf6199cdd6e940d1970dd43d8090100d P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.336169 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet ef0994b1e51b4c688354998730e7b743 with delete_type TABLET_DATA_DELETED (replaced by tablet db3c5684202e452b92f883d9a45f6e97) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.336172 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 678b4453675b42ab98832a20f1b0271b
I20250115 03:06:22.336673 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 678b4453675b42ab98832a20f1b0271b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.337080 14166 master_service.cc:946] ReplaceTablet: received request to replace tablet e9725e15494e4a988db499414b94f84a from {username='slave'} at 127.0.0.1:60432
I20250115 03:06:22.337870 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 745344db22da4102bad39c2f121548a1
I20250115 03:06:22.338181 14165 catalog_manager.cc:6596] ReplaceTablet: tablet 745344db22da4102bad39c2f121548a1 deleted and replaced by tablet 9099a50c9e9f487dba708e5cbdff329c
I20250115 03:06:22.338359 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 678b4453675b42ab98832a20f1b0271b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:22.338871 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 745344db22da4102bad39c2f121548a1 with delete_type TABLET_DATA_DELETED (replaced by tablet 9099a50c9e9f487dba708e5cbdff329c) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.338992 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet 745344db22da4102bad39c2f121548a1 with delete_type TABLET_DATA_DELETED (replaced by tablet 9099a50c9e9f487dba708e5cbdff329c) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:22.339572 14811 tablet_bootstrap.cc:492] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.339658 14811 ts_tablet_manager.cc:1397] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.034s user 0.000s sys 0.001s
I20250115 03:06:22.340085 14811 raft_consensus.cc:357] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.340194 14811 raft_consensus.cc:383] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.340229 14811 raft_consensus.cc:738] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.340325 14811 consensus_queue.cc:260] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.340586 14811 ts_tablet_manager.cc:1428] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20250115 03:06:22.340765 14811 tablet_bootstrap.cc:492] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.341065 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f617d2044baa48a383e6d59fb6123ba0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.341544 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f617d2044baa48a383e6d59fb6123ba0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.341674 14811 tablet_bootstrap.cc:654] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.344290 16292 raft_consensus.cc:491] T 5a2894e4736b4191963a61d908bc673c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.344372 16292 raft_consensus.cc:513] T 5a2894e4736b4191963a61d908bc673c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.344679 16292 leader_election.cc:290] T 5a2894e4736b4191963a61d908bc673c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.345249 14809 tablet_bootstrap.cc:492] T bf6199cdd6e940d1970dd43d8090100d P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.345336 14809 ts_tablet_manager.cc:1397] T bf6199cdd6e940d1970dd43d8090100d P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20250115 03:06:22.345770 14809 raft_consensus.cc:357] T bf6199cdd6e940d1970dd43d8090100d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.345867 14809 raft_consensus.cc:383] T bf6199cdd6e940d1970dd43d8090100d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.345899 14809 raft_consensus.cc:738] T bf6199cdd6e940d1970dd43d8090100d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.345959 14812 tablet_bootstrap.cc:492] T 75c581f11be84e6293a3d14966fd584f P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.346038 14812 ts_tablet_manager.cc:1397] T 75c581f11be84e6293a3d14966fd584f P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.002s
I20250115 03:06:22.346001 14809 consensus_queue.cc:260] T bf6199cdd6e940d1970dd43d8090100d P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.346290 14809 ts_tablet_manager.cc:1428] T bf6199cdd6e940d1970dd43d8090100d P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.346477 14809 tablet_bootstrap.cc:492] T d643bd0947c54bcfa79f15d88e1e53e4 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.346900 14412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a2894e4736b4191963a61d908bc673c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.347399 14809 tablet_bootstrap.cc:654] T d643bd0947c54bcfa79f15d88e1e53e4 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.347437 14412 raft_consensus.cc:2388] T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c0dae7aff4b44c699275b95ba639770 in current term 1: Already voted for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in this term.
I20250115 03:06:22.347528 14812 raft_consensus.cc:357] T 75c581f11be84e6293a3d14966fd584f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.347836 14812 raft_consensus.cc:383] T 75c581f11be84e6293a3d14966fd584f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.347885 14812 raft_consensus.cc:738] T 75c581f11be84e6293a3d14966fd584f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.348011 14812 consensus_queue.cc:260] T 75c581f11be84e6293a3d14966fd584f P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.348369 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a2894e4736b4191963a61d908bc673c" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.348726 14534 tablet_service.cc:1467] Processing CreateTablet for tablet a52345cdcd434b1b92f7bc62b90a5a46 (DEFAULT_TABLE table=default.Table_ef9433c3090f4deea9bf840e58e460e5 [id=0790b08615854d3688f940aba20b6520]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.348809 14812 ts_tablet_manager.cc:1428] T 75c581f11be84e6293a3d14966fd584f P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20250115 03:06:22.349098 14812 tablet_bootstrap.cc:492] T 0c580cdf3a854df988df32a9294c83bc P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.349126 14535 tablet_service.cc:1467] Processing CreateTablet for tablet 824fe68c383d4483991841a9f4c1b4ab (DEFAULT_TABLE table=default.Table_36648b7bea684eecb64fad0ae41e7b53 [id=0f6a5506404243d0a16d7c643e25c730]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.349381 14527 tablet_service.cc:1467] Processing CreateTablet for tablet fcf46748ff3f43a1b74ab505b4c1e9e1 (DEFAULT_TABLE table=default.Table_b7bf51cb55344d7083f4fda2cfa2e1fd [id=9934f2543427487a86ba56c0d24e2ce9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.349699 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet 678b4453675b42ab98832a20f1b0271b with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:22.349819 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet 745344db22da4102bad39c2f121548a1 with delete_type TABLET_DATA_DELETED (replaced by tablet 9099a50c9e9f487dba708e5cbdff329c) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:22.349814 14626 leader_election.cc:304] T 5a2894e4736b4191963a61d908bc673c P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:22.349953 16292 raft_consensus.cc:3054] T 5a2894e4736b4191963a61d908bc673c P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.350207 14812 tablet_bootstrap.cc:654] T 0c580cdf3a854df988df32a9294c83bc P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.351243 14812 tablet_bootstrap.cc:492] T 0c580cdf3a854df988df32a9294c83bc P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.351374 14812 ts_tablet_manager.cc:1397] T 0c580cdf3a854df988df32a9294c83bc P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20250115 03:06:22.351812 14812 raft_consensus.cc:357] T 0c580cdf3a854df988df32a9294c83bc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.351958 14812 raft_consensus.cc:383] T 0c580cdf3a854df988df32a9294c83bc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.352025 14812 raft_consensus.cc:738] T 0c580cdf3a854df988df32a9294c83bc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.352169 14812 consensus_queue.cc:260] T 0c580cdf3a854df988df32a9294c83bc P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.352473 14809 tablet_bootstrap.cc:492] T d643bd0947c54bcfa79f15d88e1e53e4 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.352547 14809 ts_tablet_manager.cc:1397] T d643bd0947c54bcfa79f15d88e1e53e4 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:22.352916 14809 raft_consensus.cc:357] T d643bd0947c54bcfa79f15d88e1e53e4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.353044 14809 raft_consensus.cc:383] T d643bd0947c54bcfa79f15d88e1e53e4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.353082 14809 raft_consensus.cc:738] T d643bd0947c54bcfa79f15d88e1e53e4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.353194 14809 consensus_queue.cc:260] T d643bd0947c54bcfa79f15d88e1e53e4 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.353305 14812 ts_tablet_manager.cc:1428] T 0c580cdf3a854df988df32a9294c83bc P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:22.353458 14809 ts_tablet_manager.cc:1428] T d643bd0947c54bcfa79f15d88e1e53e4 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.353629 14809 tablet_bootstrap.cc:492] T fdaf7c090ef642b8b339f61a5d3fb06e P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.354094 14812 tablet_bootstrap.cc:492] T f72c9bf04a3942ceacc37fd407bff033 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.354394 14166 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet e9725e15494e4a988db499414b94f84a
I20250115 03:06:22.354504 14809 tablet_bootstrap.cc:654] T fdaf7c090ef642b8b339f61a5d3fb06e P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.354812 14166 catalog_manager.cc:6596] ReplaceTablet: tablet e9725e15494e4a988db499414b94f84a deleted and replaced by tablet ea1ff51dc99e4cf5aa310cbaa5cb16c0
I20250115 03:06:22.355122 14811 tablet_bootstrap.cc:492] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.355203 14811 ts_tablet_manager.cc:1397] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20250115 03:06:22.355597 14811 raft_consensus.cc:357] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.355713 14811 raft_consensus.cc:383] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.355748 14811 raft_consensus.cc:738] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.355871 14811 consensus_queue.cc:260] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.356132 14811 ts_tablet_manager.cc:1428] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.356403 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet e9725e15494e4a988db499414b94f84a with delete_type TABLET_DATA_DELETED (replaced by tablet ea1ff51dc99e4cf5aa310cbaa5cb16c0) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.356441 14664 tablet_service.cc:1514] Processing DeleteTablet for tablet e9725e15494e4a988db499414b94f84a with delete_type TABLET_DATA_DELETED (replaced by tablet ea1ff51dc99e4cf5aa310cbaa5cb16c0) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:22.357568 14811 tablet_bootstrap.cc:492] T f72c9bf04a3942ceacc37fd407bff033 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.358525 16411 raft_consensus.cc:491] T b3334f58b90b4947af52150546ccaa28 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.358592 16411 raft_consensus.cc:513] T b3334f58b90b4947af52150546ccaa28 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.359395 14812 tablet_bootstrap.cc:654] T f72c9bf04a3942ceacc37fd407bff033 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.359386 16411 leader_election.cc:290] T b3334f58b90b4947af52150546ccaa28 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.359434 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3334f58b90b4947af52150546ccaa28" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.359784 14358 leader_election.cc:343] T b3334f58b90b4947af52150546ccaa28 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.361497 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3334f58b90b4947af52150546ccaa28" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:22.361753 14356 leader_election.cc:343] T b3334f58b90b4947af52150546ccaa28 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.361816 14811 tablet_bootstrap.cc:654] T f72c9bf04a3942ceacc37fd407bff033 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.361820 14356 leader_election.cc:304] T b3334f58b90b4947af52150546ccaa28 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.361987 16411 raft_consensus.cc:2743] T b3334f58b90b4947af52150546ccaa28 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.362901 14811 tablet_bootstrap.cc:492] T f72c9bf04a3942ceacc37fd407bff033 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.363018 14811 ts_tablet_manager.cc:1397] T f72c9bf04a3942ceacc37fd407bff033 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
W20250115 03:06:22.364881 16535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250115 03:06:22.365227 16535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250115 03:06:22.365283 16535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20250115 03:06:22.366099 16411 raft_consensus.cc:491] T 1202288f2442477badeb35667dd08dab P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.366173 16411 raft_consensus.cc:513] T 1202288f2442477badeb35667dd08dab P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.366454 16411 leader_election.cc:290] T 1202288f2442477badeb35667dd08dab P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.363237 14530 tablet_service.cc:1514] Processing DeleteTablet for tablet b61cc4b11251486a8089ca9bf44ac91e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:22.363781 14523 tablet_service.cc:1514] Processing DeleteTablet for tablet ca117aca95f640be880b640c85e18cd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:17 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:22.367394 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1202288f2442477badeb35667dd08dab" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.367439 14811 raft_consensus.cc:357] T f72c9bf04a3942ceacc37fd407bff033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
W20250115 03:06:22.367707 14356 leader_election.cc:343] T 1202288f2442477badeb35667dd08dab P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.367532 14811 raft_consensus.cc:383] T f72c9bf04a3942ceacc37fd407bff033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.367823 14811 raft_consensus.cc:738] T f72c9bf04a3942ceacc37fd407bff033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.367959 14811 consensus_queue.cc:260] T f72c9bf04a3942ceacc37fd407bff033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.368350 14532 tablet_service.cc:1514] Processing DeleteTablet for tablet e9725e15494e4a988db499414b94f84a with delete_type TABLET_DATA_DELETED (replaced by tablet ea1ff51dc99e4cf5aa310cbaa5cb16c0) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:22.368487 14526 tablet_service.cc:1514] Processing DeleteTablet for tablet 103f78d0d9fc42308e1e23316895be33 with delete_type TABLET_DATA_DELETED (replaced by tablet 9c436975bd034068a6d58d5831db1896) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:22.369114 16088 raft_consensus.cc:491] T 6ed47b26cdf744acb28fb657a1b41087 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.369199 16088 raft_consensus.cc:513] T 6ed47b26cdf744acb28fb657a1b41087 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.369503 16088 leader_election.cc:290] T 6ed47b26cdf744acb28fb657a1b41087 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.369900 14412 raft_consensus.cc:1270] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:22.369954 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1202288f2442477badeb35667dd08dab" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.370185 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ed47b26cdf744acb28fb657a1b41087" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.370288 14557 raft_consensus.cc:2463] T 6ed47b26cdf744acb28fb657a1b41087 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.370369 14811 ts_tablet_manager.cc:1428] T f72c9bf04a3942ceacc37fd407bff033 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
W20250115 03:06:22.370478 14628 leader_election.cc:343] T 6ed47b26cdf744acb28fb657a1b41087 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 6ed47b26cdf744acb28fb657a1b41087
W20250115 03:06:22.370627 16535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250115 03:06:22.370633 14358 leader_election.cc:343] T 1202288f2442477badeb35667dd08dab P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.370692 14358 leader_election.cc:304] T 1202288f2442477badeb35667dd08dab P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.370792 14626 leader_election.cc:304] T 6ed47b26cdf744acb28fb657a1b41087 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:22.370858 16411 raft_consensus.cc:2743] T 1202288f2442477badeb35667dd08dab P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:22.371003 16535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250115 03:06:22.371043 16535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250115 03:06:22.371096 16535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250115 03:06:22.371127 16088 raft_consensus.cc:2798] T 6ed47b26cdf744acb28fb657a1b41087 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.371181 16088 raft_consensus.cc:491] T 6ed47b26cdf744acb28fb657a1b41087 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.371204 14811 tablet_bootstrap.cc:492] T a8a5743475814401ac2c0a51cfd70207 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.371217 16088 raft_consensus.cc:3054] T 6ed47b26cdf744acb28fb657a1b41087 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.372097 14811 tablet_bootstrap.cc:654] T a8a5743475814401ac2c0a51cfd70207 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.372407 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 1202288f2442477badeb35667dd08dab (table default.Table_c0e04439c1b240b9a233912aeb3e71c1 [id=8b4cf3ac45024f328efaa975fd6c6014]). Expected version 1 got 0
I20250115 03:06:22.372608 16620 consensus_queue.cc:1035] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.373968 14664 tablet_service.cc:1467] Processing CreateTablet for tablet 1e4b273297134df094d9ad90a08e6606 (DEFAULT_TABLE table=default.Table_0de3c93a1d034957a27236437dc2fbca [id=61e83ea87623447db5bc05654bd173e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.373972 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 1e4b273297134df094d9ad90a08e6606 (DEFAULT_TABLE table=default.Table_0de3c93a1d034957a27236437dc2fbca [id=61e83ea87623447db5bc05654bd173e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.374022 14526 tablet_service.cc:1467] Processing CreateTablet for tablet 1e4b273297134df094d9ad90a08e6606 (DEFAULT_TABLE table=default.Table_0de3c93a1d034957a27236437dc2fbca [id=61e83ea87623447db5bc05654bd173e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.374948 14390 tablet_service.cc:1514] Processing DeleteTablet for tablet b3334f58b90b4947af52150546ccaa28 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.375080 14669 tablet_service.cc:1467] Processing CreateTablet for tablet 62f5e9bed95648ab92233e6f979d9398 (DEFAULT_TABLE table=default.Table_8bd7e1f9b8fe4c60aea31b8b7bcf4aa8 [id=4824353236b849c4beb127506a7ec5e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.375113 14391 tablet_service.cc:1514] Processing DeleteTablet for tablet 60997f625d074960ad75aef5ba14b9c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.375698 14391 tablet_service.cc:1467] Processing CreateTablet for tablet 62f5e9bed95648ab92233e6f979d9398 (DEFAULT_TABLE table=default.Table_8bd7e1f9b8fe4c60aea31b8b7bcf4aa8 [id=4824353236b849c4beb127506a7ec5e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.375730 14532 tablet_service.cc:1467] Processing CreateTablet for tablet 62f5e9bed95648ab92233e6f979d9398 (DEFAULT_TABLE table=default.Table_8bd7e1f9b8fe4c60aea31b8b7bcf4aa8 [id=4824353236b849c4beb127506a7ec5e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.375926 16556 raft_consensus.cc:491] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.376011 16556 raft_consensus.cc:513] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.376312 16556 leader_election.cc:290] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.376502 16535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.186.212:37407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.13.186.254:39901,127.13.186.253:35411,127.13.186.252:41855
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.186.254:39901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.186.254
--webserver_port=37481
--never_fsync=true
--heap_profile_path=/tmp/kudu.16535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 75b75ca2f58f1a5aba69207830e3558011033d99
build type DEBUG
built by None at 15 Jan 2025 02:43:26 UTC on 24a791456cd2
build id 4181
I20250115 03:06:22.376525 14665 tablet_service.cc:1467] Processing CreateTablet for tablet 7aa19e3f2b4e4c56978bf2940d810341 (DEFAULT_TABLE table=default.Table_fbb74332af2948bcb49ebad772550bc5 [id=c0a5251f78f94d77b2ffc03daeecc116]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.376894 14390 tablet_service.cc:1467] Processing CreateTablet for tablet 7aa19e3f2b4e4c56978bf2940d810341 (DEFAULT_TABLE table=default.Table_fbb74332af2948bcb49ebad772550bc5 [id=c0a5251f78f94d77b2ffc03daeecc116]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.377300 14523 tablet_service.cc:1467] Processing CreateTablet for tablet 7aa19e3f2b4e4c56978bf2940d810341 (DEFAULT_TABLE table=default.Table_fbb74332af2948bcb49ebad772550bc5 [id=c0a5251f78f94d77b2ffc03daeecc116]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.378016 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b61cc4b11251486a8089ca9bf44ac91e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.378039 14530 tablet_service.cc:1467] Processing CreateTablet for tablet c6e2d1f580df4f47a27d8d24a29b4cd1 (DEFAULT_TABLE table=default.Table_58a8eab6cfd443b284d979891ef43040 [id=13c2d2cd95fb45db8081667ca1412d2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.378124 14557 raft_consensus.cc:2463] T b61cc4b11251486a8089ca9bf44ac91e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.378181 14400 tablet_service.cc:1467] Processing CreateTablet for tablet c6e2d1f580df4f47a27d8d24a29b4cd1 (DEFAULT_TABLE table=default.Table_58a8eab6cfd443b284d979891ef43040 [id=13c2d2cd95fb45db8081667ca1412d2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.378630 16535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250115 03:06:22.379158 14397 tablet_service.cc:1467] Processing CreateTablet for tablet b1b4e99ea1d44993b629422138abd5ff (DEFAULT_TABLE table=default.Table_7269ada5c0804361a76bdaaad1c12b0c [id=263ef1416d9a4546bbfdbf212b3b78da]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:22.379545 14628 leader_election.cc:343] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: b61cc4b11251486a8089ca9bf44ac91e
I20250115 03:06:22.379689 14387 tablet_service.cc:1467] Processing CreateTablet for tablet 005efe16787f41d1afba600237b9cec8 (DEFAULT_TABLE table=default.Table_c87c8d70caff4b60905ff5c5ae70f4ab [id=223f6778cd8d4972831175bce0740ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.379869 16620 raft_consensus.cc:491] T f72c9bf04a3942ceacc37fd407bff033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.379947 16620 raft_consensus.cc:513] T f72c9bf04a3942ceacc37fd407bff033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.380062 14626 leader_election.cc:304] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:22.380159 16535 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250115 03:06:22.380203 16556 raft_consensus.cc:2798] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.380244 16556 raft_consensus.cc:491] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.380285 16556 raft_consensus.cc:3054] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.380805 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f72c9bf04a3942ceacc37fd407bff033" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.381160 14493 leader_election.cc:343] T f72c9bf04a3942ceacc37fd407bff033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.381903 16620 leader_election.cc:290] T f72c9bf04a3942ceacc37fd407bff033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.382020 16620 raft_consensus.cc:491] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.382186 16620 raft_consensus.cc:513] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.382746 14419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca14a7d574842ee94a93afb85ca1afd" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.382782 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet bf6199cdd6e940d1970dd43d8090100d (table default.Table_0d782d9b6b9b4020b10ab81b8541375e [id=0f03d804c7114afcb8d40e7e00025ede]). Expected version 2 got 0
I20250115 03:06:22.382858 14419 raft_consensus.cc:2463] T fca14a7d574842ee94a93afb85ca1afd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.382875 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet dda418ed197e4af2aa511d3f1851663e (table default.Table_f92234da7e9c4042a36f7acbd9c276d5 [id=cbab87c958a346199acdad65ab75be3c]). Expected version 4 got 0
I20250115 03:06:22.383172 14493 leader_election.cc:304] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.383356 16189 raft_consensus.cc:2798] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.383400 16189 raft_consensus.cc:491] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.383431 16189 raft_consensus.cc:3054] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.383639 14386 tablet_service.cc:1514] Processing DeleteTablet for tablet d643bd0947c54bcfa79f15d88e1e53e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:22.383777 14493 leader_election.cc:343] T f72c9bf04a3942ceacc37fd407bff033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: f72c9bf04a3942ceacc37fd407bff033
I20250115 03:06:22.383834 14493 leader_election.cc:304] T f72c9bf04a3942ceacc37fd407bff033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:22.384503 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca14a7d574842ee94a93afb85ca1afd" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.384609 14691 raft_consensus.cc:2463] T fca14a7d574842ee94a93afb85ca1afd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.385031 16620 leader_election.cc:290] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.385159 16620 raft_consensus.cc:2743] T f72c9bf04a3942ceacc37fd407bff033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.385958 14519 tablet_service.cc:1467] Processing CreateTablet for tablet b1b4e99ea1d44993b629422138abd5ff (DEFAULT_TABLE table=default.Table_7269ada5c0804361a76bdaaad1c12b0c [id=263ef1416d9a4546bbfdbf212b3b78da]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.386229 14524 tablet_service.cc:1467] Processing CreateTablet for tablet 005efe16787f41d1afba600237b9cec8 (DEFAULT_TABLE table=default.Table_c87c8d70caff4b60905ff5c5ae70f4ab [id=223f6778cd8d4972831175bce0740ede]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.387723 14422 raft_consensus.cc:2463] T 7f9818d0493547358fd5654a01d4f456 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:22.388126 16265 tablet.cc:1721] T 21524aef80f74a6c899ea66b9b022389 P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:22.388271 16608 tablet.cc:1721] T 21524aef80f74a6c899ea66b9b022389 P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:22.390493 14853 raft_consensus.cc:513] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.390810 14853 leader_election.cc:290] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.391378 16575 tablet.cc:1721] T 21524aef80f74a6c899ea66b9b022389 P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:22.391816 14692 raft_consensus.cc:2463] T dda418ed197e4af2aa511d3f1851663e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:22.391880 14819 tablet_replica.cc:331] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:22.392011 14819 raft_consensus.cc:2238] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:22.392107 14819 raft_consensus.cc:2267] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
W20250115 03:06:22.392091 16644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 03:06:22.392320 16646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250115 03:06:22.392405 14819 ts_tablet_manager.cc:1905] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:22.393507 14668 tablet_service.cc:1467] Processing CreateTablet for tablet c6e2d1f580df4f47a27d8d24a29b4cd1 (DEFAULT_TABLE table=default.Table_58a8eab6cfd443b284d979891ef43040 [id=13c2d2cd95fb45db8081667ca1412d2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.393738 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee1266087cbf44e591798dbd84782db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.393752 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6e2d1f580df4f47a27d8d24a29b4cd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.394011 14388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3db354a510143e08f07b267aec1dd5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.394276 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 707b29d359ff496a948d3a73cc7eb583. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.394546 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 824fe68c383d4483991841a9f4c1b4ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.394605 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 6132d15b17ab4d0aaa10a7ea04cda7ac (table default.Table_867788c99c794aadbb2dc125669537dc [id=01aa6b0cc55b46d0b4216c601c16b065]) successfully deleted
I20250115 03:06:22.394798 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a52345cdcd434b1b92f7bc62b90a5a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.394951 14656 tablet_service.cc:1467] Processing CreateTablet for tablet b1b4e99ea1d44993b629422138abd5ff (DEFAULT_TABLE table=default.Table_7269ada5c0804361a76bdaaad1c12b0c [id=263ef1416d9a4546bbfdbf212b3b78da]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.395045 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcf46748ff3f43a1b74ab505b4c1e9e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.395295 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e4b273297134df094d9ad90a08e6606. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.395570 14391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62f5e9bed95648ab92233e6f979d9398. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.395665 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1b4e99ea1d44993b629422138abd5ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.395799 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6e2d1f580df4f47a27d8d24a29b4cd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.396262 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7aa19e3f2b4e4c56978bf2940d810341. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.396217 14655 tablet_service.cc:1467] Processing CreateTablet for tablet 005efe16787f41d1afba600237b9cec8 (DEFAULT_TABLE table=default.Table_c87c8d70caff4b60905ff5c5ae70f4ab [id=223f6778cd8d4972831175bce0740ede]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:22.396637 16643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250115 03:06:22.398066 14688 raft_consensus.cc:1270] T dda418ed197e4af2aa511d3f1851663e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Refusing update from remote peer 84b1408700ef479cbb5a7b4d2f0bebf1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 03:06:22.399183 14387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 005efe16787f41d1afba600237b9cec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.399717 14493 leader_election.cc:304] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.400063 16618 raft_consensus.cc:2798] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:22.400141 16618 raft_consensus.cc:695] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:22.400254 16618 consensus_queue.cc:237] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.400570 14853 consensus_queue.cc:1035] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:22.400772 14493 leader_election.cc:343] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: ca117aca95f640be880b640c85e18cd4
I20250115 03:06:22.400976 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca117aca95f640be880b640c85e18cd4" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:22.401070 14688 raft_consensus.cc:3054] T ca117aca95f640be880b640c85e18cd4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.402560 14853 raft_consensus.cc:491] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.402628 14853 raft_consensus.cc:513] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.402913 14853 leader_election.cc:290] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.403129 16535 server_base.cc:1034] running on GCE node
I20250115 03:06:22.403312 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "781f6a9148cd4bd684f24c294c0b019b" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.403481 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "781f6a9148cd4bd684f24c294c0b019b" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.403577 14422 raft_consensus.cc:2463] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.403796 16535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250115 03:06:22.404844 16535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250115 03:06:22.405252 16608 tablet.cc:1721] T dda418ed197e4af2aa511d3f1851663e P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:22.405978 16287 mvcc.cc:204] Tried to move back new op lower bound from 7114384926169128960 to 7114384924725444608. Current Snapshot: MvccSnapshot[applied={T|T < 7114384926169128960}]
I20250115 03:06:22.406440 14493 leader_election.cc:304] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
W20250115 03:06:22.406595 14493 leader_election.cc:343] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.406803 14853 raft_consensus.cc:2798] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.406855 14853 raft_consensus.cc:491] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.406893 14853 raft_consensus.cc:3054] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.407075 16454 tablet.cc:1721] T 676ec8b9dc8241d4ae40488d3ce8f61e P 5c0dae7aff4b44c699275b95ba639770: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:22.407531 16411 raft_consensus.cc:491] T 60997f625d074960ad75aef5ba14b9c2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.407603 16411 raft_consensus.cc:513] T 60997f625d074960ad75aef5ba14b9c2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.407910 16411 leader_election.cc:290] T 60997f625d074960ad75aef5ba14b9c2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.408535 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60997f625d074960ad75aef5ba14b9c2" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.408704 16554 tablet.cc:1721] T dda418ed197e4af2aa511d3f1851663e P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:22.409332 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60997f625d074960ad75aef5ba14b9c2" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.409647 14358 leader_election.cc:343] T 60997f625d074960ad75aef5ba14b9c2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:22.409874 14356 leader_election.cc:343] T 60997f625d074960ad75aef5ba14b9c2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.409940 14356 leader_election.cc:304] T 60997f625d074960ad75aef5ba14b9c2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.410111 16411 raft_consensus.cc:2743] T 60997f625d074960ad75aef5ba14b9c2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.410385 14668 tablet_service.cc:1467] Processing CreateTablet for tablet b8b721500ba146d4a97fa52c86814a35 (DEFAULT_TABLE table=default.Table_f617d2044baa48a383e6d59fb6123ba0 [id=57ed57a903d14a50af4733efd8949934]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.411048 14525 tablet_service.cc:1467] Processing CreateTablet for tablet b8b721500ba146d4a97fa52c86814a35 (DEFAULT_TABLE table=default.Table_f617d2044baa48a383e6d59fb6123ba0 [id=57ed57a903d14a50af4733efd8949934]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.411610 14395 tablet_service.cc:1467] Processing CreateTablet for tablet b8b721500ba146d4a97fa52c86814a35 (DEFAULT_TABLE table=default.Table_f617d2044baa48a383e6d59fb6123ba0 [id=57ed57a903d14a50af4733efd8949934]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.411782 14399 tablet_service.cc:1467] Processing CreateTablet for tablet 9099a50c9e9f487dba708e5cbdff329c (DEFAULT_TABLE table=default.Table_365313892043408c92be8460c31d958e [id=f22f08e9af4a4496b100e7d4b5982121]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.412071 14520 tablet_service.cc:1467] Processing CreateTablet for tablet 9099a50c9e9f487dba708e5cbdff329c (DEFAULT_TABLE table=default.Table_365313892043408c92be8460c31d958e [id=f22f08e9af4a4496b100e7d4b5982121]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.412133 14657 tablet_service.cc:1467] Processing CreateTablet for tablet 9099a50c9e9f487dba708e5cbdff329c (DEFAULT_TABLE table=default.Table_365313892043408c92be8460c31d958e [id=f22f08e9af4a4496b100e7d4b5982121]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.412575 14386 tablet_service.cc:1467] Processing CreateTablet for tablet ea1ff51dc99e4cf5aa310cbaa5cb16c0 (DEFAULT_TABLE table=default.Table_c94a10c892a343a4bf3917501b740480 [id=da4dae59def44b45836e087d42312315]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.412712 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_f3e6282c66774e58a910ea88782b43ea" } new_table_name: "default.Table_31f2afe4c3e14688b4dbab0777d3107d" modify_external_catalogs: true
I20250115 03:06:22.412767 16535 hybrid_clock.cc:648] HybridClock initialized: now 1736910382412600 us; error 177 us; skew 500 ppm
I20250115 03:06:22.412737 14522 tablet_service.cc:1467] Processing CreateTablet for tablet ea1ff51dc99e4cf5aa310cbaa5cb16c0 (DEFAULT_TABLE table=default.Table_c94a10c892a343a4bf3917501b740480 [id=da4dae59def44b45836e087d42312315]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.416143 14811 tablet_bootstrap.cc:492] T a8a5743475814401ac2c0a51cfd70207 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.416268 14811 ts_tablet_manager.cc:1397] T a8a5743475814401ac2c0a51cfd70207 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.045s user 0.001s sys 0.000s
W20250115 03:06:22.416426 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 87e79385cac549b49e6c3477020ba8d1 consensus state has no leader
I20250115 03:06:22.416730 14811 raft_consensus.cc:357] T a8a5743475814401ac2c0a51cfd70207 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.416833 14811 raft_consensus.cc:383] T a8a5743475814401ac2c0a51cfd70207 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.416869 14811 raft_consensus.cc:738] T a8a5743475814401ac2c0a51cfd70207 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.417019 14811 consensus_queue.cc:260] T a8a5743475814401ac2c0a51cfd70207 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.417279 14811 ts_tablet_manager.cc:1428] T a8a5743475814401ac2c0a51cfd70207 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.417443 14811 tablet_bootstrap.cc:492] T 83a4f95eb9494a5295a81a8a51f58bd3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.418046 16535 webserver.cc:458] Webserver started at http://127.13.186.254:37481/ using document root <none> and password file <none>
I20250115 03:06:22.418455 14811 tablet_bootstrap.cc:654] T 83a4f95eb9494a5295a81a8a51f58bd3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.418769 16535 fs_manager.cc:362] Metadata directory not provided
I20250115 03:06:22.418839 16535 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250115 03:06:22.419951 14809 tablet_bootstrap.cc:492] T fdaf7c090ef642b8b339f61a5d3fb06e P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.420039 14809 ts_tablet_manager.cc:1397] T fdaf7c090ef642b8b339f61a5d3fb06e P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.066s user 0.001s sys 0.000s
I20250115 03:06:22.420459 14809 raft_consensus.cc:357] T fdaf7c090ef642b8b339f61a5d3fb06e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.420555 14809 raft_consensus.cc:383] T fdaf7c090ef642b8b339f61a5d3fb06e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.420583 14809 raft_consensus.cc:738] T fdaf7c090ef642b8b339f61a5d3fb06e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.420673 14809 consensus_queue.cc:260] T fdaf7c090ef642b8b339f61a5d3fb06e P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.420958 14809 ts_tablet_manager.cc:1428] T fdaf7c090ef642b8b339f61a5d3fb06e P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.421133 14809 tablet_bootstrap.cc:492] T 980b69b60a354519b0691ecb0bbd6c70 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.422308 14809 tablet_bootstrap.cc:654] T 980b69b60a354519b0691ecb0bbd6c70 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.422549 16568 raft_consensus.cc:491] T 75c581f11be84e6293a3d14966fd584f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.422631 16568 raft_consensus.cc:513] T 75c581f11be84e6293a3d14966fd584f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.422953 16568 leader_election.cc:290] T 75c581f11be84e6293a3d14966fd584f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.423297 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75c581f11be84e6293a3d14966fd584f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.423403 14557 raft_consensus.cc:2463] T 75c581f11be84e6293a3d14966fd584f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.423638 14626 leader_election.cc:304] T 75c581f11be84e6293a3d14966fd584f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:22.423790 16568 raft_consensus.cc:2798] T 75c581f11be84e6293a3d14966fd584f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.423838 16568 raft_consensus.cc:491] T 75c581f11be84e6293a3d14966fd584f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.423885 16568 raft_consensus.cc:3054] T 75c581f11be84e6293a3d14966fd584f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:22.425192 14628 leader_election.cc:343] T 75c581f11be84e6293a3d14966fd584f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 75c581f11be84e6293a3d14966fd584f
I20250115 03:06:22.426556 14168 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet dda418ed197e4af2aa511d3f1851663e (table default.Table_f92234da7e9c4042a36f7acbd9c276d5 [id=cbab87c958a346199acdad65ab75be3c]). Expected version 4 got 0
I20250115 03:06:22.427146 14811 tablet_bootstrap.cc:492] T 83a4f95eb9494a5295a81a8a51f58bd3 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.427286 14811 ts_tablet_manager.cc:1397] T 83a4f95eb9494a5295a81a8a51f58bd3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20250115 03:06:22.427631 14811 raft_consensus.cc:357] T 83a4f95eb9494a5295a81a8a51f58bd3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.427724 14811 raft_consensus.cc:383] T 83a4f95eb9494a5295a81a8a51f58bd3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.427754 14811 raft_consensus.cc:738] T 83a4f95eb9494a5295a81a8a51f58bd3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.427912 14811 consensus_queue.cc:260] T 83a4f95eb9494a5295a81a8a51f58bd3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.428135 14811 ts_tablet_manager.cc:1428] T 83a4f95eb9494a5295a81a8a51f58bd3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.428284 14811 tablet_bootstrap.cc:492] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.429234 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e69034f6c27e4ba8bccd3bfa8d47428e" } new_table_name: "default.Table_564fcc3818e74713863a9af747ef30bc" modify_external_catalogs: true
I20250115 03:06:22.429378 14811 tablet_bootstrap.cc:654] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.429646 14385 tablet_service.cc:1514] Processing DeleteTablet for tablet fdaf7c090ef642b8b339f61a5d3fb06e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.430794 14811 tablet_bootstrap.cc:492] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.430876 14811 ts_tablet_manager.cc:1397] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:22.431062 16618 raft_consensus.cc:491] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.430893 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_9d40428922614f03985ba101c29255a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:22.431144 16618 raft_consensus.cc:513] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.431269 14811 raft_consensus.cc:357] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.431382 14811 raft_consensus.cc:383] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.431419 14811 raft_consensus.cc:738] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.431499 16618 leader_election.cc:290] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.431548 14811 consensus_queue.cc:260] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
W20250115 03:06:22.431793 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9d40428922614f03985ba101c29255a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.432128 14811 ts_tablet_manager.cc:1428] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.432272 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f47da0e523d14388898a4be9c37bb809" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.432370 14692 raft_consensus.cc:2463] T f47da0e523d14388898a4be9c37bb809 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.432286 14811 tablet_bootstrap.cc:492] T 238e68300f304266aaed94bc3b76a01d P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.433219 16411 raft_consensus.cc:491] T d643bd0947c54bcfa79f15d88e1e53e4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.433293 16411 raft_consensus.cc:513] T d643bd0947c54bcfa79f15d88e1e53e4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.433329 14493 leader_election.cc:304] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:22.433568 16573 raft_consensus.cc:2798] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.433578 16411 leader_election.cc:290] T d643bd0947c54bcfa79f15d88e1e53e4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.433622 16573 raft_consensus.cc:491] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.433672 16573 raft_consensus.cc:3054] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.433929 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d643bd0947c54bcfa79f15d88e1e53e4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.434278 14358 leader_election.cc:343] T d643bd0947c54bcfa79f15d88e1e53e4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.430912 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_9091a95606104776ab8c2479b53af477"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:22.434613 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2faf003f0c414f759fe0c2e2115b3d85"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:22.434827 14811 tablet_bootstrap.cc:654] T 238e68300f304266aaed94bc3b76a01d P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:22.434829 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9091a95606104776ab8c2479b53af477 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:22.434906 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2faf003f0c414f759fe0c2e2115b3d85 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.435745 14809 tablet_bootstrap.cc:492] T 980b69b60a354519b0691ecb0bbd6c70 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.435822 14809 ts_tablet_manager.cc:1397] T 980b69b60a354519b0691ecb0bbd6c70 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
W20250115 03:06:22.436115 14493 leader_election.cc:343] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: f47da0e523d14388898a4be9c37bb809
I20250115 03:06:22.436136 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d643bd0947c54bcfa79f15d88e1e53e4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.436214 14809 raft_consensus.cc:357] T 980b69b60a354519b0691ecb0bbd6c70 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.436298 14809 raft_consensus.cc:383] T 980b69b60a354519b0691ecb0bbd6c70 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.436328 14809 raft_consensus.cc:738] T 980b69b60a354519b0691ecb0bbd6c70 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.436429 14809 consensus_queue.cc:260] T 980b69b60a354519b0691ecb0bbd6c70 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
W20250115 03:06:22.436527 14356 leader_election.cc:343] T d643bd0947c54bcfa79f15d88e1e53e4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.436587 14356 leader_election.cc:304] T d643bd0947c54bcfa79f15d88e1e53e4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.436688 14809 ts_tablet_manager.cc:1428] T 980b69b60a354519b0691ecb0bbd6c70 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.436767 16411 raft_consensus.cc:2743] T d643bd0947c54bcfa79f15d88e1e53e4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.436842 14809 tablet_bootstrap.cc:492] T 9bb0b7a8acfd40738abcc951936b1116 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
W20250115 03:06:22.437172 14165 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 607fc76ce73744c4b8ac5975515e1fa3 consensus state has no leader
I20250115 03:06:22.437987 16618 raft_consensus.cc:491] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.438071 16618 raft_consensus.cc:513] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.438344 16618 leader_election.cc:290] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.438851 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "103f78d0d9fc42308e1e23316895be33" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.438949 14692 raft_consensus.cc:2463] T 103f78d0d9fc42308e1e23316895be33 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.439149 14809 tablet_bootstrap.cc:654] T 9bb0b7a8acfd40738abcc951936b1116 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.439193 14493 leader_election.cc:304] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:22.439530 16618 raft_consensus.cc:2798] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.439607 16618 raft_consensus.cc:491] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.439656 16618 raft_consensus.cc:3054] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:22.440016 14493 leader_election.cc:343] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 103f78d0d9fc42308e1e23316895be33
I20250115 03:06:22.440984 14385 tablet_service.cc:1514] Processing DeleteTablet for tablet 980b69b60a354519b0691ecb0bbd6c70 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.445588 14811 tablet_bootstrap.cc:492] T 238e68300f304266aaed94bc3b76a01d P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.445672 14811 ts_tablet_manager.cc:1397] T 238e68300f304266aaed94bc3b76a01d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.000s
I20250115 03:06:22.446105 14811 raft_consensus.cc:357] T 238e68300f304266aaed94bc3b76a01d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.446200 14811 raft_consensus.cc:383] T 238e68300f304266aaed94bc3b76a01d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.446235 14811 raft_consensus.cc:738] T 238e68300f304266aaed94bc3b76a01d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.445536 14809 tablet_bootstrap.cc:492] T 9bb0b7a8acfd40738abcc951936b1116 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.446339 14811 consensus_queue.cc:260] T 238e68300f304266aaed94bc3b76a01d P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.446446 14809 ts_tablet_manager.cc:1397] T 9bb0b7a8acfd40738abcc951936b1116 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20250115 03:06:22.446573 14811 ts_tablet_manager.cc:1428] T 238e68300f304266aaed94bc3b76a01d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:22.446830 14809 raft_consensus.cc:357] T 9bb0b7a8acfd40738abcc951936b1116 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.446923 14809 raft_consensus.cc:383] T 9bb0b7a8acfd40738abcc951936b1116 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.446959 14809 raft_consensus.cc:738] T 9bb0b7a8acfd40738abcc951936b1116 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.447067 14809 consensus_queue.cc:260] T 9bb0b7a8acfd40738abcc951936b1116 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.447225 14812 tablet_bootstrap.cc:492] T f72c9bf04a3942ceacc37fd407bff033 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.447309 14812 ts_tablet_manager.cc:1397] T f72c9bf04a3942ceacc37fd407bff033 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.094s user 0.001s sys 0.000s
I20250115 03:06:22.447321 14809 ts_tablet_manager.cc:1428] T 9bb0b7a8acfd40738abcc951936b1116 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.447489 14809 tablet_bootstrap.cc:492] T 820d5af6daa74cadb53d2a5229854a60 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.447768 14812 raft_consensus.cc:357] T f72c9bf04a3942ceacc37fd407bff033 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.447873 14812 raft_consensus.cc:383] T f72c9bf04a3942ceacc37fd407bff033 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.447903 14812 raft_consensus.cc:738] T f72c9bf04a3942ceacc37fd407bff033 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.448009 14812 consensus_queue.cc:260] T f72c9bf04a3942ceacc37fd407bff033 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.448262 14812 ts_tablet_manager.cc:1428] T f72c9bf04a3942ceacc37fd407bff033 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.448347 14809 tablet_bootstrap.cc:654] T 820d5af6daa74cadb53d2a5229854a60 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.448367 16535 fs_manager.cc:714] Time spent opening directory manager: real 0.028s user 0.004s sys 0.000s
I20250115 03:06:22.448441 14812 tablet_bootstrap.cc:492] T 781f6a9148cd4bd684f24c294c0b019b P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.449785 14811 tablet_bootstrap.cc:492] T 194d943d173340e7a8fba69967d9b090 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.449890 14812 tablet_bootstrap.cc:654] T 781f6a9148cd4bd684f24c294c0b019b P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.450963 14811 tablet_bootstrap.cc:654] T 194d943d173340e7a8fba69967d9b090 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.451309 16661 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250115 03:06:22.452531 16535 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:22.452641 16535 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data,/tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/wal
uuid: "4225fe2fbfa84937bb41fed702ca66e4"
format_stamp: "Formatted at 2025-01-15 03:06:09 on dist-test-slave-c74p"
I20250115 03:06:22.453135 16535 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20250115 03:06:22.455365 14156 catalog_manager.cc:4670] Async tablet task AlterTable RPC for tablet e47012a789bd469f9691e5a4162ff512 (table default.Table_a838d05cbe294b9e8ec38d0c1a3f6221 [id=31ee40f75b634219ba0ae4b38e2c757e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e47012a789bd469f9691e5a4162ff512 consensus state has no leader
I20250115 03:06:22.455636 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_3f9adcb39f2945e88044d6f0ed9797ad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:22.455713 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_8b8c2a1323c94760a450e447b9a49789"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:22.455719 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_6770369286d841e39d362e0111c8733c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.455955 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3f9adcb39f2945e88044d6f0ed9797ad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:22.456030 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6770369286d841e39d362e0111c8733c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.456395 14812 tablet_bootstrap.cc:492] T 781f6a9148cd4bd684f24c294c0b019b P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.456480 14812 ts_tablet_manager.cc:1397] T 781f6a9148cd4bd684f24c294c0b019b P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20250115 03:06:22.456602 14385 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bb0b7a8acfd40738abcc951936b1116 with delete_type TABLET_DATA_DELETED (replaced by tablet 00aad81f4ca3427cb683de71148cff5c) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:22.456971 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b8c2a1323c94760a450e447b9a49789 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.456909 14812 raft_consensus.cc:357] T 781f6a9148cd4bd684f24c294c0b019b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.457442 14812 raft_consensus.cc:383] T 781f6a9148cd4bd684f24c294c0b019b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.457536 14812 raft_consensus.cc:738] T 781f6a9148cd4bd684f24c294c0b019b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.457772 14812 consensus_queue.cc:260] T 781f6a9148cd4bd684f24c294c0b019b P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.458421 14166 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_4f962cfd7b5b45629185faf7991c8bd3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.458715 14166 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f962cfd7b5b45629185faf7991c8bd3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.458127 14812 ts_tablet_manager.cc:1428] T 781f6a9148cd4bd684f24c294c0b019b P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20250115 03:06:22.459079 14812 tablet_bootstrap.cc:492] T 83a4f95eb9494a5295a81a8a51f58bd3 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.460505 14812 tablet_bootstrap.cc:654] T 83a4f95eb9494a5295a81a8a51f58bd3 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.465759 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_48bca7fdfb0544a59c84fc547c19e499"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.466058 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_48bca7fdfb0544a59c84fc547c19e499 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.467401 16411 raft_consensus.cc:491] T ef0994b1e51b4c688354998730e7b743 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.467481 16411 raft_consensus.cc:513] T ef0994b1e51b4c688354998730e7b743 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.467816 16411 leader_election.cc:290] T ef0994b1e51b4c688354998730e7b743 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.468187 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef0994b1e51b4c688354998730e7b743" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.468230 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef0994b1e51b4c688354998730e7b743" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.468575 14356 leader_election.cc:343] T ef0994b1e51b4c688354998730e7b743 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:22.468962 14358 leader_election.cc:343] T ef0994b1e51b4c688354998730e7b743 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.469030 14358 leader_election.cc:304] T ef0994b1e51b4c688354998730e7b743 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.469465 16411 raft_consensus.cc:2743] T ef0994b1e51b4c688354998730e7b743 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.470490 14812 tablet_bootstrap.cc:492] T 83a4f95eb9494a5295a81a8a51f58bd3 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.470569 14812 ts_tablet_manager.cc:1397] T 83a4f95eb9494a5295a81a8a51f58bd3 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.001s
I20250115 03:06:22.471189 14812 raft_consensus.cc:357] T 83a4f95eb9494a5295a81a8a51f58bd3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.471302 14812 raft_consensus.cc:383] T 83a4f95eb9494a5295a81a8a51f58bd3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.471338 14812 raft_consensus.cc:738] T 83a4f95eb9494a5295a81a8a51f58bd3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.471443 14812 consensus_queue.cc:260] T 83a4f95eb9494a5295a81a8a51f58bd3 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.471797 14812 ts_tablet_manager.cc:1428] T 83a4f95eb9494a5295a81a8a51f58bd3 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.471961 14812 tablet_bootstrap.cc:492] T 3e170ced1612442ebcfdaff6d9826ea3 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.472807 14811 tablet_bootstrap.cc:492] T 194d943d173340e7a8fba69967d9b090 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.472834 14812 tablet_bootstrap.cc:654] T 3e170ced1612442ebcfdaff6d9826ea3 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.472889 14811 ts_tablet_manager.cc:1397] T 194d943d173340e7a8fba69967d9b090 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20250115 03:06:22.473330 14811 raft_consensus.cc:357] T 194d943d173340e7a8fba69967d9b090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.473424 14811 raft_consensus.cc:383] T 194d943d173340e7a8fba69967d9b090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.473464 14811 raft_consensus.cc:738] T 194d943d173340e7a8fba69967d9b090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.473587 14811 consensus_queue.cc:260] T 194d943d173340e7a8fba69967d9b090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.473830 14811 ts_tablet_manager.cc:1428] T 194d943d173340e7a8fba69967d9b090 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.473971 14811 tablet_bootstrap.cc:492] T a3eaec2fff184aad816349fc36fa3db1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.474764 14811 tablet_bootstrap.cc:654] T a3eaec2fff184aad816349fc36fa3db1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.477561 14809 tablet_bootstrap.cc:492] T 820d5af6daa74cadb53d2a5229854a60 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.477644 14809 ts_tablet_manager.cc:1397] T 820d5af6daa74cadb53d2a5229854a60 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.000s
I20250115 03:06:22.477893 14654 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3db354a510143e08f07b267aec1dd5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.478063 14809 raft_consensus.cc:357] T 820d5af6daa74cadb53d2a5229854a60 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.478154 14809 raft_consensus.cc:383] T 820d5af6daa74cadb53d2a5229854a60 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.478184 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 707b29d359ff496a948d3a73cc7eb583. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.478183 14809 raft_consensus.cc:738] T 820d5af6daa74cadb53d2a5229854a60 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.478281 14809 consensus_queue.cc:260] T 820d5af6daa74cadb53d2a5229854a60 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.478384 14663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 824fe68c383d4483991841a9f4c1b4ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.478530 14809 ts_tablet_manager.cc:1428] T 820d5af6daa74cadb53d2a5229854a60 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:22.478583 14667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a52345cdcd434b1b92f7bc62b90a5a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.478698 14809 tablet_bootstrap.cc:492] T 815c5e096e4d4b0cbfb7c15bac571b10 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.478786 14661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcf46748ff3f43a1b74ab505b4c1e9e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.478976 14664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e4b273297134df094d9ad90a08e6606. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.479167 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62f5e9bed95648ab92233e6f979d9398. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.479358 14665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7aa19e3f2b4e4c56978bf2940d810341. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.479550 14656 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1b4e99ea1d44993b629422138abd5ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.479677 14809 tablet_bootstrap.cc:654] T 815c5e096e4d4b0cbfb7c15bac571b10 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.479768 14655 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 005efe16787f41d1afba600237b9cec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.480000 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8b721500ba146d4a97fa52c86814a35. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.480253 14657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9099a50c9e9f487dba708e5cbdff329c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.483579 16265 tablet.cc:1721] T 676ec8b9dc8241d4ae40488d3ce8f61e P 84b1408700ef479cbb5a7b4d2f0bebf1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:22.483984 14858 raft_consensus.cc:513] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.484344 14858 leader_election.cc:290] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.484422 14858 raft_consensus.cc:491] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.484474 14858 raft_consensus.cc:513] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.484254 16535 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250115 03:06:22.484741 14858 leader_election.cc:290] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.484949 14420 raft_consensus.cc:2463] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:22.485186 14687 raft_consensus.cc:2463] T 7f9818d0493547358fd5654a01d4f456 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:22.485208 16535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250115 03:06:22.485356 16535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250115 03:06:22.485930 14812 tablet_bootstrap.cc:492] T 3e170ced1612442ebcfdaff6d9826ea3 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.485997 14812 ts_tablet_manager.cc:1397] T 3e170ced1612442ebcfdaff6d9826ea3 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20250115 03:06:22.486446 14812 raft_consensus.cc:357] T 3e170ced1612442ebcfdaff6d9826ea3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.486537 14812 raft_consensus.cc:383] T 3e170ced1612442ebcfdaff6d9826ea3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.486567 14812 raft_consensus.cc:738] T 3e170ced1612442ebcfdaff6d9826ea3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.486658 14812 consensus_queue.cc:260] T 3e170ced1612442ebcfdaff6d9826ea3 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.487003 14812 ts_tablet_manager.cc:1428] T 3e170ced1612442ebcfdaff6d9826ea3 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.487147 14812 tablet_bootstrap.cc:492] T 238e68300f304266aaed94bc3b76a01d P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.488006 14812 tablet_bootstrap.cc:654] T 238e68300f304266aaed94bc3b76a01d P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.491428 14658 tablet_service.cc:1467] Processing CreateTablet for tablet ea1ff51dc99e4cf5aa310cbaa5cb16c0 (DEFAULT_TABLE table=default.Table_c94a10c892a343a4bf3917501b740480 [id=da4dae59def44b45836e087d42312315]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.492496 16620 raft_consensus.cc:491] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.492571 16620 raft_consensus.cc:513] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.492877 16620 leader_election.cc:290] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.493819 16535 rpc_server.cc:307] RPC server started. Bound to: 127.13.186.254:39901
I20250115 03:06:22.493841 16720 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.13.186.254:39901 every 8 connection(s)
I20250115 03:06:22.494959 14812 tablet_bootstrap.cc:492] T 238e68300f304266aaed94bc3b76a01d P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.495007 16535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/master-0/data/info.pb
I20250115 03:06:22.495028 14812 ts_tablet_manager.cc:1397] T 238e68300f304266aaed94bc3b76a01d P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20250115 03:06:22.495405 14812 raft_consensus.cc:357] T 238e68300f304266aaed94bc3b76a01d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.495503 14812 raft_consensus.cc:383] T 238e68300f304266aaed94bc3b76a01d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.495536 14812 raft_consensus.cc:738] T 238e68300f304266aaed94bc3b76a01d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.495667 14812 consensus_queue.cc:260] T 238e68300f304266aaed94bc3b76a01d P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.496117 14812 ts_tablet_manager.cc:1428] T 238e68300f304266aaed94bc3b76a01d P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.496269 14812 tablet_bootstrap.cc:492] T a8a5743475814401ac2c0a51cfd70207 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.497238 14687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68a04d57feea40eeb8e06cb6a2c61119" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:22.497342 14687 raft_consensus.cc:3054] T 68a04d57feea40eeb8e06cb6a2c61119 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.497735 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68a04d57feea40eeb8e06cb6a2c61119" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:22.497840 14420 raft_consensus.cc:3054] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.497936 14812 tablet_bootstrap.cc:654] T a8a5743475814401ac2c0a51cfd70207 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.498195 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e170ced1612442ebcfdaff6d9826ea3" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.498296 14692 raft_consensus.cc:2463] T 3e170ced1612442ebcfdaff6d9826ea3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
W20250115 03:06:22.498642 14493 leader_election.cc:343] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 3e170ced1612442ebcfdaff6d9826ea3
I20250115 03:06:22.498883 14812 tablet_bootstrap.cc:492] T a8a5743475814401ac2c0a51cfd70207 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.498951 14812 ts_tablet_manager.cc:1397] T a8a5743475814401ac2c0a51cfd70207 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20250115 03:06:22.499337 14166 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_fa0e824291a14ef5982d469c2a7fe69c" } new_table_name: "default.Table_8415f0a99bc6442290cc4d04887bdfdd" modify_external_catalogs: true
I20250115 03:06:22.500142 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aadcec8aca2b4ece915707c17f60b8b5" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.500159 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_04eafaab3f094b2998237e6ca1440386" } modify_external_catalogs: true
I20250115 03:06:22.500159 14416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aadcec8aca2b4ece915707c17f60b8b5" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.500240 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_04eafaab3f094b2998237e6ca1440386" } modify_external_catalogs: true
I20250115 03:06:22.500257 14416 raft_consensus.cc:2463] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.500877 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_320deab8a90b44af8922fcc44f378bbf" } modify_external_catalogs: true
I20250115 03:06:22.500989 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_320deab8a90b44af8922fcc44f378bbf" } modify_external_catalogs: true
I20250115 03:06:22.501217 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_05cfde56d7dd49b994fe32031c716aeb" } modify_external_catalogs: true
I20250115 03:06:22.501272 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_05cfde56d7dd49b994fe32031c716aeb" } modify_external_catalogs: true
I20250115 03:06:22.501271 16721 sys_catalog.cc:263] Verifying existing consensus state
I20250115 03:06:22.501719 14628 leader_election.cc:304] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.502033 16567 raft_consensus.cc:2798] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:22.502127 16567 raft_consensus.cc:695] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:22.502243 16567 consensus_queue.cc:237] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.502519 14812 raft_consensus.cc:357] T a8a5743475814401ac2c0a51cfd70207 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.502600 14812 raft_consensus.cc:383] T a8a5743475814401ac2c0a51cfd70207 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.502629 14812 raft_consensus.cc:738] T a8a5743475814401ac2c0a51cfd70207 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.502709 14812 consensus_queue.cc:260] T a8a5743475814401ac2c0a51cfd70207 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.503032 14812 ts_tablet_manager.cc:1428] T a8a5743475814401ac2c0a51cfd70207 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20250115 03:06:22.503119 14059 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskE2XgYg/build/debug/bin/kudu as pid 16535
I20250115 03:06:22.503181 14812 tablet_bootstrap.cc:492] T a3eaec2fff184aad816349fc36fa3db1 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.503542 16721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4: Bootstrap starting.
I20250115 03:06:22.504009 14812 tablet_bootstrap.cc:654] T a3eaec2fff184aad816349fc36fa3db1 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.504302 14493 leader_election.cc:304] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.504457 14493 leader_election.cc:304] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:22.504681 16620 raft_consensus.cc:2798] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20250115 03:06:22.504724 14493 leader_election.cc:343] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.504746 16620 raft_consensus.cc:491] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.504781 16620 raft_consensus.cc:3054] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.504863 16617 raft_consensus.cc:2798] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.504896 16617 raft_consensus.cc:491] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.504937 16617 raft_consensus.cc:3054] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.509043 16668 raft_consensus.cc:491] T c2afa048b3fd4362b50c8a6da306c25a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.509142 16668 raft_consensus.cc:513] T c2afa048b3fd4362b50c8a6da306c25a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.509472 16668 leader_election.cc:290] T c2afa048b3fd4362b50c8a6da306c25a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.510102 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2afa048b3fd4362b50c8a6da306c25a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.510211 14557 raft_consensus.cc:2463] T c2afa048b3fd4362b50c8a6da306c25a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.510510 14626 leader_election.cc:304] T c2afa048b3fd4362b50c8a6da306c25a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:22.510847 14811 tablet_bootstrap.cc:492] T a3eaec2fff184aad816349fc36fa3db1 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.510848 14162 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_7c5632f41fd641dabc346cc2c77f2988" } new_table_name: "default.Table_888e1854ea244e68914b76bc41d90137" modify_external_catalogs: true
I20250115 03:06:22.510911 14811 ts_tablet_manager.cc:1397] T a3eaec2fff184aad816349fc36fa3db1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.037s user 0.001s sys 0.000s
I20250115 03:06:22.511065 14809 tablet_bootstrap.cc:492] T 815c5e096e4d4b0cbfb7c15bac571b10 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.511138 14809 ts_tablet_manager.cc:1397] T 815c5e096e4d4b0cbfb7c15bac571b10 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.000s
I20250115 03:06:22.511323 14811 raft_consensus.cc:357] T a3eaec2fff184aad816349fc36fa3db1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.511405 14811 raft_consensus.cc:383] T a3eaec2fff184aad816349fc36fa3db1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.511432 14811 raft_consensus.cc:738] T a3eaec2fff184aad816349fc36fa3db1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.511535 14811 consensus_queue.cc:260] T a3eaec2fff184aad816349fc36fa3db1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.511595 14809 raft_consensus.cc:357] T 815c5e096e4d4b0cbfb7c15bac571b10 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.511703 14809 raft_consensus.cc:383] T 815c5e096e4d4b0cbfb7c15bac571b10 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.511736 14809 raft_consensus.cc:738] T 815c5e096e4d4b0cbfb7c15bac571b10 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.511770 14811 ts_tablet_manager.cc:1428] T a3eaec2fff184aad816349fc36fa3db1 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.511839 14809 consensus_queue.cc:260] T 815c5e096e4d4b0cbfb7c15bac571b10 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.511901 14811 tablet_bootstrap.cc:492] T de14b1d525cc42808500279ae6b60e80 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.512108 14809 ts_tablet_manager.cc:1428] T 815c5e096e4d4b0cbfb7c15bac571b10 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:22.512424 14809 tablet_bootstrap.cc:492] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.512423 14164 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_fd907927ba11487b907e7130bdfb6092" } modify_external_catalogs: true
I20250115 03:06:22.512511 14164 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_fd907927ba11487b907e7130bdfb6092" } modify_external_catalogs: true
I20250115 03:06:22.513339 14809 tablet_bootstrap.cc:654] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.513859 16668 raft_consensus.cc:2798] T c2afa048b3fd4362b50c8a6da306c25a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.513919 16668 raft_consensus.cc:491] T c2afa048b3fd4362b50c8a6da306c25a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.513957 16668 raft_consensus.cc:3054] T c2afa048b3fd4362b50c8a6da306c25a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.513960 14811 tablet_bootstrap.cc:654] T de14b1d525cc42808500279ae6b60e80 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:22.517107 14628 leader_election.cc:343] T c2afa048b3fd4362b50c8a6da306c25a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: c2afa048b3fd4362b50c8a6da306c25a
I20250115 03:06:22.518868 16411 raft_consensus.cc:491] T 041a30d6ee3247df883d00b5896b0eb0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.518947 16411 raft_consensus.cc:513] T 041a30d6ee3247df883d00b5896b0eb0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.519042 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_1288c4fed9bb4bee8a2a567a7b16990d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:22.519256 16411 leader_election.cc:290] T 041a30d6ee3247df883d00b5896b0eb0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:22.519357 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1288c4fed9bb4bee8a2a567a7b16990d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.519647 14811 tablet_bootstrap.cc:492] T de14b1d525cc42808500279ae6b60e80 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.519716 14811 ts_tablet_manager.cc:1397] T de14b1d525cc42808500279ae6b60e80 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20250115 03:06:22.519804 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "041a30d6ee3247df883d00b5896b0eb0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.520124 14811 raft_consensus.cc:357] T de14b1d525cc42808500279ae6b60e80 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
W20250115 03:06:22.520184 14356 leader_election.cc:343] T 041a30d6ee3247df883d00b5896b0eb0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.520212 14811 raft_consensus.cc:383] T de14b1d525cc42808500279ae6b60e80 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.520241 14811 raft_consensus.cc:738] T de14b1d525cc42808500279ae6b60e80 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.520330 14811 consensus_queue.cc:260] T de14b1d525cc42808500279ae6b60e80 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.520576 14811 ts_tablet_manager.cc:1428] T de14b1d525cc42808500279ae6b60e80 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.520706 14811 tablet_bootstrap.cc:492] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
W20250115 03:06:22.521205 14166 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet de552db1a01443589d95ba8993ef6f06 consensus state has no leader
I20250115 03:06:22.521420 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet b6175f691fec45bf9853ee90959a1c98
I20250115 03:06:22.521548 16411 raft_consensus.cc:491] T 2e7cf4fc3be8446292b0c91141eeadcc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.521597 14811 tablet_bootstrap.cc:654] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.521615 16411 raft_consensus.cc:513] T 2e7cf4fc3be8446292b0c91141eeadcc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.521919 16411 leader_election.cc:290] T 2e7cf4fc3be8446292b0c91141eeadcc P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.522398 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e7cf4fc3be8446292b0c91141eeadcc" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.522507 14385 tablet_service.cc:1514] Processing DeleteTablet for tablet b6175f691fec45bf9853ee90959a1c98 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.522701 14812 tablet_bootstrap.cc:492] T a3eaec2fff184aad816349fc36fa3db1 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.522778 14812 ts_tablet_manager.cc:1397] T a3eaec2fff184aad816349fc36fa3db1 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.001s
I20250115 03:06:22.522799 14858 raft_consensus.cc:491] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.522855 14858 raft_consensus.cc:513] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.523118 14858 leader_election.cc:290] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.523167 14812 raft_consensus.cc:357] T a3eaec2fff184aad816349fc36fa3db1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.523262 14812 raft_consensus.cc:383] T a3eaec2fff184aad816349fc36fa3db1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.523293 14812 raft_consensus.cc:738] T a3eaec2fff184aad816349fc36fa3db1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.523429 14812 consensus_queue.cc:260] T a3eaec2fff184aad816349fc36fa3db1 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
W20250115 03:06:22.523499 14356 leader_election.cc:343] T 2e7cf4fc3be8446292b0c91141eeadcc P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.523694 14812 ts_tablet_manager.cc:1428] T a3eaec2fff184aad816349fc36fa3db1 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.524055 14812 tablet_bootstrap.cc:492] T de14b1d525cc42808500279ae6b60e80 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.524824 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 820d5af6daa74cadb53d2a5229854a60 (table default.Table_31c0e931dbe844369b67344077333efa [id=8a3b8556dcae4ded8de4adcb19115d83]). Expected version 1 got 0
I20250115 03:06:22.524962 14812 tablet_bootstrap.cc:654] T de14b1d525cc42808500279ae6b60e80 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.524996 14537 tablet_service.cc:1514] Processing DeleteTablet for tablet b6175f691fec45bf9853ee90959a1c98 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:22.526579 14385 tablet_service.cc:1467] Processing CreateTablet for tablet 3f75507ac77a4a458e2394d3c9a16a91 (DEFAULT_TABLE table=default.Table_9d40428922614f03985ba101c29255a9 [id=b28583f60a0c4535a26249841af49923]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.526660 14384 tablet_service.cc:1467] Processing CreateTablet for tablet 1d3149d4e2744f7fa1032663ea2b46e7 (DEFAULT_TABLE table=default.Table_2faf003f0c414f759fe0c2e2115b3d85 [id=6b2ce97872bb414e96a4823cde9ab442]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.527104 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c580cdf3a854df988df32a9294c83bc" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.527206 14691 raft_consensus.cc:2463] T 0c580cdf3a854df988df32a9294c83bc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.526669 14383 tablet_service.cc:1467] Processing CreateTablet for tablet 3c1a685ffa324f56be0d8e3c2827974b (DEFAULT_TABLE table=default.Table_9091a95606104776ab8c2479b53af477 [id=3707a028ba604e7db8dc77c059a60f4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.527765 14531 tablet_service.cc:1467] Processing CreateTablet for tablet 1d3149d4e2744f7fa1032663ea2b46e7 (DEFAULT_TABLE table=default.Table_2faf003f0c414f759fe0c2e2115b3d85 [id=6b2ce97872bb414e96a4823cde9ab442]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.527819 14537 tablet_service.cc:1467] Processing CreateTablet for tablet 3f75507ac77a4a458e2394d3c9a16a91 (DEFAULT_TABLE table=default.Table_9d40428922614f03985ba101c29255a9 [id=b28583f60a0c4535a26249841af49923]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.528067 14528 tablet_service.cc:1467] Processing CreateTablet for tablet 3c1a685ffa324f56be0d8e3c2827974b (DEFAULT_TABLE table=default.Table_9091a95606104776ab8c2479b53af477 [id=3707a028ba604e7db8dc77c059a60f4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.528271 14493 leader_election.cc:304] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
W20250115 03:06:22.528479 14493 leader_election.cc:343] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 0c580cdf3a854df988df32a9294c83bc
I20250115 03:06:22.529011 14858 raft_consensus.cc:2798] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.529062 14858 raft_consensus.cc:491] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.529098 14858 raft_consensus.cc:3054] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.531553 14811 tablet_bootstrap.cc:492] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.531626 14811 ts_tablet_manager.cc:1397] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20250115 03:06:22.531982 14811 raft_consensus.cc:357] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.532075 14811 raft_consensus.cc:383] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.532105 14811 raft_consensus.cc:738] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.532198 14811 consensus_queue.cc:260] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.532316 14739 heartbeater.cc:346] Connected to a master server at 127.13.186.254:39901
I20250115 03:06:22.532528 14811 ts_tablet_manager.cc:1428] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.532696 14811 tablet_bootstrap.cc:492] T e47012a789bd469f9691e5a4162ff512 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.533550 14811 tablet_bootstrap.cc:654] T e47012a789bd469f9691e5a4162ff512 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.533572 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e7cf4fc3be8446292b0c91141eeadcc" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.534524 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "041a30d6ee3247df883d00b5896b0eb0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.534633 14812 tablet_bootstrap.cc:492] T de14b1d525cc42808500279ae6b60e80 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.534703 14812 ts_tablet_manager.cc:1397] T de14b1d525cc42808500279ae6b60e80 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20250115 03:06:22.535079 14812 raft_consensus.cc:357] T de14b1d525cc42808500279ae6b60e80 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.535169 14812 raft_consensus.cc:383] T de14b1d525cc42808500279ae6b60e80 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.535202 14812 raft_consensus.cc:738] T de14b1d525cc42808500279ae6b60e80 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
W20250115 03:06:22.535276 14358 leader_election.cc:343] T 2e7cf4fc3be8446292b0c91141eeadcc P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.535302 14812 consensus_queue.cc:260] T de14b1d525cc42808500279ae6b60e80 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.535349 14358 leader_election.cc:304] T 2e7cf4fc3be8446292b0c91141eeadcc P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.535537 16411 raft_consensus.cc:2743] T 2e7cf4fc3be8446292b0c91141eeadcc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.535559 14812 ts_tablet_manager.cc:1428] T de14b1d525cc42808500279ae6b60e80 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:22.535707 14812 tablet_bootstrap.cc:492] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:22.536237 14358 leader_election.cc:343] T 041a30d6ee3247df883d00b5896b0eb0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.536309 14358 leader_election.cc:304] T 041a30d6ee3247df883d00b5896b0eb0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.536469 16411 raft_consensus.cc:2743] T 041a30d6ee3247df883d00b5896b0eb0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.536577 14812 tablet_bootstrap.cc:654] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.536697 16567 consensus_queue.cc:1035] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.537097 16735 raft_consensus.cc:491] T 194d943d173340e7a8fba69967d9b090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.537168 16735 raft_consensus.cc:513] T 194d943d173340e7a8fba69967d9b090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.537330 14604 heartbeater.cc:346] Connected to a master server at 127.13.186.254:39901
I20250115 03:06:22.537711 14469 heartbeater.cc:346] Connected to a master server at 127.13.186.254:39901
I20250115 03:06:22.538208 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "194d943d173340e7a8fba69967d9b090" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.538277 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "194d943d173340e7a8fba69967d9b090" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.538389 14422 raft_consensus.cc:2463] T 194d943d173340e7a8fba69967d9b090 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
W20250115 03:06:22.538512 14493 leader_election.cc:343] T 194d943d173340e7a8fba69967d9b090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.538745 14493 leader_election.cc:304] T 194d943d173340e7a8fba69967d9b090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770
I20250115 03:06:22.538894 16735 leader_election.cc:290] T 194d943d173340e7a8fba69967d9b090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.539012 16735 raft_consensus.cc:2798] T 194d943d173340e7a8fba69967d9b090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.539047 16735 raft_consensus.cc:491] T 194d943d173340e7a8fba69967d9b090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.539076 16735 raft_consensus.cc:3054] T 194d943d173340e7a8fba69967d9b090 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.542784 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_655e9d02aacf4b88ba521f0f57a5f522" } modify_external_catalogs: true
I20250115 03:06:22.542882 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_655e9d02aacf4b88ba521f0f57a5f522" } modify_external_catalogs: true
I20250115 03:06:22.545713 14809 tablet_bootstrap.cc:492] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.545812 14809 ts_tablet_manager.cc:1397] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.000s
I20250115 03:06:22.546319 14809 raft_consensus.cc:357] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.546419 14809 raft_consensus.cc:383] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.546452 14809 raft_consensus.cc:738] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.546547 14809 consensus_queue.cc:260] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.546823 14809 ts_tablet_manager.cc:1428] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.546964 14809 tablet_bootstrap.cc:492] T b0a7def7d9a7418381b2d5a93a9a9265 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.547832 14809 tablet_bootstrap.cc:654] T b0a7def7d9a7418381b2d5a93a9a9265 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:22.548556 14162 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 917a0c0014a741ba879b7fbda8c9cd3d consensus state has no leader
I20250115 03:06:22.548802 14164 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 85b4eb3b20bb4908884e5716e7b2a369
I20250115 03:06:22.549219 14529 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b4eb3b20bb4908884e5716e7b2a369 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:22.549252 14811 tablet_bootstrap.cc:492] T e47012a789bd469f9691e5a4162ff512 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.548943 16736 consensus_queue.cc:1035] T 4b39608e19bf44b3a79a8616ccdad30a P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.549335 14811 ts_tablet_manager.cc:1397] T e47012a789bd469f9691e5a4162ff512 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20250115 03:06:22.549728 14811 raft_consensus.cc:357] T e47012a789bd469f9691e5a4162ff512 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.549813 14811 raft_consensus.cc:383] T e47012a789bd469f9691e5a4162ff512 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.549844 14811 raft_consensus.cc:738] T e47012a789bd469f9691e5a4162ff512 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.549952 14811 consensus_queue.cc:260] T e47012a789bd469f9691e5a4162ff512 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.550223 14811 ts_tablet_manager.cc:1428] T e47012a789bd469f9691e5a4162ff512 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.550380 14811 tablet_bootstrap.cc:492] T 78cc96de8b414ac19285338ccb1024e0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.550940 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet c8ba7e498bcf4359a1a588c1daea3fb0
I20250115 03:06:22.550958 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet b2d42233d2234f7da262753c3187be43
I20250115 03:06:22.551219 14811 tablet_bootstrap.cc:654] T 78cc96de8b414ac19285338ccb1024e0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.551914 14529 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ba7e498bcf4359a1a588c1daea3fb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:22.552054 14529 tablet_service.cc:1514] Processing DeleteTablet for tablet b2d42233d2234f7da262753c3187be43 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:22.553275 16741 raft_consensus.cc:491] T a8a5743475814401ac2c0a51cfd70207 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.553349 16741 raft_consensus.cc:513] T a8a5743475814401ac2c0a51cfd70207 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.553896 16741 leader_election.cc:290] T a8a5743475814401ac2c0a51cfd70207 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.554337 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a5743475814401ac2c0a51cfd70207" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.554451 14692 raft_consensus.cc:2463] T a8a5743475814401ac2c0a51cfd70207 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.554805 14416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a5743475814401ac2c0a51cfd70207" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.554818 14493 leader_election.cc:304] T a8a5743475814401ac2c0a51cfd70207 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:22.554910 14416 raft_consensus.cc:2463] T a8a5743475814401ac2c0a51cfd70207 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.555462 14809 tablet_bootstrap.cc:492] T b0a7def7d9a7418381b2d5a93a9a9265 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.555502 16739 raft_consensus.cc:2798] T a8a5743475814401ac2c0a51cfd70207 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.555538 14809 ts_tablet_manager.cc:1397] T b0a7def7d9a7418381b2d5a93a9a9265 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:22.555552 16739 raft_consensus.cc:491] T a8a5743475814401ac2c0a51cfd70207 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.555593 16739 raft_consensus.cc:3054] T a8a5743475814401ac2c0a51cfd70207 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.555961 14809 raft_consensus.cc:357] T b0a7def7d9a7418381b2d5a93a9a9265 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.556046 14809 raft_consensus.cc:383] T b0a7def7d9a7418381b2d5a93a9a9265 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.556077 14809 raft_consensus.cc:738] T b0a7def7d9a7418381b2d5a93a9a9265 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.556169 14809 consensus_queue.cc:260] T b0a7def7d9a7418381b2d5a93a9a9265 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.556442 14809 ts_tablet_manager.cc:1428] T b0a7def7d9a7418381b2d5a93a9a9265 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.556592 14809 tablet_bootstrap.cc:492] T 015ec42caa1b468ea10f25c8f26e278a P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.557433 14809 tablet_bootstrap.cc:654] T 015ec42caa1b468ea10f25c8f26e278a P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.559461 14809 tablet_bootstrap.cc:492] T 015ec42caa1b468ea10f25c8f26e278a P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.560223 14809 ts_tablet_manager.cc:1397] T 015ec42caa1b468ea10f25c8f26e278a P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20250115 03:06:22.560652 14809 raft_consensus.cc:357] T 015ec42caa1b468ea10f25c8f26e278a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.560791 14809 raft_consensus.cc:383] T 015ec42caa1b468ea10f25c8f26e278a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.560861 14809 raft_consensus.cc:738] T 015ec42caa1b468ea10f25c8f26e278a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.561102 14809 consensus_queue.cc:260] T 015ec42caa1b468ea10f25c8f26e278a P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.561424 14809 ts_tablet_manager.cc:1428] T 015ec42caa1b468ea10f25c8f26e278a P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:22.561806 14809 tablet_bootstrap.cc:492] T 4c4ce5c2921341ce837ee4f5f7d131b7 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.562759 14809 tablet_bootstrap.cc:654] T 4c4ce5c2921341ce837ee4f5f7d131b7 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.564810 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_9f9940dec82b49098561c9a2e90b3475" } new_table_name: "default.Table_9c0eccc42d6e441ba33a1e497b11074e" modify_external_catalogs: true
I20250115 03:06:22.565271 14812 tablet_bootstrap.cc:492] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.565351 14812 ts_tablet_manager.cc:1397] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.000s
I20250115 03:06:22.565754 14812 raft_consensus.cc:357] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.565845 14812 raft_consensus.cc:383] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.565871 14812 raft_consensus.cc:738] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.565966 14812 consensus_queue.cc:260] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.566195 14812 ts_tablet_manager.cc:1428] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.566324 14812 tablet_bootstrap.cc:492] T 194d943d173340e7a8fba69967d9b090 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.567132 14812 tablet_bootstrap.cc:654] T 194d943d173340e7a8fba69967d9b090 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.567435 14811 tablet_bootstrap.cc:492] T 78cc96de8b414ac19285338ccb1024e0 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.567504 14811 ts_tablet_manager.cc:1397] T 78cc96de8b414ac19285338ccb1024e0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20250115 03:06:22.567870 14820 ts_tablet_manager.cc:1918] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20250115 03:06:22.567931 14820 log.cc:1198] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-1/wal/wals/1016cf85491d4ceda132ee57325b48a4
I20250115 03:06:22.568141 14820 ts_tablet_manager.cc:1939] T 1016cf85491d4ceda132ee57325b48a4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting consensus metadata
I20250115 03:06:22.569104 14526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e4b273297134df094d9ad90a08e6606. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.569231 14520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9099a50c9e9f487dba708e5cbdff329c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.569357 14532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62f5e9bed95648ab92233e6f979d9398. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.569474 14522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea1ff51dc99e4cf5aa310cbaa5cb16c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.569574 14523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7aa19e3f2b4e4c56978bf2940d810341. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.569696 14537 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f75507ac77a4a458e2394d3c9a16a91. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.570034 14530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6e2d1f580df4f47a27d8d24a29b4cd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.570277 14528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c1a685ffa324f56be0d8e3c2827974b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.570536 14521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3db354a510143e08f07b267aec1dd5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.570767 14531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3149d4e2744f7fa1032663ea2b46e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.571058 14819 ts_tablet_manager.cc:1918] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:22.571123 14819 log.cc:1198] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/de14b1d525cc42808500279ae6b60e80
I20250115 03:06:22.571322 14819 ts_tablet_manager.cc:1939] T de14b1d525cc42808500279ae6b60e80 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
I20250115 03:06:22.571463 14417 raft_consensus.cc:2463] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:22.571717 14853 raft_consensus.cc:513] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.571802 14392 tablet_service.cc:1514] Processing DeleteTablet for tablet 815c5e096e4d4b0cbfb7c15bac571b10 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.571910 14392 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b4eb3b20bb4908884e5716e7b2a369 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.571985 14392 tablet_service.cc:1514] Processing DeleteTablet for tablet b2d42233d2234f7da262753c3187be43 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.572057 14392 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ba7e498bcf4359a1a588c1daea3fb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.572053 14853 leader_election.cc:290] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.572273 16558 raft_consensus.cc:513] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.572572 14692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "781f6a9148cd4bd684f24c294c0b019b" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:22.572624 16558 leader_election.cc:290] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:22.572659 14692 raft_consensus.cc:3054] T 781f6a9148cd4bd684f24c294c0b019b P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.572785 14536 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 707b29d359ff496a948d3a73cc7eb583. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.573063 14534 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a52345cdcd434b1b92f7bc62b90a5a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.573258 14535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 824fe68c383d4483991841a9f4c1b4ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.573446 14527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcf46748ff3f43a1b74ab505b4c1e9e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.582398 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_55bbd621f94d45fc8b93a7211619f08a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.582697 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_55bbd621f94d45fc8b93a7211619f08a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.584316 14417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "781f6a9148cd4bd684f24c294c0b019b" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:22.584399 14417 raft_consensus.cc:3054] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.585107 14557 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b5fd91a0ea64b3b85b7571e5daa08b2" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:22.585204 14557 raft_consensus.cc:3054] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.585367 14658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea1ff51dc99e4cf5aa310cbaa5cb16c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.585498 14823 tablet_replica.cc:331] T 0e1ce43146ed4e4d82a7c134216e2872 P 5c0dae7aff4b44c699275b95ba639770: stopping tablet replica
I20250115 03:06:22.585634 14823 raft_consensus.cc:2238] T 0e1ce43146ed4e4d82a7c134216e2872 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:22.585740 14823 raft_consensus.cc:2267] T 0e1ce43146ed4e4d82a7c134216e2872 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:22.586061 14823 ts_tablet_manager.cc:1905] T 0e1ce43146ed4e4d82a7c134216e2872 P 5c0dae7aff4b44c699275b95ba639770: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:22.586313 14812 tablet_bootstrap.cc:492] T 194d943d173340e7a8fba69967d9b090 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.586377 14812 ts_tablet_manager.cc:1397] T 194d943d173340e7a8fba69967d9b090 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.000s
I20250115 03:06:22.586791 14812 raft_consensus.cc:357] T 194d943d173340e7a8fba69967d9b090 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.586877 14812 raft_consensus.cc:383] T 194d943d173340e7a8fba69967d9b090 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.586906 14812 raft_consensus.cc:738] T 194d943d173340e7a8fba69967d9b090 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.587007 14812 consensus_queue.cc:260] T 194d943d173340e7a8fba69967d9b090 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.587256 14812 ts_tablet_manager.cc:1428] T 194d943d173340e7a8fba69967d9b090 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.587340 14811 raft_consensus.cc:357] T 78cc96de8b414ac19285338ccb1024e0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.587404 14812 tablet_bootstrap.cc:492] T e47012a789bd469f9691e5a4162ff512 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.587368 14416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b5fd91a0ea64b3b85b7571e5daa08b2" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:22.587431 14811 raft_consensus.cc:383] T 78cc96de8b414ac19285338ccb1024e0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.587458 14416 raft_consensus.cc:3054] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.587464 14811 raft_consensus.cc:738] T 78cc96de8b414ac19285338ccb1024e0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.587565 14811 consensus_queue.cc:260] T 78cc96de8b414ac19285338ccb1024e0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.587793 14811 ts_tablet_manager.cc:1428] T 78cc96de8b414ac19285338ccb1024e0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.020s user 0.001s sys 0.000s
I20250115 03:06:22.587811 14525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8b721500ba146d4a97fa52c86814a35. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.588084 14524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 005efe16787f41d1afba600237b9cec8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.588603 14812 tablet_bootstrap.cc:654] T e47012a789bd469f9691e5a4162ff512 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.588649 14519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1b4e99ea1d44993b629422138abd5ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.591526 14812 tablet_bootstrap.cc:492] T e47012a789bd469f9691e5a4162ff512 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.591607 14812 ts_tablet_manager.cc:1397] T e47012a789bd469f9691e5a4162ff512 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250115 03:06:22.591991 14812 raft_consensus.cc:357] T e47012a789bd469f9691e5a4162ff512 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.592090 14812 raft_consensus.cc:383] T e47012a789bd469f9691e5a4162ff512 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.592132 14812 raft_consensus.cc:738] T e47012a789bd469f9691e5a4162ff512 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.592242 14812 consensus_queue.cc:260] T e47012a789bd469f9691e5a4162ff512 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.588645 14811 tablet_bootstrap.cc:492] T d5e4743e016e49ed8ba26b1ac96b1470 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.592516 14812 ts_tablet_manager.cc:1428] T e47012a789bd469f9691e5a4162ff512 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.592670 14812 tablet_bootstrap.cc:492] T aadcec8aca2b4ece915707c17f60b8b5 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.593298 14811 tablet_bootstrap.cc:654] T d5e4743e016e49ed8ba26b1ac96b1470 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.593317 14158 catalog_manager.cc:4872] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): tablet 1016cf85491d4ceda132ee57325b48a4 (table default.Table_2e115074bf7049bfbb1239b4d9c1ca2e [id=6cf622de00cb490bb794837223aa7bcb]) successfully deleted
W20250115 03:06:22.593492 14158 catalog_manager.cc:4852] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet ec0f3424ebcb442597a94ccdbb25eeb6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ec0f3424ebcb442597a94ccdbb25eeb6 already in progress: creating tablet
I20250115 03:06:22.593554 14812 tablet_bootstrap.cc:654] T aadcec8aca2b4ece915707c17f60b8b5 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:22.593611 14158 catalog_manager.cc:4839] TS 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): delete failed for tablet b2629ae9d9854ae4b6e7ddddbb6f341b because the tablet was not found. No further retry: Not found: Tablet not found: b2629ae9d9854ae4b6e7ddddbb6f341b
I20250115 03:06:22.596904 16727 raft_consensus.cc:491] T f72c9bf04a3942ceacc37fd407bff033 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.597075 16727 raft_consensus.cc:513] T f72c9bf04a3942ceacc37fd407bff033 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
W20250115 03:06:22.597787 14163 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 87b234185fc64505b097f2ae7d52d813 consensus state has no leader
I20250115 03:06:22.600419 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f72c9bf04a3942ceacc37fd407bff033" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.600508 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet cd07196be4b54a90865aecea55493aa1
I20250115 03:06:22.600539 14553 raft_consensus.cc:2463] T f72c9bf04a3942ceacc37fd407bff033 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.601403 14389 tablet_service.cc:1514] Processing DeleteTablet for tablet cd07196be4b54a90865aecea55493aa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.601650 16727 leader_election.cc:290] T f72c9bf04a3942ceacc37fd407bff033 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.601779 14626 leader_election.cc:304] T f72c9bf04a3942ceacc37fd407bff033 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
W20250115 03:06:22.604660 14628 leader_election.cc:343] T f72c9bf04a3942ceacc37fd407bff033 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: f72c9bf04a3942ceacc37fd407bff033
I20250115 03:06:22.605516 14853 raft_consensus.cc:491] T a3eaec2fff184aad816349fc36fa3db1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.605592 14853 raft_consensus.cc:513] T a3eaec2fff184aad816349fc36fa3db1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.605895 14853 leader_election.cc:290] T a3eaec2fff184aad816349fc36fa3db1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.606496 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3eaec2fff184aad816349fc36fa3db1" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.606494 14853 consensus_queue.cc:1035] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.606598 14691 raft_consensus.cc:2463] T a3eaec2fff184aad816349fc36fa3db1 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.606973 14529 tablet_service.cc:1514] Processing DeleteTablet for tablet cd07196be4b54a90865aecea55493aa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:47222
I20250115 03:06:22.606987 14493 leader_election.cc:304] T a3eaec2fff184aad816349fc36fa3db1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:22.607673 14811 tablet_bootstrap.cc:492] T d5e4743e016e49ed8ba26b1ac96b1470 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
W20250115 03:06:22.607733 14493 leader_election.cc:343] T a3eaec2fff184aad816349fc36fa3db1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: a3eaec2fff184aad816349fc36fa3db1
I20250115 03:06:22.607741 14811 ts_tablet_manager.cc:1397] T d5e4743e016e49ed8ba26b1ac96b1470 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20250115 03:06:22.608279 14811 raft_consensus.cc:357] T d5e4743e016e49ed8ba26b1ac96b1470 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.608419 14811 raft_consensus.cc:383] T d5e4743e016e49ed8ba26b1ac96b1470 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.608496 14811 raft_consensus.cc:738] T d5e4743e016e49ed8ba26b1ac96b1470 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.608625 16747 raft_consensus.cc:2798] T a3eaec2fff184aad816349fc36fa3db1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.608700 16747 raft_consensus.cc:491] T a3eaec2fff184aad816349fc36fa3db1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.608745 16747 raft_consensus.cc:3054] T a3eaec2fff184aad816349fc36fa3db1 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.608634 14811 consensus_queue.cc:260] T d5e4743e016e49ed8ba26b1ac96b1470 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.611923 14809 tablet_bootstrap.cc:492] T 4c4ce5c2921341ce837ee4f5f7d131b7 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.611930 14811 ts_tablet_manager.cc:1428] T d5e4743e016e49ed8ba26b1ac96b1470 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:22.612012 14809 ts_tablet_manager.cc:1397] T 4c4ce5c2921341ce837ee4f5f7d131b7 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.050s user 0.000s sys 0.001s
I20250115 03:06:22.612825 14809 raft_consensus.cc:357] T 4c4ce5c2921341ce837ee4f5f7d131b7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.612948 14809 raft_consensus.cc:383] T 4c4ce5c2921341ce837ee4f5f7d131b7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.612988 14809 raft_consensus.cc:738] T 4c4ce5c2921341ce837ee4f5f7d131b7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.613126 14809 consensus_queue.cc:260] T 4c4ce5c2921341ce837ee4f5f7d131b7 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.613425 14809 ts_tablet_manager.cc:1428] T 4c4ce5c2921341ce837ee4f5f7d131b7 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.613598 14809 tablet_bootstrap.cc:492] T 7340ac89e5524c35948f3acc6c941cb5 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.613795 14812 tablet_bootstrap.cc:492] T aadcec8aca2b4ece915707c17f60b8b5 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.613873 14812 ts_tablet_manager.cc:1397] T aadcec8aca2b4ece915707c17f60b8b5 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.000s
I20250115 03:06:22.614243 14812 raft_consensus.cc:357] T aadcec8aca2b4ece915707c17f60b8b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.614333 14812 raft_consensus.cc:383] T aadcec8aca2b4ece915707c17f60b8b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.614364 14812 raft_consensus.cc:738] T aadcec8aca2b4ece915707c17f60b8b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.614485 14812 consensus_queue.cc:260] T aadcec8aca2b4ece915707c17f60b8b5 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.614531 14809 tablet_bootstrap.cc:654] T 7340ac89e5524c35948f3acc6c941cb5 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.614524 16736 raft_consensus.cc:2798] T f72c9bf04a3942ceacc37fd407bff033 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.614573 16736 raft_consensus.cc:491] T f72c9bf04a3942ceacc37fd407bff033 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.614606 16736 raft_consensus.cc:3054] T f72c9bf04a3942ceacc37fd407bff033 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.615106 14812 ts_tablet_manager.cc:1428] T aadcec8aca2b4ece915707c17f60b8b5 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.615269 14812 tablet_bootstrap.cc:492] T fe8d94c8fecc4077ba052390810c7430 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.616225 14812 tablet_bootstrap.cc:654] T fe8d94c8fecc4077ba052390810c7430 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.616212 14171 catalog_manager.cc:5526] T 4b39608e19bf44b3a79a8616ccdad30a P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } }
I20250115 03:06:22.616763 14811 tablet_bootstrap.cc:492] T cef4442bfcfd4ad4b343a2785c3a9a74 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.617852 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a96d43a0d2eb4f5c86b28defd2c3de4f" } new_table_name: "default.Table_517dc67d82984752aec6733979bd1c71" modify_external_catalogs: true
I20250115 03:06:22.619112 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 781f6a9148cd4bd684f24c294c0b019b (table default.Table_d64c7e1e71a3478687247bda551f3fe2 [id=b9f32421ef8a4c549dab58ffb0f71fa7]). Expected version 1 got 0
I20250115 03:06:22.619246 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 8efa998709034fe9bb37c7c608c99e76 (table default.Table_b6002fb5cc1248bba2eb7abbd5a5d407 [id=4c458a80b1a2490da1cf689843cae584]). Expected version 1 got 0
I20250115 03:06:22.619493 14167 catalog_manager.cc:5526] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:22.619673 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 5a2894e4736b4191963a61d908bc673c (table default.Table_df79ed42f26f4b68b197877c445b9665 [id=bb6453f2ab97404cbd3ba32a87976e52]). Expected version 1 got 0
I20250115 03:06:22.619820 14167 catalog_manager.cc:5526] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20250115 03:06:22.619964 14167 catalog_manager.cc:5618] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693) does not have the latest schema for tablet 6bd4e4200f9b489ea5dd2154aa6f382d (table default.Table_83b1fd83225a4454b29b7c61bc4639d4 [id=299ac71cfb98480f873eb659419e522c]). Expected version 1 got 0
I20250115 03:06:22.621423 14529 tablet_service.cc:1467] Processing CreateTablet for tablet a97b4ad4327446cbae405f3d43a17781 (DEFAULT_TABLE table=default.Table_4f962cfd7b5b45629185faf7991c8bd3 [id=a2abece3dc784cb8a7c503b92a39bbf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.621793 14389 tablet_service.cc:1467] Processing CreateTablet for tablet a97b4ad4327446cbae405f3d43a17781 (DEFAULT_TABLE table=default.Table_4f962cfd7b5b45629185faf7991c8bd3 [id=a2abece3dc784cb8a7c503b92a39bbf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.624387 14392 tablet_service.cc:1467] Processing CreateTablet for tablet eee4e25e788646c9abaf9ad9ccb0a92c (DEFAULT_TABLE table=default.Table_6770369286d841e39d362e0111c8733c [id=69bc7ddbc89844ebb1471556cd9130af]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.625308 14812 tablet_bootstrap.cc:492] T fe8d94c8fecc4077ba052390810c7430 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.625375 14812 ts_tablet_manager.cc:1397] T fe8d94c8fecc4077ba052390810c7430 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20250115 03:06:22.625654 14853 raft_consensus.cc:491] T 238e68300f304266aaed94bc3b76a01d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.625736 14853 raft_consensus.cc:513] T 238e68300f304266aaed94bc3b76a01d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.625867 14812 raft_consensus.cc:357] T fe8d94c8fecc4077ba052390810c7430 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.625953 14812 raft_consensus.cc:383] T fe8d94c8fecc4077ba052390810c7430 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.625978 14812 raft_consensus.cc:738] T fe8d94c8fecc4077ba052390810c7430 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.626037 14853 leader_election.cc:290] T 238e68300f304266aaed94bc3b76a01d P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.626067 14812 consensus_queue.cc:260] T fe8d94c8fecc4077ba052390810c7430 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.626303 14812 ts_tablet_manager.cc:1428] T fe8d94c8fecc4077ba052390810c7430 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.626436 14812 tablet_bootstrap.cc:492] T 78b5e2e16442407880b75d256e6b2614 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.626675 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "238e68300f304266aaed94bc3b76a01d" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.626777 14689 raft_consensus.cc:2463] T 238e68300f304266aaed94bc3b76a01d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.627171 14493 leader_election.cc:304] T 238e68300f304266aaed94bc3b76a01d P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:22.627269 14812 tablet_bootstrap.cc:654] T 78b5e2e16442407880b75d256e6b2614 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.627298 14853 raft_consensus.cc:2798] T 238e68300f304266aaed94bc3b76a01d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.627342 14853 raft_consensus.cc:491] T 238e68300f304266aaed94bc3b76a01d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.627377 14853 raft_consensus.cc:3054] T 238e68300f304266aaed94bc3b76a01d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.628120 14412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "238e68300f304266aaed94bc3b76a01d" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.628221 14412 raft_consensus.cc:2463] T 238e68300f304266aaed94bc3b76a01d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
W20250115 03:06:22.633805 14165 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 71137cf62a374522af620e0ddb782fa8 consensus state has no leader
I20250115 03:06:22.635879 14171 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 6bd4e4200f9b489ea5dd2154aa6f382d (table default.Table_83b1fd83225a4454b29b7c61bc4639d4 [id=299ac71cfb98480f873eb659419e522c]). Expected version 1 got 0
I20250115 03:06:22.638672 14811 tablet_bootstrap.cc:654] T cef4442bfcfd4ad4b343a2785c3a9a74 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.639430 14171 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_660872e60c104e56be9287f66d74c387" } new_table_name: "default.Table_374565ae2c0d4226abd13599c2e4f096" modify_external_catalogs: true
I20250115 03:06:22.639998 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_5daefae8f3b94003a8941d6f94dda850"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.640434 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5daefae8f3b94003a8941d6f94dda850 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.640663 16411 raft_consensus.cc:491] T 015ec42caa1b468ea10f25c8f26e278a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.640758 16411 raft_consensus.cc:513] T 015ec42caa1b468ea10f25c8f26e278a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.641119 16411 leader_election.cc:290] T 015ec42caa1b468ea10f25c8f26e278a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.641315 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_91a3670ada684454977bfbc2bf08a599"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:22.641525 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015ec42caa1b468ea10f25c8f26e278a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:22.641582 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91a3670ada684454977bfbc2bf08a599 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.641606 14812 tablet_bootstrap.cc:492] T 78b5e2e16442407880b75d256e6b2614 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.641673 14812 ts_tablet_manager.cc:1397] T 78b5e2e16442407880b75d256e6b2614 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
W20250115 03:06:22.641860 14356 leader_election.cc:343] T 015ec42caa1b468ea10f25c8f26e278a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.642074 14812 raft_consensus.cc:357] T 78b5e2e16442407880b75d256e6b2614 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.642159 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015ec42caa1b468ea10f25c8f26e278a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.642171 14812 raft_consensus.cc:383] T 78b5e2e16442407880b75d256e6b2614 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20250115 03:06:22.642513 14358 leader_election.cc:343] T 015ec42caa1b468ea10f25c8f26e278a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.642570 14812 raft_consensus.cc:738] T 78b5e2e16442407880b75d256e6b2614 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.642575 14358 leader_election.cc:304] T 015ec42caa1b468ea10f25c8f26e278a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.642678 14812 consensus_queue.cc:260] T 78b5e2e16442407880b75d256e6b2614 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.642940 16411 raft_consensus.cc:2743] T 015ec42caa1b468ea10f25c8f26e278a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.642969 14812 ts_tablet_manager.cc:1428] T 78b5e2e16442407880b75d256e6b2614 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.643141 14812 tablet_bootstrap.cc:492] T f2b8bd783b614bf9b5526de197c51df3 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.644039 14812 tablet_bootstrap.cc:654] T f2b8bd783b614bf9b5526de197c51df3 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.645341 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_f9c13724577e4bbbb8c20649b1cc6eb8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.645627 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f9c13724577e4bbbb8c20649b1cc6eb8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.645853 16567 raft_consensus.cc:491] T 194d943d173340e7a8fba69967d9b090 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.645925 16567 raft_consensus.cc:513] T 194d943d173340e7a8fba69967d9b090 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.646066 16727 consensus_queue.cc:1035] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.646481 14412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "194d943d173340e7a8fba69967d9b090" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.646579 14412 raft_consensus.cc:2463] T 194d943d173340e7a8fba69967d9b090 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.646981 14553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "194d943d173340e7a8fba69967d9b090" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.646210 16567 leader_election.cc:290] T 194d943d173340e7a8fba69967d9b090 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:22.647603 14628 leader_election.cc:304] T 194d943d173340e7a8fba69967d9b090 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.647768 16727 raft_consensus.cc:2798] T 194d943d173340e7a8fba69967d9b090 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.647857 16727 raft_consensus.cc:491] T 194d943d173340e7a8fba69967d9b090 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.648020 16727 raft_consensus.cc:3054] T 194d943d173340e7a8fba69967d9b090 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.649760 14812 tablet_bootstrap.cc:492] T f2b8bd783b614bf9b5526de197c51df3 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.649833 14812 ts_tablet_manager.cc:1397] T f2b8bd783b614bf9b5526de197c51df3 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:22.650652 14812 raft_consensus.cc:357] T f2b8bd783b614bf9b5526de197c51df3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.651634 14812 raft_consensus.cc:383] T f2b8bd783b614bf9b5526de197c51df3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.652063 14812 raft_consensus.cc:738] T f2b8bd783b614bf9b5526de197c51df3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.652315 14812 consensus_queue.cc:260] T f2b8bd783b614bf9b5526de197c51df3 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.652448 14165 catalog_manager.cc:5526] T ad0e9216bd154f4a8a37874d3d8a08c2 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } }
I20250115 03:06:22.652742 14812 ts_tablet_manager.cc:1428] T f2b8bd783b614bf9b5526de197c51df3 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:22.652966 14812 tablet_bootstrap.cc:492] T cef4442bfcfd4ad4b343a2785c3a9a74 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.653957 14812 tablet_bootstrap.cc:654] T cef4442bfcfd4ad4b343a2785c3a9a74 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.663448 15617 tablet.cc:1721] T 676ec8b9dc8241d4ae40488d3ce8f61e P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20250115 03:06:22.666379 15617 tablet.cc:1721] T dda418ed197e4af2aa511d3f1851663e P e8df7247ee9c43e0aa18216da2661f75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20250115 03:06:22.667654 16573 raft_consensus.cc:513] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.668020 16573 leader_election.cc:290] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.668349 14811 tablet_bootstrap.cc:492] T cef4442bfcfd4ad4b343a2785c3a9a74 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.668419 14811 ts_tablet_manager.cc:1397] T cef4442bfcfd4ad4b343a2785c3a9a74 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.052s user 0.003s sys 0.000s
I20250115 03:06:22.668838 14811 raft_consensus.cc:357] T cef4442bfcfd4ad4b343a2785c3a9a74 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.668941 14811 raft_consensus.cc:383] T cef4442bfcfd4ad4b343a2785c3a9a74 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.668973 14811 raft_consensus.cc:738] T cef4442bfcfd4ad4b343a2785c3a9a74 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.669147 14689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f47da0e523d14388898a4be9c37bb809" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:22.669098 14811 consensus_queue.cc:260] T cef4442bfcfd4ad4b343a2785c3a9a74 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.669234 14689 raft_consensus.cc:3054] T f47da0e523d14388898a4be9c37bb809 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.669534 14811 ts_tablet_manager.cc:1428] T cef4442bfcfd4ad4b343a2785c3a9a74 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.669699 14811 tablet_bootstrap.cc:492] T 78b5e2e16442407880b75d256e6b2614 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.669901 16001 raft_consensus.cc:513] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.670240 16001 leader_election.cc:290] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.670478 14556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e41e31ffae4bb092cb08d83eda16b5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:22.670568 14556 raft_consensus.cc:3054] T a8e41e31ffae4bb092cb08d83eda16b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.670739 14412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e41e31ffae4bb092cb08d83eda16b5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:22.670820 14412 raft_consensus.cc:3054] T a8e41e31ffae4bb092cb08d83eda16b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.670967 14811 tablet_bootstrap.cc:654] T 78b5e2e16442407880b75d256e6b2614 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:22.671299 14493 leader_election.cc:343] T f47da0e523d14388898a4be9c37bb809 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: f47da0e523d14388898a4be9c37bb809
I20250115 03:06:22.671592 14420 raft_consensus.cc:2463] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:22.673157 14493 leader_election.cc:304] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.673418 16573 raft_consensus.cc:2798] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:22.673496 16573 raft_consensus.cc:695] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:22.673635 16573 consensus_queue.cc:237] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.676457 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_16f20fbeaf6c4ddf9712b913666e5a62"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:22.676793 14809 tablet_bootstrap.cc:492] T 7340ac89e5524c35948f3acc6c941cb5 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
W20250115 03:06:22.676817 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_16f20fbeaf6c4ddf9712b913666e5a62 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.676891 14809 ts_tablet_manager.cc:1397] T 7340ac89e5524c35948f3acc6c941cb5 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.063s user 0.002s sys 0.000s
I20250115 03:06:22.677634 14809 raft_consensus.cc:357] T 7340ac89e5524c35948f3acc6c941cb5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.677736 14809 raft_consensus.cc:383] T 7340ac89e5524c35948f3acc6c941cb5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.677764 14809 raft_consensus.cc:738] T 7340ac89e5524c35948f3acc6c941cb5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.677862 14809 consensus_queue.cc:260] T 7340ac89e5524c35948f3acc6c941cb5 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.678069 14809 ts_tablet_manager.cc:1428] T 7340ac89e5524c35948f3acc6c941cb5 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.678215 14809 tablet_bootstrap.cc:492] T b9f67c35591f4fdeb8a926a085171b49 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.678366 14823 ts_tablet_manager.cc:1918] T 0e1ce43146ed4e4d82a7c134216e2872 P 5c0dae7aff4b44c699275b95ba639770: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:22.678423 14823 log.cc:1198] T 0e1ce43146ed4e4d82a7c134216e2872 P 5c0dae7aff4b44c699275b95ba639770: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-2/wal/wals/0e1ce43146ed4e4d82a7c134216e2872
I20250115 03:06:22.678620 14823 ts_tablet_manager.cc:1939] T 0e1ce43146ed4e4d82a7c134216e2872 P 5c0dae7aff4b44c699275b95ba639770: Deleting consensus metadata
I20250115 03:06:22.679325 14809 tablet_bootstrap.cc:654] T b9f67c35591f4fdeb8a926a085171b49 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.680451 16763 consensus_queue.cc:1035] T ad0e9216bd154f4a8a37874d3d8a08c2 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.681455 14811 tablet_bootstrap.cc:492] T 78b5e2e16442407880b75d256e6b2614 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.681650 14811 ts_tablet_manager.cc:1397] T 78b5e2e16442407880b75d256e6b2614 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20250115 03:06:22.682379 14811 raft_consensus.cc:357] T 78b5e2e16442407880b75d256e6b2614 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.682533 14811 raft_consensus.cc:383] T 78b5e2e16442407880b75d256e6b2614 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.682606 14811 raft_consensus.cc:738] T 78b5e2e16442407880b75d256e6b2614 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.682771 14811 consensus_queue.cc:260] T 78b5e2e16442407880b75d256e6b2614 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.683149 14811 ts_tablet_manager.cc:1428] T 78b5e2e16442407880b75d256e6b2614 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.683362 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b898f210bcf24098907e4c59c1278e80"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:22.683463 14811 tablet_bootstrap.cc:492] T fe8d94c8fecc4077ba052390810c7430 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
W20250115 03:06:22.683667 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b898f210bcf24098907e4c59c1278e80 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.684381 14811 tablet_bootstrap.cc:654] T fe8d94c8fecc4077ba052390810c7430 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.684481 16573 raft_consensus.cc:491] T e47012a789bd469f9691e5a4162ff512 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.684552 16573 raft_consensus.cc:513] T e47012a789bd469f9691e5a4162ff512 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.684867 16573 leader_election.cc:290] T e47012a789bd469f9691e5a4162ff512 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.685349 14809 tablet_bootstrap.cc:492] T b9f67c35591f4fdeb8a926a085171b49 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.685437 14809 ts_tablet_manager.cc:1397] T b9f67c35591f4fdeb8a926a085171b49 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20250115 03:06:22.685405 14686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e47012a789bd469f9691e5a4162ff512" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.685477 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e47012a789bd469f9691e5a4162ff512" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.685518 14686 raft_consensus.cc:2463] T e47012a789bd469f9691e5a4162ff512 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.685568 14420 raft_consensus.cc:2463] T e47012a789bd469f9691e5a4162ff512 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.685847 14809 raft_consensus.cc:357] T b9f67c35591f4fdeb8a926a085171b49 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.685933 14809 raft_consensus.cc:383] T b9f67c35591f4fdeb8a926a085171b49 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.685911 14493 leader_election.cc:304] T e47012a789bd469f9691e5a4162ff512 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.685961 14809 raft_consensus.cc:738] T b9f67c35591f4fdeb8a926a085171b49 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.686069 14809 consensus_queue.cc:260] T b9f67c35591f4fdeb8a926a085171b49 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.686156 16573 raft_consensus.cc:2798] T e47012a789bd469f9691e5a4162ff512 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.686200 16573 raft_consensus.cc:491] T e47012a789bd469f9691e5a4162ff512 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.686234 16573 raft_consensus.cc:3054] T e47012a789bd469f9691e5a4162ff512 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.686278 14809 ts_tablet_manager.cc:1428] T b9f67c35591f4fdeb8a926a085171b49 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.686415 14809 tablet_bootstrap.cc:492] T 9c436975bd034068a6d58d5831db1896 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.687297 14809 tablet_bootstrap.cc:654] T 9c436975bd034068a6d58d5831db1896 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.690891 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0fb325f01dbc4e24990e73ac9d846313"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.691628 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0fb325f01dbc4e24990e73ac9d846313 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.694363 14811 tablet_bootstrap.cc:492] T fe8d94c8fecc4077ba052390810c7430 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.694455 14811 ts_tablet_manager.cc:1397] T fe8d94c8fecc4077ba052390810c7430 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20250115 03:06:22.694869 14811 raft_consensus.cc:357] T fe8d94c8fecc4077ba052390810c7430 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.694958 14811 raft_consensus.cc:383] T fe8d94c8fecc4077ba052390810c7430 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.694990 14811 raft_consensus.cc:738] T fe8d94c8fecc4077ba052390810c7430 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.695116 14811 consensus_queue.cc:260] T fe8d94c8fecc4077ba052390810c7430 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.695554 14811 ts_tablet_manager.cc:1428] T fe8d94c8fecc4077ba052390810c7430 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.695710 14811 tablet_bootstrap.cc:492] T f2b8bd783b614bf9b5526de197c51df3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.698531 14809 tablet_bootstrap.cc:492] T 9c436975bd034068a6d58d5831db1896 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.698606 14809 ts_tablet_manager.cc:1397] T 9c436975bd034068a6d58d5831db1896 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20250115 03:06:22.698969 14809 raft_consensus.cc:357] T 9c436975bd034068a6d58d5831db1896 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.699059 14809 raft_consensus.cc:383] T 9c436975bd034068a6d58d5831db1896 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.699083 14809 raft_consensus.cc:738] T 9c436975bd034068a6d58d5831db1896 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.699177 14809 consensus_queue.cc:260] T 9c436975bd034068a6d58d5831db1896 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.699397 14809 ts_tablet_manager.cc:1428] T 9c436975bd034068a6d58d5831db1896 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.699517 14809 tablet_bootstrap.cc:492] T 2b0e3df3c79c40ba958b6e2453fa666f P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.700309 14809 tablet_bootstrap.cc:654] T 2b0e3df3c79c40ba958b6e2453fa666f P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.701686 16721 log.cc:826] T 00000000000000000000000000000000 P 4225fe2fbfa84937bb41fed702ca66e4: Log is configured to *not* fsync() on all Append() calls
I20250115 03:06:22.698940 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_6d27f634a64f4390a1b36e24e867106b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.702154 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6d27f634a64f4390a1b36e24e867106b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.704737 16406 raft_consensus.cc:491] T bf6199cdd6e940d1970dd43d8090100d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.704810 16406 raft_consensus.cc:513] T bf6199cdd6e940d1970dd43d8090100d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.705117 16406 leader_election.cc:290] T bf6199cdd6e940d1970dd43d8090100d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.705474 14809 tablet_bootstrap.cc:492] T 2b0e3df3c79c40ba958b6e2453fa666f P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.705471 14686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6199cdd6e940d1970dd43d8090100d" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.705541 14809 ts_tablet_manager.cc:1397] T 2b0e3df3c79c40ba958b6e2453fa666f P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
W20250115 03:06:22.705809 14358 leader_election.cc:343] T bf6199cdd6e940d1970dd43d8090100d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.706243 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6199cdd6e940d1970dd43d8090100d" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.706310 14809 raft_consensus.cc:357] T 2b0e3df3c79c40ba958b6e2453fa666f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.706411 14809 raft_consensus.cc:383] T 2b0e3df3c79c40ba958b6e2453fa666f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.706454 14809 raft_consensus.cc:738] T 2b0e3df3c79c40ba958b6e2453fa666f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.706555 14809 consensus_queue.cc:260] T 2b0e3df3c79c40ba958b6e2453fa666f P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:22.706655 14356 leader_election.cc:343] T bf6199cdd6e940d1970dd43d8090100d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.706734 14356 leader_election.cc:304] T bf6199cdd6e940d1970dd43d8090100d P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.706823 14809 ts_tablet_manager.cc:1428] T 2b0e3df3c79c40ba958b6e2453fa666f P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:22.706866 16406 raft_consensus.cc:2743] T bf6199cdd6e940d1970dd43d8090100d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.706949 14811 tablet_bootstrap.cc:654] T f2b8bd783b614bf9b5526de197c51df3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.706974 14809 tablet_bootstrap.cc:492] T 15263a3b32be418288770ce348c082d7 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.707808 14809 tablet_bootstrap.cc:654] T 15263a3b32be418288770ce348c082d7 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.701390 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_1f46782fd85640939416bcf0de0784e0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.710047 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1f46782fd85640939416bcf0de0784e0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.710582 16558 raft_consensus.cc:491] T 78b5e2e16442407880b75d256e6b2614 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.710659 16558 raft_consensus.cc:513] T 78b5e2e16442407880b75d256e6b2614 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.710960 16558 leader_election.cc:290] T 78b5e2e16442407880b75d256e6b2614 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.711678 14550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78b5e2e16442407880b75d256e6b2614" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.711807 14550 raft_consensus.cc:2463] T 78b5e2e16442407880b75d256e6b2614 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.712137 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78b5e2e16442407880b75d256e6b2614" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.712260 14420 raft_consensus.cc:2463] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.712323 14626 leader_election.cc:304] T 78b5e2e16442407880b75d256e6b2614 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:22.712615 16001 raft_consensus.cc:2798] T 78b5e2e16442407880b75d256e6b2614 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.712671 16001 raft_consensus.cc:491] T 78b5e2e16442407880b75d256e6b2614 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.712719 16001 raft_consensus.cc:3054] T 78b5e2e16442407880b75d256e6b2614 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.715603 14171 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet bfeafbf380df41738ebbcc11c19a7f85 (table default.Table_374565ae2c0d4226abd13599c2e4f096 [id=03ee00da1e024ebab51b3a366d2c0cf3]). Expected version 3 got 2
I20250115 03:06:22.717259 14809 tablet_bootstrap.cc:492] T 15263a3b32be418288770ce348c082d7 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.717386 14809 ts_tablet_manager.cc:1397] T 15263a3b32be418288770ce348c082d7 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20250115 03:06:22.717815 14809 raft_consensus.cc:357] T 15263a3b32be418288770ce348c082d7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.717954 14809 raft_consensus.cc:383] T 15263a3b32be418288770ce348c082d7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.718014 14809 raft_consensus.cc:738] T 15263a3b32be418288770ce348c082d7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.718156 14809 consensus_queue.cc:260] T 15263a3b32be418288770ce348c082d7 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.718564 14809 ts_tablet_manager.cc:1428] T 15263a3b32be418288770ce348c082d7 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.718748 14809 tablet_bootstrap.cc:492] T 85fcafa24fa1448c995d30dd93b0737f P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.719585 14809 tablet_bootstrap.cc:654] T 85fcafa24fa1448c995d30dd93b0737f P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.728075 14809 tablet_bootstrap.cc:492] T 85fcafa24fa1448c995d30dd93b0737f P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.728145 14809 ts_tablet_manager.cc:1397] T 85fcafa24fa1448c995d30dd93b0737f P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:22.728540 14809 raft_consensus.cc:357] T 85fcafa24fa1448c995d30dd93b0737f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.728621 14809 raft_consensus.cc:383] T 85fcafa24fa1448c995d30dd93b0737f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.728646 14809 raft_consensus.cc:738] T 85fcafa24fa1448c995d30dd93b0737f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.728760 14809 consensus_queue.cc:260] T 85fcafa24fa1448c995d30dd93b0737f P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.729034 14809 ts_tablet_manager.cc:1428] T 85fcafa24fa1448c995d30dd93b0737f P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.729171 14809 tablet_bootstrap.cc:492] T 12b78b3abfe84714a97ecc89f2fb9e33 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.729759 14811 tablet_bootstrap.cc:492] T f2b8bd783b614bf9b5526de197c51df3 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.729838 14811 ts_tablet_manager.cc:1397] T f2b8bd783b614bf9b5526de197c51df3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.000s
I20250115 03:06:22.730382 16741 raft_consensus.cc:491] T d5e4743e016e49ed8ba26b1ac96b1470 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.730464 16741 raft_consensus.cc:513] T d5e4743e016e49ed8ba26b1ac96b1470 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.730795 16741 leader_election.cc:290] T d5e4743e016e49ed8ba26b1ac96b1470 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.731217 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6b937e191b8f4365a3d2ef4a4885db50" } modify_external_catalogs: true
I20250115 03:06:22.731333 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6b937e191b8f4365a3d2ef4a4885db50" } modify_external_catalogs: true
I20250115 03:06:22.731572 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e4743e016e49ed8ba26b1ac96b1470" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.731580 14686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e4743e016e49ed8ba26b1ac96b1470" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.731671 14420 raft_consensus.cc:2463] T d5e4743e016e49ed8ba26b1ac96b1470 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.731797 14811 raft_consensus.cc:357] T f2b8bd783b614bf9b5526de197c51df3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.731880 14811 raft_consensus.cc:383] T f2b8bd783b614bf9b5526de197c51df3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.731909 14811 raft_consensus.cc:738] T f2b8bd783b614bf9b5526de197c51df3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.732014 14811 consensus_queue.cc:260] T f2b8bd783b614bf9b5526de197c51df3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.732276 14811 ts_tablet_manager.cc:1428] T f2b8bd783b614bf9b5526de197c51df3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20250115 03:06:22.732411 14811 tablet_bootstrap.cc:492] T 87b234185fc64505b097f2ae7d52d813 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.733348 14811 tablet_bootstrap.cc:654] T 87b234185fc64505b097f2ae7d52d813 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.733347 14809 tablet_bootstrap.cc:654] T 12b78b3abfe84714a97ecc89f2fb9e33 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.733585 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0d782d9b6b9b4020b10ab81b8541375e" } new_table_name: "default.Table_536ecd46da8c4a05aff6cc5416c935e4" modify_external_catalogs: true
W20250115 03:06:22.734071 14493 leader_election.cc:343] T d5e4743e016e49ed8ba26b1ac96b1470 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.734443 14493 leader_election.cc:304] T d5e4743e016e49ed8ba26b1ac96b1470 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770
I20250115 03:06:22.734601 16748 raft_consensus.cc:2798] T d5e4743e016e49ed8ba26b1ac96b1470 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.734652 16748 raft_consensus.cc:491] T d5e4743e016e49ed8ba26b1ac96b1470 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.734709 16748 raft_consensus.cc:3054] T d5e4743e016e49ed8ba26b1ac96b1470 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.735850 16406 raft_consensus.cc:491] T 9bb0b7a8acfd40738abcc951936b1116 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.735949 16406 raft_consensus.cc:513] T 9bb0b7a8acfd40738abcc951936b1116 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.736276 16406 leader_election.cc:290] T 9bb0b7a8acfd40738abcc951936b1116 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.736629 14684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb0b7a8acfd40738abcc951936b1116" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.736989 14358 leader_election.cc:343] T 9bb0b7a8acfd40738abcc951936b1116 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.737073 14550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb0b7a8acfd40738abcc951936b1116" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:22.738333 14356 leader_election.cc:343] T 9bb0b7a8acfd40738abcc951936b1116 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.738399 14356 leader_election.cc:304] T 9bb0b7a8acfd40738abcc951936b1116 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.738525 16406 raft_consensus.cc:2743] T 9bb0b7a8acfd40738abcc951936b1116 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.739164 14171 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_0b6cd33c30014981b78899a738418cbf [id=63edd09bfaaa4d56b4314ce760c87d3d] alter complete (version 1)
W20250115 03:06:22.741627 14165 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bf6199cdd6e940d1970dd43d8090100d consensus state has no leader
I20250115 03:06:22.742554 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet edf17b41198b4f8fbe5db22c58aa6bb2
I20250115 03:06:22.745813 14811 tablet_bootstrap.cc:492] T 87b234185fc64505b097f2ae7d52d813 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.745898 14811 ts_tablet_manager.cc:1397] T 87b234185fc64505b097f2ae7d52d813 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.001s
I20250115 03:06:22.746263 14812 tablet_bootstrap.cc:492] T cef4442bfcfd4ad4b343a2785c3a9a74 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.746341 14812 ts_tablet_manager.cc:1397] T cef4442bfcfd4ad4b343a2785c3a9a74 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.093s user 0.002s sys 0.000s
I20250115 03:06:22.746313 14811 raft_consensus.cc:357] T 87b234185fc64505b097f2ae7d52d813 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.746393 14811 raft_consensus.cc:383] T 87b234185fc64505b097f2ae7d52d813 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.746423 14811 raft_consensus.cc:738] T 87b234185fc64505b097f2ae7d52d813 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.746526 14811 consensus_queue.cc:260] T 87b234185fc64505b097f2ae7d52d813 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.746783 14812 raft_consensus.cc:357] T cef4442bfcfd4ad4b343a2785c3a9a74 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.746876 14812 raft_consensus.cc:383] T cef4442bfcfd4ad4b343a2785c3a9a74 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.746905 14812 raft_consensus.cc:738] T cef4442bfcfd4ad4b343a2785c3a9a74 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.746945 14811 ts_tablet_manager.cc:1428] T 87b234185fc64505b097f2ae7d52d813 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:22.747010 14812 consensus_queue.cc:260] T cef4442bfcfd4ad4b343a2785c3a9a74 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.747092 14811 tablet_bootstrap.cc:492] T 041a30d6ee3247df883d00b5896b0eb0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.747280 14812 ts_tablet_manager.cc:1428] T cef4442bfcfd4ad4b343a2785c3a9a74 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.747433 14812 tablet_bootstrap.cc:492] T d5e4743e016e49ed8ba26b1ac96b1470 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.747974 14811 tablet_bootstrap.cc:654] T 041a30d6ee3247df883d00b5896b0eb0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.748328 14812 tablet_bootstrap.cc:654] T d5e4743e016e49ed8ba26b1ac96b1470 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.751415 14809 tablet_bootstrap.cc:492] T 12b78b3abfe84714a97ecc89f2fb9e33 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.751500 14809 ts_tablet_manager.cc:1397] T 12b78b3abfe84714a97ecc89f2fb9e33 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.022s user 0.002s sys 0.000s
I20250115 03:06:22.751966 14809 raft_consensus.cc:357] T 12b78b3abfe84714a97ecc89f2fb9e33 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.752053 14809 raft_consensus.cc:383] T 12b78b3abfe84714a97ecc89f2fb9e33 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.752086 14809 raft_consensus.cc:738] T 12b78b3abfe84714a97ecc89f2fb9e33 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.752192 14809 consensus_queue.cc:260] T 12b78b3abfe84714a97ecc89f2fb9e33 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.752472 14809 ts_tablet_manager.cc:1428] T 12b78b3abfe84714a97ecc89f2fb9e33 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.752614 14809 tablet_bootstrap.cc:492] T ec0f3424ebcb442597a94ccdbb25eeb6 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.753566 14809 tablet_bootstrap.cc:654] T ec0f3424ebcb442597a94ccdbb25eeb6 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.754137 16763 raft_consensus.cc:491] T fe8d94c8fecc4077ba052390810c7430 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.754215 16763 raft_consensus.cc:513] T fe8d94c8fecc4077ba052390810c7430 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.754603 16763 leader_election.cc:290] T fe8d94c8fecc4077ba052390810c7430 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:22.755790 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_47ede7f24148438ba5b165a821f676bc" } new_table_name: "default.Table_b8bae9b2aa5c41dab5608fcd84dbdc3a" modify_external_catalogs: true
I20250115 03:06:22.756582 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_82eeb56321174250b56cbfba4d9d1744"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:22.756700 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3f9710271c4c45a586f35f80a9e271c1" } modify_external_catalogs: true
I20250115 03:06:22.756770 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3f9710271c4c45a586f35f80a9e271c1" } modify_external_catalogs: true
I20250115 03:06:22.757026 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe8d94c8fecc4077ba052390810c7430" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.757120 14420 raft_consensus.cc:2463] T fe8d94c8fecc4077ba052390810c7430 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.757318 14812 tablet_bootstrap.cc:492] T d5e4743e016e49ed8ba26b1ac96b1470 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.757381 14628 leader_election.cc:304] T fe8d94c8fecc4077ba052390810c7430 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.757555 16763 raft_consensus.cc:2798] T fe8d94c8fecc4077ba052390810c7430 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.757601 16763 raft_consensus.cc:491] T fe8d94c8fecc4077ba052390810c7430 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.757637 16763 raft_consensus.cc:3054] T fe8d94c8fecc4077ba052390810c7430 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.757385 14812 ts_tablet_manager.cc:1397] T d5e4743e016e49ed8ba26b1ac96b1470 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20250115 03:06:22.758466 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe8d94c8fecc4077ba052390810c7430" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.758579 14555 raft_consensus.cc:2463] T fe8d94c8fecc4077ba052390810c7430 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.758550 14812 raft_consensus.cc:357] T d5e4743e016e49ed8ba26b1ac96b1470 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.758713 14812 raft_consensus.cc:383] T d5e4743e016e49ed8ba26b1ac96b1470 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.758747 14812 raft_consensus.cc:738] T d5e4743e016e49ed8ba26b1ac96b1470 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.758874 14812 consensus_queue.cc:260] T d5e4743e016e49ed8ba26b1ac96b1470 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.759164 14811 tablet_bootstrap.cc:492] T 041a30d6ee3247df883d00b5896b0eb0 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.759248 14811 ts_tablet_manager.cc:1397] T 041a30d6ee3247df883d00b5896b0eb0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.001s
I20250115 03:06:22.759367 14812 ts_tablet_manager.cc:1428] T d5e4743e016e49ed8ba26b1ac96b1470 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20250115 03:06:22.756901 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_82eeb56321174250b56cbfba4d9d1744 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.759584 14812 tablet_bootstrap.cc:492] T 78cc96de8b414ac19285338ccb1024e0 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.759671 14811 raft_consensus.cc:357] T 041a30d6ee3247df883d00b5896b0eb0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.759779 14811 raft_consensus.cc:383] T 041a30d6ee3247df883d00b5896b0eb0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.759814 14811 raft_consensus.cc:738] T 041a30d6ee3247df883d00b5896b0eb0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.759918 14811 consensus_queue.cc:260] T 041a30d6ee3247df883d00b5896b0eb0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.760191 14811 ts_tablet_manager.cc:1428] T 041a30d6ee3247df883d00b5896b0eb0 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.760352 14811 tablet_bootstrap.cc:492] T 2e7cf4fc3be8446292b0c91141eeadcc P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.760844 14812 tablet_bootstrap.cc:654] T 78cc96de8b414ac19285338ccb1024e0 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.761253 14811 tablet_bootstrap.cc:654] T 2e7cf4fc3be8446292b0c91141eeadcc P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.761440 14809 tablet_bootstrap.cc:492] T ec0f3424ebcb442597a94ccdbb25eeb6 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.761531 14809 ts_tablet_manager.cc:1397] T ec0f3424ebcb442597a94ccdbb25eeb6 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:22.761919 14809 raft_consensus.cc:357] T ec0f3424ebcb442597a94ccdbb25eeb6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.762015 14809 raft_consensus.cc:383] T ec0f3424ebcb442597a94ccdbb25eeb6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.762049 14809 raft_consensus.cc:738] T ec0f3424ebcb442597a94ccdbb25eeb6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.762162 14809 consensus_queue.cc:260] T ec0f3424ebcb442597a94ccdbb25eeb6 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.762415 14809 ts_tablet_manager.cc:1428] T ec0f3424ebcb442597a94ccdbb25eeb6 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:22.762570 14809 tablet_bootstrap.cc:492] T 9ab3c40051ad42f692cebbd96144b60a P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.763437 14809 tablet_bootstrap.cc:654] T 9ab3c40051ad42f692cebbd96144b60a P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.763447 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_59522d805bce49cd9b837dd585b3cfcf" } modify_external_catalogs: true
I20250115 03:06:22.763538 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_59522d805bce49cd9b837dd585b3cfcf" } modify_external_catalogs: true
I20250115 03:06:22.763811 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b6002fb5cc1248bba2eb7abbd5a5d407" } modify_external_catalogs: true
I20250115 03:06:22.763887 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b6002fb5cc1248bba2eb7abbd5a5d407" } modify_external_catalogs: true
I20250115 03:06:22.766738 14811 tablet_bootstrap.cc:492] T 2e7cf4fc3be8446292b0c91141eeadcc P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.766825 14811 ts_tablet_manager.cc:1397] T 2e7cf4fc3be8446292b0c91141eeadcc P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20250115 03:06:22.767179 14811 raft_consensus.cc:357] T 2e7cf4fc3be8446292b0c91141eeadcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.767256 14811 raft_consensus.cc:383] T 2e7cf4fc3be8446292b0c91141eeadcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.767282 14811 raft_consensus.cc:738] T 2e7cf4fc3be8446292b0c91141eeadcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.767385 14811 consensus_queue.cc:260] T 2e7cf4fc3be8446292b0c91141eeadcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.767635 14811 ts_tablet_manager.cc:1428] T 2e7cf4fc3be8446292b0c91141eeadcc P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.767769 14811 tablet_bootstrap.cc:492] T 1202288f2442477badeb35667dd08dab P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.768462 14811 tablet_bootstrap.cc:654] T 1202288f2442477badeb35667dd08dab P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.769554 16618 raft_consensus.cc:513] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.769867 16618 leader_election.cc:290] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.770192 14684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "103f78d0d9fc42308e1e23316895be33" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:22.770296 14684 raft_consensus.cc:3054] T 103f78d0d9fc42308e1e23316895be33 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.770704 15153 raft_consensus.cc:2743] T 6bd4e4200f9b489ea5dd2154aa6f382d P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.770941 14417 raft_consensus.cc:2463] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:22.772814 14388 tablet_service.cc:1467] Processing CreateTablet for tablet 2742a2d5a8244400b15c7c85f51b4bc7 (DEFAULT_TABLE table=default.Table_8b8c2a1323c94760a450e447b9a49789 [id=0411ce1e2061458dbc42348a839e6c70]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.773916 14388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2742a2d5a8244400b15c7c85f51b4bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:22.775557 14812 tablet_bootstrap.cc:492] T 78cc96de8b414ac19285338ccb1024e0 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.775631 14812 ts_tablet_manager.cc:1397] T 78cc96de8b414ac19285338ccb1024e0 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20250115 03:06:22.776053 14812 raft_consensus.cc:357] T 78cc96de8b414ac19285338ccb1024e0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.776139 14812 raft_consensus.cc:383] T 78cc96de8b414ac19285338ccb1024e0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.776167 14812 raft_consensus.cc:738] T 78cc96de8b414ac19285338ccb1024e0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.776273 14812 consensus_queue.cc:260] T 78cc96de8b414ac19285338ccb1024e0 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.776577 14812 ts_tablet_manager.cc:1428] T 78cc96de8b414ac19285338ccb1024e0 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.776742 14812 tablet_bootstrap.cc:492] T 041a30d6ee3247df883d00b5896b0eb0 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.777619 14812 tablet_bootstrap.cc:654] T 041a30d6ee3247df883d00b5896b0eb0 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.779268 16738 consensus_queue.cc:1035] T 6bd4e4200f9b489ea5dd2154aa6f382d P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.779666 16406 raft_consensus.cc:491] T b9f67c35591f4fdeb8a926a085171b49 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.779747 16406 raft_consensus.cc:513] T b9f67c35591f4fdeb8a926a085171b49 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.780062 16406 leader_election.cc:290] T b9f67c35591f4fdeb8a926a085171b49 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.780251 14493 leader_election.cc:304] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.780512 16738 raft_consensus.cc:2798] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:22.780581 16738 raft_consensus.cc:695] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:22.780712 16738 consensus_queue.cc:237] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.781812 14686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9f67c35591f4fdeb8a926a085171b49" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.782537 14358 leader_election.cc:343] T b9f67c35591f4fdeb8a926a085171b49 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.783041 14812 tablet_bootstrap.cc:492] T 041a30d6ee3247df883d00b5896b0eb0 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.783107 14812 ts_tablet_manager.cc:1397] T 041a30d6ee3247df883d00b5896b0eb0 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250115 03:06:22.783476 14812 raft_consensus.cc:357] T 041a30d6ee3247df883d00b5896b0eb0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.783550 14812 raft_consensus.cc:383] T 041a30d6ee3247df883d00b5896b0eb0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.783576 14812 raft_consensus.cc:738] T 041a30d6ee3247df883d00b5896b0eb0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.783655 14812 consensus_queue.cc:260] T 041a30d6ee3247df883d00b5896b0eb0 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.783890 14812 ts_tablet_manager.cc:1428] T 041a30d6ee3247df883d00b5896b0eb0 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.784011 14812 tablet_bootstrap.cc:492] T 87b234185fc64505b097f2ae7d52d813 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.784799 14812 tablet_bootstrap.cc:654] T 87b234185fc64505b097f2ae7d52d813 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.786978 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9f67c35591f4fdeb8a926a085171b49" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:22.787451 14493 leader_election.cc:343] T 103f78d0d9fc42308e1e23316895be33 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 103f78d0d9fc42308e1e23316895be33
W20250115 03:06:22.787971 14356 leader_election.cc:343] T b9f67c35591f4fdeb8a926a085171b49 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.788038 14356 leader_election.cc:304] T b9f67c35591f4fdeb8a926a085171b49 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.788157 16406 raft_consensus.cc:2743] T b9f67c35591f4fdeb8a926a085171b49 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.789594 14812 tablet_bootstrap.cc:492] T 87b234185fc64505b097f2ae7d52d813 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.789660 14812 ts_tablet_manager.cc:1397] T 87b234185fc64505b097f2ae7d52d813 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250115 03:06:22.790007 14812 raft_consensus.cc:357] T 87b234185fc64505b097f2ae7d52d813 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.790083 14812 raft_consensus.cc:383] T 87b234185fc64505b097f2ae7d52d813 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.790107 14812 raft_consensus.cc:738] T 87b234185fc64505b097f2ae7d52d813 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.790186 14812 consensus_queue.cc:260] T 87b234185fc64505b097f2ae7d52d813 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.790406 14812 ts_tablet_manager.cc:1428] T 87b234185fc64505b097f2ae7d52d813 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.790527 14812 tablet_bootstrap.cc:492] T 836541bcc82d4644b4c5eb2f3f25c030 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.791561 14812 tablet_bootstrap.cc:654] T 836541bcc82d4644b4c5eb2f3f25c030 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.795606 14812 tablet_bootstrap.cc:492] T 836541bcc82d4644b4c5eb2f3f25c030 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.795681 14812 ts_tablet_manager.cc:1397] T 836541bcc82d4644b4c5eb2f3f25c030 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20250115 03:06:22.796077 14812 raft_consensus.cc:357] T 836541bcc82d4644b4c5eb2f3f25c030 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.796162 14812 raft_consensus.cc:383] T 836541bcc82d4644b4c5eb2f3f25c030 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.796193 14812 raft_consensus.cc:738] T 836541bcc82d4644b4c5eb2f3f25c030 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.796291 14812 consensus_queue.cc:260] T 836541bcc82d4644b4c5eb2f3f25c030 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.796550 14812 ts_tablet_manager.cc:1428] T 836541bcc82d4644b4c5eb2f3f25c030 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.796706 14812 tablet_bootstrap.cc:492] T 1202288f2442477badeb35667dd08dab P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.797561 14812 tablet_bootstrap.cc:654] T 1202288f2442477badeb35667dd08dab P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.801414 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_2f4d77d2fd6d4dcb8bcee6f7214324cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.801764 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2f4d77d2fd6d4dcb8bcee6f7214324cd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.803280 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2e11c8e596be45598a86f0439162b164" } new_table_name: "default.Table_81d2375d57464237ab3c2b8b382b5cd8" modify_external_catalogs: true
I20250115 03:06:22.804903 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 9f88a8cc2995427eb2082117e553812c
I20250115 03:06:22.806491 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_36648b7bea684eecb64fad0ae41e7b53" } modify_external_catalogs: true
I20250115 03:06:22.806574 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_36648b7bea684eecb64fad0ae41e7b53" } modify_external_catalogs: true
I20250115 03:06:22.808034 14812 tablet_bootstrap.cc:492] T 1202288f2442477badeb35667dd08dab P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.808110 14812 ts_tablet_manager.cc:1397] T 1202288f2442477badeb35667dd08dab P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20250115 03:06:22.808470 14812 raft_consensus.cc:357] T 1202288f2442477badeb35667dd08dab P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.808560 14812 raft_consensus.cc:383] T 1202288f2442477badeb35667dd08dab P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.808593 14812 raft_consensus.cc:738] T 1202288f2442477badeb35667dd08dab P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.808712 14812 consensus_queue.cc:260] T 1202288f2442477badeb35667dd08dab P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.809000 14812 ts_tablet_manager.cc:1428] T 1202288f2442477badeb35667dd08dab P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.809154 14812 tablet_bootstrap.cc:492] T 2e7cf4fc3be8446292b0c91141eeadcc P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.809573 14809 tablet_bootstrap.cc:492] T 9ab3c40051ad42f692cebbd96144b60a P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.809648 14809 ts_tablet_manager.cc:1397] T 9ab3c40051ad42f692cebbd96144b60a P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.000s
I20250115 03:06:22.809736 14811 tablet_bootstrap.cc:492] T 1202288f2442477badeb35667dd08dab P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.809813 14811 ts_tablet_manager.cc:1397] T 1202288f2442477badeb35667dd08dab P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.042s user 0.002s sys 0.000s
I20250115 03:06:22.809953 14812 tablet_bootstrap.cc:654] T 2e7cf4fc3be8446292b0c91141eeadcc P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.810062 14809 raft_consensus.cc:357] T 9ab3c40051ad42f692cebbd96144b60a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.810151 14809 raft_consensus.cc:383] T 9ab3c40051ad42f692cebbd96144b60a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.810180 14809 raft_consensus.cc:738] T 9ab3c40051ad42f692cebbd96144b60a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.810250 14811 raft_consensus.cc:357] T 1202288f2442477badeb35667dd08dab P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.810287 14809 consensus_queue.cc:260] T 9ab3c40051ad42f692cebbd96144b60a P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.810364 14811 raft_consensus.cc:383] T 1202288f2442477badeb35667dd08dab P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.810400 14811 raft_consensus.cc:738] T 1202288f2442477badeb35667dd08dab P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.810557 14809 ts_tablet_manager.cc:1428] T 9ab3c40051ad42f692cebbd96144b60a P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.810519 14811 consensus_queue.cc:260] T 1202288f2442477badeb35667dd08dab P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.810703 14809 tablet_bootstrap.cc:492] T b9430861457247d9a324b43bd83fbae0 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.810792 14811 ts_tablet_manager.cc:1428] T 1202288f2442477badeb35667dd08dab P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.810946 14811 tablet_bootstrap.cc:492] T b3334f58b90b4947af52150546ccaa28 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.811548 14809 tablet_bootstrap.cc:654] T b9430861457247d9a324b43bd83fbae0 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.812559 16406 raft_consensus.cc:491] T fdaf7c090ef642b8b339f61a5d3fb06e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.812633 16406 raft_consensus.cc:513] T fdaf7c090ef642b8b339f61a5d3fb06e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.812918 16406 leader_election.cc:290] T fdaf7c090ef642b8b339f61a5d3fb06e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.813544 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdaf7c090ef642b8b339f61a5d3fb06e" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:22.813845 14356 leader_election.cc:343] T fdaf7c090ef642b8b339f61a5d3fb06e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.814815 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdaf7c090ef642b8b339f61a5d3fb06e" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.815141 14358 leader_election.cc:343] T fdaf7c090ef642b8b339f61a5d3fb06e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.815200 14358 leader_election.cc:304] T fdaf7c090ef642b8b339f61a5d3fb06e P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.815366 16406 raft_consensus.cc:2743] T fdaf7c090ef642b8b339f61a5d3fb06e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.819810 14168 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_f92234da7e9c4042a36f7acbd9c276d5 [id=cbab87c958a346199acdad65ab75be3c] alter complete (version 4)
I20250115 03:06:22.821408 14812 tablet_bootstrap.cc:492] T 2e7cf4fc3be8446292b0c91141eeadcc P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.821491 14812 ts_tablet_manager.cc:1397] T 2e7cf4fc3be8446292b0c91141eeadcc P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20250115 03:06:22.821851 14812 raft_consensus.cc:357] T 2e7cf4fc3be8446292b0c91141eeadcc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.821959 14812 raft_consensus.cc:383] T 2e7cf4fc3be8446292b0c91141eeadcc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.821992 14812 raft_consensus.cc:738] T 2e7cf4fc3be8446292b0c91141eeadcc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
W20250115 03:06:22.822121 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5b06b08f0a864629997efc0c98e84e04 consensus state has no leader
I20250115 03:06:22.822095 14812 consensus_queue.cc:260] T 2e7cf4fc3be8446292b0c91141eeadcc P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.822342 14812 ts_tablet_manager.cc:1428] T 2e7cf4fc3be8446292b0c91141eeadcc P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:22.822487 14812 tablet_bootstrap.cc:492] T ef0994b1e51b4c688354998730e7b743 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.823377 14812 tablet_bootstrap.cc:654] T ef0994b1e51b4c688354998730e7b743 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.824265 14809 tablet_bootstrap.cc:492] T b9430861457247d9a324b43bd83fbae0 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.824316 14812 tablet_bootstrap.cc:492] T ef0994b1e51b4c688354998730e7b743 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.824331 14809 ts_tablet_manager.cc:1397] T b9430861457247d9a324b43bd83fbae0 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20250115 03:06:22.824383 14812 ts_tablet_manager.cc:1397] T ef0994b1e51b4c688354998730e7b743 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:22.824710 14809 raft_consensus.cc:357] T b9430861457247d9a324b43bd83fbae0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.824797 14809 raft_consensus.cc:383] T b9430861457247d9a324b43bd83fbae0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.824824 14809 raft_consensus.cc:738] T b9430861457247d9a324b43bd83fbae0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.824779 14812 raft_consensus.cc:357] T ef0994b1e51b4c688354998730e7b743 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.824862 14812 raft_consensus.cc:383] T ef0994b1e51b4c688354998730e7b743 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.824892 14812 raft_consensus.cc:738] T ef0994b1e51b4c688354998730e7b743 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.824954 14809 consensus_queue.cc:260] T b9430861457247d9a324b43bd83fbae0 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.825110 14812 consensus_queue.cc:260] T ef0994b1e51b4c688354998730e7b743 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.825235 14809 ts_tablet_manager.cc:1428] T b9430861457247d9a324b43bd83fbae0 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.825434 14811 tablet_bootstrap.cc:654] T b3334f58b90b4947af52150546ccaa28 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.825430 14809 tablet_bootstrap.cc:492] T 4535ac88f78f48da83d9d2f951afdfa6 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.825444 14812 ts_tablet_manager.cc:1428] T ef0994b1e51b4c688354998730e7b743 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:22.825579 14812 tablet_bootstrap.cc:492] T d643bd0947c54bcfa79f15d88e1e53e4 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.826306 14809 tablet_bootstrap.cc:654] T 4535ac88f78f48da83d9d2f951afdfa6 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.826392 14812 tablet_bootstrap.cc:654] T d643bd0947c54bcfa79f15d88e1e53e4 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.827026 14811 tablet_bootstrap.cc:492] T b3334f58b90b4947af52150546ccaa28 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.827111 14811 ts_tablet_manager.cc:1397] T b3334f58b90b4947af52150546ccaa28 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20250115 03:06:22.827507 14811 raft_consensus.cc:357] T b3334f58b90b4947af52150546ccaa28 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.827611 14811 raft_consensus.cc:383] T b3334f58b90b4947af52150546ccaa28 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.827649 14811 raft_consensus.cc:738] T b3334f58b90b4947af52150546ccaa28 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.827800 14811 consensus_queue.cc:260] T b3334f58b90b4947af52150546ccaa28 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.828042 14811 ts_tablet_manager.cc:1428] T b3334f58b90b4947af52150546ccaa28 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.828075 16618 raft_consensus.cc:491] T 041a30d6ee3247df883d00b5896b0eb0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.828184 14811 tablet_bootstrap.cc:492] T 60997f625d074960ad75aef5ba14b9c2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.828164 16618 raft_consensus.cc:513] T 041a30d6ee3247df883d00b5896b0eb0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.828473 16618 leader_election.cc:290] T 041a30d6ee3247df883d00b5896b0eb0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.828838 14811 tablet_bootstrap.cc:654] T 60997f625d074960ad75aef5ba14b9c2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.833202 14417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "041a30d6ee3247df883d00b5896b0eb0" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.833319 14417 raft_consensus.cc:2463] T 041a30d6ee3247df883d00b5896b0eb0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.833451 16406 raft_consensus.cc:491] T b0a7def7d9a7418381b2d5a93a9a9265 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.833511 16406 raft_consensus.cc:513] T b0a7def7d9a7418381b2d5a93a9a9265 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.833786 14493 leader_election.cc:304] T 041a30d6ee3247df883d00b5896b0eb0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.833832 16406 leader_election.cc:290] T b0a7def7d9a7418381b2d5a93a9a9265 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.833930 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "041a30d6ee3247df883d00b5896b0eb0" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.834038 14691 raft_consensus.cc:2463] T 041a30d6ee3247df883d00b5896b0eb0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.834044 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 29c72f33a2034c4eae81c7dfcca27ca4
I20250115 03:06:22.834174 16738 raft_consensus.cc:2798] T 041a30d6ee3247df883d00b5896b0eb0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.834228 16738 raft_consensus.cc:491] T 041a30d6ee3247df883d00b5896b0eb0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.834266 16738 raft_consensus.cc:3054] T 041a30d6ee3247df883d00b5896b0eb0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.834482 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 8efa998709034fe9bb37c7c608c99e76
I20250115 03:06:22.834990 14811 tablet_bootstrap.cc:492] T 60997f625d074960ad75aef5ba14b9c2 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.835057 14811 ts_tablet_manager.cc:1397] T 60997f625d074960ad75aef5ba14b9c2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:22.835476 14811 raft_consensus.cc:357] T 60997f625d074960ad75aef5ba14b9c2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.835562 14811 raft_consensus.cc:383] T 60997f625d074960ad75aef5ba14b9c2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.835592 14811 raft_consensus.cc:738] T 60997f625d074960ad75aef5ba14b9c2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.835691 14811 consensus_queue.cc:260] T 60997f625d074960ad75aef5ba14b9c2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.835970 14811 ts_tablet_manager.cc:1428] T 60997f625d074960ad75aef5ba14b9c2 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.836114 14811 tablet_bootstrap.cc:492] T ef0994b1e51b4c688354998730e7b743 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.836395 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0a7def7d9a7418381b2d5a93a9a9265" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.836774 14358 leader_election.cc:343] T b0a7def7d9a7418381b2d5a93a9a9265 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.836874 14811 tablet_bootstrap.cc:654] T ef0994b1e51b4c688354998730e7b743 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.837244 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0a7def7d9a7418381b2d5a93a9a9265" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:22.837671 14356 leader_election.cc:343] T b0a7def7d9a7418381b2d5a93a9a9265 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.837736 14356 leader_election.cc:304] T b0a7def7d9a7418381b2d5a93a9a9265 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.837917 16406 raft_consensus.cc:2743] T b0a7def7d9a7418381b2d5a93a9a9265 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.838452 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 824fe68c383d4483991841a9f4c1b4ab
I20250115 03:06:22.839069 16741 raft_consensus.cc:491] T 83a4f95eb9494a5295a81a8a51f58bd3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.839138 16741 raft_consensus.cc:513] T 83a4f95eb9494a5295a81a8a51f58bd3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.839423 16741 leader_election.cc:290] T 83a4f95eb9494a5295a81a8a51f58bd3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.839565 14809 tablet_bootstrap.cc:492] T 4535ac88f78f48da83d9d2f951afdfa6 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.839625 14809 ts_tablet_manager.cc:1397] T 4535ac88f78f48da83d9d2f951afdfa6 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.000s
I20250115 03:06:22.839797 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83a4f95eb9494a5295a81a8a51f58bd3" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.839895 14691 raft_consensus.cc:2463] T 83a4f95eb9494a5295a81a8a51f58bd3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.839995 14809 raft_consensus.cc:357] T 4535ac88f78f48da83d9d2f951afdfa6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.840032 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83a4f95eb9494a5295a81a8a51f58bd3" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.840133 14422 raft_consensus.cc:2463] T 83a4f95eb9494a5295a81a8a51f58bd3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.840412 14493 leader_election.cc:304] T 83a4f95eb9494a5295a81a8a51f58bd3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.840631 16618 raft_consensus.cc:2798] T 83a4f95eb9494a5295a81a8a51f58bd3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.840678 16618 raft_consensus.cc:491] T 83a4f95eb9494a5295a81a8a51f58bd3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.840719 16618 raft_consensus.cc:3054] T 83a4f95eb9494a5295a81a8a51f58bd3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.840953 14812 tablet_bootstrap.cc:492] T d643bd0947c54bcfa79f15d88e1e53e4 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.841019 14812 ts_tablet_manager.cc:1397] T d643bd0947c54bcfa79f15d88e1e53e4 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.001s
I20250115 03:06:22.841430 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_d0c0059d665043bda7f63aaae45c079e" } new_table_name: "default.Table_1e82291a1f76499cbe13faaf6248ec82" modify_external_catalogs: true
I20250115 03:06:22.841394 14812 raft_consensus.cc:357] T d643bd0947c54bcfa79f15d88e1e53e4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.841567 14812 raft_consensus.cc:383] T d643bd0947c54bcfa79f15d88e1e53e4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.841598 14812 raft_consensus.cc:738] T d643bd0947c54bcfa79f15d88e1e53e4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.841688 16741 raft_consensus.cc:491] T de14b1d525cc42808500279ae6b60e80 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.841714 14812 consensus_queue.cc:260] T d643bd0947c54bcfa79f15d88e1e53e4 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.841753 16741 raft_consensus.cc:513] T de14b1d525cc42808500279ae6b60e80 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.841945 14812 ts_tablet_manager.cc:1428] T d643bd0947c54bcfa79f15d88e1e53e4 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.842031 16741 leader_election.cc:290] T de14b1d525cc42808500279ae6b60e80 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.842068 14812 tablet_bootstrap.cc:492] T bf6199cdd6e940d1970dd43d8090100d P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.842479 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de14b1d525cc42808500279ae6b60e80" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.842581 14691 raft_consensus.cc:2463] T de14b1d525cc42808500279ae6b60e80 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.842870 14493 leader_election.cc:304] T de14b1d525cc42808500279ae6b60e80 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:22.842895 14812 tablet_bootstrap.cc:654] T bf6199cdd6e940d1970dd43d8090100d P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.843061 16741 raft_consensus.cc:2798] T de14b1d525cc42808500279ae6b60e80 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.843112 16741 raft_consensus.cc:491] T de14b1d525cc42808500279ae6b60e80 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.843149 16741 raft_consensus.cc:3054] T de14b1d525cc42808500279ae6b60e80 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.840070 14809 raft_consensus.cc:383] T 4535ac88f78f48da83d9d2f951afdfa6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.843552 14809 raft_consensus.cc:738] T 4535ac88f78f48da83d9d2f951afdfa6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.843703 14809 consensus_queue.cc:260] T 4535ac88f78f48da83d9d2f951afdfa6 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.844069 14809 ts_tablet_manager.cc:1428] T 4535ac88f78f48da83d9d2f951afdfa6 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:22.844233 14809 tablet_bootstrap.cc:492] T cc01ad473e404e8ba26fd4cf7b0799f6 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.845388 14809 tablet_bootstrap.cc:654] T cc01ad473e404e8ba26fd4cf7b0799f6 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.845407 14811 tablet_bootstrap.cc:492] T ef0994b1e51b4c688354998730e7b743 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.845475 14811 ts_tablet_manager.cc:1397] T ef0994b1e51b4c688354998730e7b743 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20250115 03:06:22.845885 14811 raft_consensus.cc:357] T ef0994b1e51b4c688354998730e7b743 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.845976 14811 raft_consensus.cc:383] T ef0994b1e51b4c688354998730e7b743 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.846006 14811 raft_consensus.cc:738] T ef0994b1e51b4c688354998730e7b743 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.846102 14811 consensus_queue.cc:260] T ef0994b1e51b4c688354998730e7b743 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.846439 14811 ts_tablet_manager.cc:1428] T ef0994b1e51b4c688354998730e7b743 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.846601 14811 tablet_bootstrap.cc:492] T 836541bcc82d4644b4c5eb2f3f25c030 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.847429 14811 tablet_bootstrap.cc:654] T 836541bcc82d4644b4c5eb2f3f25c030 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.847499 16567 raft_consensus.cc:491] T 2e7cf4fc3be8446292b0c91141eeadcc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.847594 16567 raft_consensus.cc:513] T 2e7cf4fc3be8446292b0c91141eeadcc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.847923 16567 leader_election.cc:290] T 2e7cf4fc3be8446292b0c91141eeadcc P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.848296 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e7cf4fc3be8446292b0c91141eeadcc" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.848397 14555 raft_consensus.cc:2463] T 2e7cf4fc3be8446292b0c91141eeadcc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.848501 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e7cf4fc3be8446292b0c91141eeadcc" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.848604 14422 raft_consensus.cc:2463] T 2e7cf4fc3be8446292b0c91141eeadcc P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.848685 14626 leader_election.cc:304] T 2e7cf4fc3be8446292b0c91141eeadcc P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:22.849138 16567 raft_consensus.cc:2798] T 2e7cf4fc3be8446292b0c91141eeadcc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.849181 16567 raft_consensus.cc:491] T 2e7cf4fc3be8446292b0c91141eeadcc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.849205 16567 raft_consensus.cc:3054] T 2e7cf4fc3be8446292b0c91141eeadcc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.849354 14809 tablet_bootstrap.cc:492] T cc01ad473e404e8ba26fd4cf7b0799f6 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.849417 14809 ts_tablet_manager.cc:1397] T cc01ad473e404e8ba26fd4cf7b0799f6 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20250115 03:06:22.849768 14809 raft_consensus.cc:357] T cc01ad473e404e8ba26fd4cf7b0799f6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.849844 14809 raft_consensus.cc:383] T cc01ad473e404e8ba26fd4cf7b0799f6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.849869 14809 raft_consensus.cc:738] T cc01ad473e404e8ba26fd4cf7b0799f6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.849963 14809 consensus_queue.cc:260] T cc01ad473e404e8ba26fd4cf7b0799f6 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.850181 14809 ts_tablet_manager.cc:1428] T cc01ad473e404e8ba26fd4cf7b0799f6 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:22.850314 14809 tablet_bootstrap.cc:492] T daea48ecc69d441ab66fcb40ea2e7a66 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.851133 14809 tablet_bootstrap.cc:654] T daea48ecc69d441ab66fcb40ea2e7a66 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.853272 14809 tablet_bootstrap.cc:492] T daea48ecc69d441ab66fcb40ea2e7a66 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.853343 14809 ts_tablet_manager.cc:1397] T daea48ecc69d441ab66fcb40ea2e7a66 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:22.853417 14812 tablet_bootstrap.cc:492] T bf6199cdd6e940d1970dd43d8090100d P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.853494 14812 ts_tablet_manager.cc:1397] T bf6199cdd6e940d1970dd43d8090100d P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20250115 03:06:22.853648 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_78fb2ed5aedf41f18778eb7a136fd9b0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:22.853753 14809 raft_consensus.cc:357] T daea48ecc69d441ab66fcb40ea2e7a66 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.853840 14809 raft_consensus.cc:383] T daea48ecc69d441ab66fcb40ea2e7a66 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.853868 14809 raft_consensus.cc:738] T daea48ecc69d441ab66fcb40ea2e7a66 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.853875 14812 raft_consensus.cc:357] T bf6199cdd6e940d1970dd43d8090100d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.853963 14812 raft_consensus.cc:383] T bf6199cdd6e940d1970dd43d8090100d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.853992 14812 raft_consensus.cc:738] T bf6199cdd6e940d1970dd43d8090100d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.853971 14809 consensus_queue.cc:260] T daea48ecc69d441ab66fcb40ea2e7a66 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.854121 14812 consensus_queue.cc:260] T bf6199cdd6e940d1970dd43d8090100d P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.854192 16411 raft_consensus.cc:491] T 85fcafa24fa1448c995d30dd93b0737f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.854210 14809 ts_tablet_manager.cc:1428] T daea48ecc69d441ab66fcb40ea2e7a66 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:22.854257 16411 raft_consensus.cc:513] T 85fcafa24fa1448c995d30dd93b0737f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.854455 14812 ts_tablet_manager.cc:1428] T bf6199cdd6e940d1970dd43d8090100d P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.854553 16411 leader_election.cc:290] T 85fcafa24fa1448c995d30dd93b0737f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.854606 14812 tablet_bootstrap.cc:492] T b3334f58b90b4947af52150546ccaa28 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.855258 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85fcafa24fa1448c995d30dd93b0737f" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.855542 14812 tablet_bootstrap.cc:654] T b3334f58b90b4947af52150546ccaa28 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:22.855659 14356 leader_election.cc:343] T 85fcafa24fa1448c995d30dd93b0737f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.856024 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85fcafa24fa1448c995d30dd93b0737f" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.856372 14358 leader_election.cc:343] T 85fcafa24fa1448c995d30dd93b0737f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.856436 14358 leader_election.cc:304] T 85fcafa24fa1448c995d30dd93b0737f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.856616 16411 raft_consensus.cc:2743] T 85fcafa24fa1448c995d30dd93b0737f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.857327 14809 tablet_bootstrap.cc:492] T b142fa43a7734d1fb6e3d04bc720a74f P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.857580 14812 tablet_bootstrap.cc:492] T b3334f58b90b4947af52150546ccaa28 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.857708 14812 ts_tablet_manager.cc:1397] T b3334f58b90b4947af52150546ccaa28 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:22.858160 14809 tablet_bootstrap.cc:654] T b142fa43a7734d1fb6e3d04bc720a74f P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.858134 14812 raft_consensus.cc:357] T b3334f58b90b4947af52150546ccaa28 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.858314 14812 raft_consensus.cc:383] T b3334f58b90b4947af52150546ccaa28 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.858398 14812 raft_consensus.cc:738] T b3334f58b90b4947af52150546ccaa28 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
W20250115 03:06:22.858464 14552 consensus_meta.cc:287] T 72abf6acd9d54f3b983b7098aaf2691d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent flushing consensus metadata: real 0.558s user 0.001s sys 0.000s
I20250115 03:06:22.858526 14552 raft_consensus.cc:2463] T 72abf6acd9d54f3b983b7098aaf2691d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:22.858583 14812 consensus_queue.cc:260] T b3334f58b90b4947af52150546ccaa28 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.858783 14412 raft_consensus.cc:2463] T a8e41e31ffae4bb092cb08d83eda16b5 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:22.858871 16411 raft_consensus.cc:491] T b9430861457247d9a324b43bd83fbae0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.858938 16411 raft_consensus.cc:513] T b9430861457247d9a324b43bd83fbae0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.859198 16411 leader_election.cc:290] T b9430861457247d9a324b43bd83fbae0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.860543 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9430861457247d9a324b43bd83fbae0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.863996 14529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a97b4ad4327446cbae405f3d43a17781. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:22.864336 14690 consensus_meta.cc:287] T d891020ed0bb4fc49155bd9f2f6ed35c P 5c0dae7aff4b44c699275b95ba639770: Time spent flushing consensus metadata: real 0.565s user 0.001s sys 0.000s
I20250115 03:06:22.864387 14690 raft_consensus.cc:2463] T d891020ed0bb4fc49155bd9f2f6ed35c P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:22.865936 14493 leader_election.cc:304] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:22.866107 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9430861457247d9a324b43bd83fbae0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:22.866199 14358 leader_election.cc:343] T b9430861457247d9a324b43bd83fbae0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:22.866480 14356 leader_election.cc:343] T b9430861457247d9a324b43bd83fbae0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.866549 14356 leader_election.cc:304] T b9430861457247d9a324b43bd83fbae0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.866763 14628 leader_election.cc:304] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.866837 16406 raft_consensus.cc:491] T 15263a3b32be418288770ce348c082d7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.866830 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6c2031ba4104492f959f905cab4861dc" } new_table_name: "default.Table_0d2ba976c32e4230bd70105bb5704704" modify_external_catalogs: true
I20250115 03:06:22.866914 16558 raft_consensus.cc:2798] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:22.866902 16406 raft_consensus.cc:513] T 15263a3b32be418288770ce348c082d7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.866984 16558 raft_consensus.cc:695] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:22.867098 16558 consensus_queue.cc:237] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.867192 16406 leader_election.cc:290] T 15263a3b32be418288770ce348c082d7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.867269 16406 raft_consensus.cc:2743] T b9430861457247d9a324b43bd83fbae0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.867801 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_1ba1ad97ed8d4607882ced8381790a9e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:22.867874 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15263a3b32be418288770ce348c082d7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:22.868156 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1ba1ad97ed8d4607882ced8381790a9e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:22.868219 14356 leader_election.cc:343] T 15263a3b32be418288770ce348c082d7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.868397 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_91c8242231ef4fc3ba7adbd6a1de8689"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.868692 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91c8242231ef4fc3ba7adbd6a1de8689 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.868669 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15263a3b32be418288770ce348c082d7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.869032 16411 raft_consensus.cc:491] T 980b69b60a354519b0691ecb0bbd6c70 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.869109 16411 raft_consensus.cc:513] T 980b69b60a354519b0691ecb0bbd6c70 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.869378 16411 leader_election.cc:290] T 980b69b60a354519b0691ecb0bbd6c70 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:22.869045 14358 leader_election.cc:343] T 15263a3b32be418288770ce348c082d7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.869517 14358 leader_election.cc:304] T 15263a3b32be418288770ce348c082d7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
W20250115 03:06:22.870215 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_78fb2ed5aedf41f18778eb7a136fd9b0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.870247 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "980b69b60a354519b0691ecb0bbd6c70" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.870353 16800 raft_consensus.cc:491] T 87b234185fc64505b097f2ae7d52d813 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.870429 16800 raft_consensus.cc:513] T 87b234185fc64505b097f2ae7d52d813 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.870626 16411 raft_consensus.cc:2743] T 15263a3b32be418288770ce348c082d7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.870784 14812 ts_tablet_manager.cc:1428] T b3334f58b90b4947af52150546ccaa28 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.013s user 0.001s sys 0.000s
W20250115 03:06:22.870882 14356 leader_election.cc:343] T 980b69b60a354519b0691ecb0bbd6c70 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.870954 14812 tablet_bootstrap.cc:492] T 60997f625d074960ad75aef5ba14b9c2 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.870983 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b234185fc64505b097f2ae7d52d813" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.871069 14691 raft_consensus.cc:2463] T 87b234185fc64505b097f2ae7d52d813 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.871105 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "980b69b60a354519b0691ecb0bbd6c70" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.871321 14493 leader_election.cc:304] T 87b234185fc64505b097f2ae7d52d813 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
W20250115 03:06:22.871536 14358 leader_election.cc:343] T 980b69b60a354519b0691ecb0bbd6c70 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.871593 14358 leader_election.cc:304] T 980b69b60a354519b0691ecb0bbd6c70 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.871850 14812 tablet_bootstrap.cc:654] T 60997f625d074960ad75aef5ba14b9c2 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.871999 14811 tablet_bootstrap.cc:492] T 836541bcc82d4644b4c5eb2f3f25c030 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.872071 14811 ts_tablet_manager.cc:1397] T 836541bcc82d4644b4c5eb2f3f25c030 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.026s user 0.001s sys 0.000s
I20250115 03:06:22.872399 16411 raft_consensus.cc:2743] T 980b69b60a354519b0691ecb0bbd6c70 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.872499 14811 raft_consensus.cc:357] T 836541bcc82d4644b4c5eb2f3f25c030 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.872597 14811 raft_consensus.cc:383] T 836541bcc82d4644b4c5eb2f3f25c030 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.872628 14811 raft_consensus.cc:738] T 836541bcc82d4644b4c5eb2f3f25c030 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.872745 14811 consensus_queue.cc:260] T 836541bcc82d4644b4c5eb2f3f25c030 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.873098 14811 ts_tablet_manager.cc:1428] T 836541bcc82d4644b4c5eb2f3f25c030 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.873355 14811 tablet_bootstrap.cc:492] T bf6199cdd6e940d1970dd43d8090100d P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.873407 16800 leader_election.cc:290] T 87b234185fc64505b097f2ae7d52d813 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.873488 16800 raft_consensus.cc:2798] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:22.873540 16800 raft_consensus.cc:695] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:22.873570 14412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b234185fc64505b097f2ae7d52d813" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.873664 14412 raft_consensus.cc:2463] T 87b234185fc64505b097f2ae7d52d813 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.873648 16800 consensus_queue.cc:237] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.874215 14811 tablet_bootstrap.cc:654] T bf6199cdd6e940d1970dd43d8090100d P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.874228 16800 raft_consensus.cc:2798] T 87b234185fc64505b097f2ae7d52d813 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.874750 16800 raft_consensus.cc:491] T 87b234185fc64505b097f2ae7d52d813 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.874792 16800 raft_consensus.cc:3054] T 87b234185fc64505b097f2ae7d52d813 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.875984 14812 tablet_bootstrap.cc:492] T 60997f625d074960ad75aef5ba14b9c2 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.876071 14812 ts_tablet_manager.cc:1397] T 60997f625d074960ad75aef5ba14b9c2 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:22.876137 14164 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet b9430861457247d9a324b43bd83fbae0 (table default.Table_2ad8dffd9f274a8184b5a06c7254b1b5 [id=8e25d7d7bd99482788f22e3697de3929]). Expected version 1 got 0
I20250115 03:06:22.876463 14812 raft_consensus.cc:357] T 60997f625d074960ad75aef5ba14b9c2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.876577 14812 raft_consensus.cc:383] T 60997f625d074960ad75aef5ba14b9c2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.876605 14812 raft_consensus.cc:738] T 60997f625d074960ad75aef5ba14b9c2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.878090 14811 tablet_bootstrap.cc:492] T bf6199cdd6e940d1970dd43d8090100d P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.878177 14811 ts_tablet_manager.cc:1397] T bf6199cdd6e940d1970dd43d8090100d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20250115 03:06:22.878559 14811 raft_consensus.cc:357] T bf6199cdd6e940d1970dd43d8090100d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.878646 14811 raft_consensus.cc:383] T bf6199cdd6e940d1970dd43d8090100d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.878674 14811 raft_consensus.cc:738] T bf6199cdd6e940d1970dd43d8090100d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.878780 14811 consensus_queue.cc:260] T bf6199cdd6e940d1970dd43d8090100d P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.879052 14811 ts_tablet_manager.cc:1428] T bf6199cdd6e940d1970dd43d8090100d P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:22.879387 14811 tablet_bootstrap.cc:492] T d643bd0947c54bcfa79f15d88e1e53e4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.880265 14811 tablet_bootstrap.cc:654] T d643bd0947c54bcfa79f15d88e1e53e4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:22.880784 14167 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fe513ca696f14838b3886aac174f2cb5 consensus state has no leader
I20250115 03:06:22.881634 16558 raft_consensus.cc:491] T f2b8bd783b614bf9b5526de197c51df3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.881748 16558 raft_consensus.cc:513] T f2b8bd783b614bf9b5526de197c51df3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.882136 16558 leader_election.cc:290] T f2b8bd783b614bf9b5526de197c51df3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.882221 14812 consensus_queue.cc:260] T 60997f625d074960ad75aef5ba14b9c2 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.882550 14812 ts_tablet_manager.cc:1428] T 60997f625d074960ad75aef5ba14b9c2 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:22.882579 14412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2b8bd783b614bf9b5526de197c51df3" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.882687 14812 tablet_bootstrap.cc:492] T 980b69b60a354519b0691ecb0bbd6c70 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.882692 14412 raft_consensus.cc:2463] T f2b8bd783b614bf9b5526de197c51df3 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.883162 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2b8bd783b614bf9b5526de197c51df3" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.883271 14552 raft_consensus.cc:2463] T f2b8bd783b614bf9b5526de197c51df3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.883682 14812 tablet_bootstrap.cc:654] T 980b69b60a354519b0691ecb0bbd6c70 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.883873 14628 leader_election.cc:304] T f2b8bd783b614bf9b5526de197c51df3 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.884759 14811 tablet_bootstrap.cc:492] T d643bd0947c54bcfa79f15d88e1e53e4 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.884835 14811 ts_tablet_manager.cc:1397] T d643bd0947c54bcfa79f15d88e1e53e4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:22.885473 14811 raft_consensus.cc:357] T d643bd0947c54bcfa79f15d88e1e53e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.885564 14811 raft_consensus.cc:383] T d643bd0947c54bcfa79f15d88e1e53e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.885594 14811 raft_consensus.cc:738] T d643bd0947c54bcfa79f15d88e1e53e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.885704 14811 consensus_queue.cc:260] T d643bd0947c54bcfa79f15d88e1e53e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.885950 14811 ts_tablet_manager.cc:1428] T d643bd0947c54bcfa79f15d88e1e53e4 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:22.886093 14811 tablet_bootstrap.cc:492] T 980b69b60a354519b0691ecb0bbd6c70 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
W20250115 03:06:22.886391 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fe0dd7895a6547e8a105492083f2ec22 consensus state has no leader
I20250115 03:06:22.886407 14809 tablet_bootstrap.cc:492] T b142fa43a7734d1fb6e3d04bc720a74f P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.886489 14809 ts_tablet_manager.cc:1397] T b142fa43a7734d1fb6e3d04bc720a74f P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.000s
I20250115 03:06:22.886942 14811 tablet_bootstrap.cc:654] T 980b69b60a354519b0691ecb0bbd6c70 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.886907 14809 raft_consensus.cc:357] T b142fa43a7734d1fb6e3d04bc720a74f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.886999 14809 raft_consensus.cc:383] T b142fa43a7734d1fb6e3d04bc720a74f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.887034 14809 raft_consensus.cc:738] T b142fa43a7734d1fb6e3d04bc720a74f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.887138 14809 consensus_queue.cc:260] T b142fa43a7734d1fb6e3d04bc720a74f P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.887420 14809 ts_tablet_manager.cc:1428] T b142fa43a7734d1fb6e3d04bc720a74f P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.887598 14809 tablet_bootstrap.cc:492] T 293a76d0d49e4a12ab951cbc7311d0f2 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.888299 14812 tablet_bootstrap.cc:492] T 980b69b60a354519b0691ecb0bbd6c70 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.888381 14812 ts_tablet_manager.cc:1397] T 980b69b60a354519b0691ecb0bbd6c70 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:22.888479 14809 tablet_bootstrap.cc:654] T 293a76d0d49e4a12ab951cbc7311d0f2 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.888787 14812 raft_consensus.cc:357] T 980b69b60a354519b0691ecb0bbd6c70 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.888880 14812 raft_consensus.cc:383] T 980b69b60a354519b0691ecb0bbd6c70 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.888912 14812 raft_consensus.cc:738] T 980b69b60a354519b0691ecb0bbd6c70 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.889039 14812 consensus_queue.cc:260] T 980b69b60a354519b0691ecb0bbd6c70 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.889187 15153 raft_consensus.cc:2798] T f2b8bd783b614bf9b5526de197c51df3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.889235 15153 raft_consensus.cc:491] T f2b8bd783b614bf9b5526de197c51df3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.889269 15153 raft_consensus.cc:3054] T f2b8bd783b614bf9b5526de197c51df3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.889564 14812 ts_tablet_manager.cc:1428] T 980b69b60a354519b0691ecb0bbd6c70 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.889786 14812 tablet_bootstrap.cc:492] T fdaf7c090ef642b8b339f61a5d3fb06e P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.890717 14812 tablet_bootstrap.cc:654] T fdaf7c090ef642b8b339f61a5d3fb06e P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.891722 14811 tablet_bootstrap.cc:492] T 980b69b60a354519b0691ecb0bbd6c70 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.891799 14811 ts_tablet_manager.cc:1397] T 980b69b60a354519b0691ecb0bbd6c70 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20250115 03:06:22.892175 14811 raft_consensus.cc:357] T 980b69b60a354519b0691ecb0bbd6c70 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.892261 14811 raft_consensus.cc:383] T 980b69b60a354519b0691ecb0bbd6c70 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.892289 14811 raft_consensus.cc:738] T 980b69b60a354519b0691ecb0bbd6c70 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.892390 14811 consensus_queue.cc:260] T 980b69b60a354519b0691ecb0bbd6c70 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.892628 14811 ts_tablet_manager.cc:1428] T 980b69b60a354519b0691ecb0bbd6c70 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:22.892771 14811 tablet_bootstrap.cc:492] T 9bb0b7a8acfd40738abcc951936b1116 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.893980 14811 tablet_bootstrap.cc:654] T 9bb0b7a8acfd40738abcc951936b1116 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.894393 16411 raft_consensus.cc:491] T ec0f3424ebcb442597a94ccdbb25eeb6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.894480 16411 raft_consensus.cc:513] T ec0f3424ebcb442597a94ccdbb25eeb6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.894776 16810 raft_consensus.cc:491] T 836541bcc82d4644b4c5eb2f3f25c030 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.894806 16411 leader_election.cc:290] T ec0f3424ebcb442597a94ccdbb25eeb6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.894843 16810 raft_consensus.cc:513] T 836541bcc82d4644b4c5eb2f3f25c030 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.895130 16810 leader_election.cc:290] T 836541bcc82d4644b4c5eb2f3f25c030 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.895426 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec0f3424ebcb442597a94ccdbb25eeb6" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.895648 14812 tablet_bootstrap.cc:492] T fdaf7c090ef642b8b339f61a5d3fb06e P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.895714 14812 ts_tablet_manager.cc:1397] T fdaf7c090ef642b8b339f61a5d3fb06e P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
W20250115 03:06:22.895768 14358 leader_election.cc:343] T ec0f3424ebcb442597a94ccdbb25eeb6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.895978 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_3a10e88b551746deb92961d0cc131814"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:22.896059 14812 raft_consensus.cc:357] T fdaf7c090ef642b8b339f61a5d3fb06e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.896142 14812 raft_consensus.cc:383] T fdaf7c090ef642b8b339f61a5d3fb06e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.896167 14812 raft_consensus.cc:738] T fdaf7c090ef642b8b339f61a5d3fb06e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.896198 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec0f3424ebcb442597a94ccdbb25eeb6" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.896255 14812 consensus_queue.cc:260] T fdaf7c090ef642b8b339f61a5d3fb06e P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:22.896307 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a10e88b551746deb92961d0cc131814 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.896075 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836541bcc82d4644b4c5eb2f3f25c030" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.896472 14812 ts_tablet_manager.cc:1428] T fdaf7c090ef642b8b339f61a5d3fb06e P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.896494 14690 raft_consensus.cc:2463] T 836541bcc82d4644b4c5eb2f3f25c030 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.896600 14812 tablet_bootstrap.cc:492] T 9bb0b7a8acfd40738abcc951936b1116 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:22.896667 14356 leader_election.cc:343] T ec0f3424ebcb442597a94ccdbb25eeb6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.896736 14356 leader_election.cc:304] T ec0f3424ebcb442597a94ccdbb25eeb6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.897015 16411 raft_consensus.cc:2743] T ec0f3424ebcb442597a94ccdbb25eeb6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.897331 14493 leader_election.cc:304] T 836541bcc82d4644b4c5eb2f3f25c030 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:22.897329 14412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "836541bcc82d4644b4c5eb2f3f25c030" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.897403 16406 raft_consensus.cc:491] T 815c5e096e4d4b0cbfb7c15bac571b10 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.897426 14412 raft_consensus.cc:2463] T 836541bcc82d4644b4c5eb2f3f25c030 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.897475 16810 raft_consensus.cc:2798] T 836541bcc82d4644b4c5eb2f3f25c030 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.897472 16406 raft_consensus.cc:513] T 815c5e096e4d4b0cbfb7c15bac571b10 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.897517 16810 raft_consensus.cc:491] T 836541bcc82d4644b4c5eb2f3f25c030 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.897554 16810 raft_consensus.cc:3054] T 836541bcc82d4644b4c5eb2f3f25c030 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.897760 16406 leader_election.cc:290] T 815c5e096e4d4b0cbfb7c15bac571b10 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.897828 16406 raft_consensus.cc:491] T daea48ecc69d441ab66fcb40ea2e7a66 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.897876 16406 raft_consensus.cc:513] T daea48ecc69d441ab66fcb40ea2e7a66 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.898128 16406 leader_election.cc:290] T daea48ecc69d441ab66fcb40ea2e7a66 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.898573 14812 tablet_bootstrap.cc:654] T 9bb0b7a8acfd40738abcc951936b1116 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.899343 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daea48ecc69d441ab66fcb40ea2e7a66" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.899533 14811 tablet_bootstrap.cc:492] T 9bb0b7a8acfd40738abcc951936b1116 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:22.899601 14811 ts_tablet_manager.cc:1397] T 9bb0b7a8acfd40738abcc951936b1116 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
W20250115 03:06:22.899650 14358 leader_election.cc:343] T daea48ecc69d441ab66fcb40ea2e7a66 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.899976 14811 raft_consensus.cc:357] T 9bb0b7a8acfd40738abcc951936b1116 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.900064 14811 raft_consensus.cc:383] T 9bb0b7a8acfd40738abcc951936b1116 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.900094 14811 raft_consensus.cc:738] T 9bb0b7a8acfd40738abcc951936b1116 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.900197 14811 consensus_queue.cc:260] T 9bb0b7a8acfd40738abcc951936b1116 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.900450 14811 ts_tablet_manager.cc:1428] T 9bb0b7a8acfd40738abcc951936b1116 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:22.900595 14811 tablet_bootstrap.cc:492] T fdaf7c090ef642b8b339f61a5d3fb06e P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:22.901437 14811 tablet_bootstrap.cc:654] T fdaf7c090ef642b8b339f61a5d3fb06e P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.902549 14809 tablet_bootstrap.cc:492] T 293a76d0d49e4a12ab951cbc7311d0f2 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.902630 14809 ts_tablet_manager.cc:1397] T 293a76d0d49e4a12ab951cbc7311d0f2 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20250115 03:06:22.902989 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "815c5e096e4d4b0cbfb7c15bac571b10" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.903010 14809 raft_consensus.cc:357] T 293a76d0d49e4a12ab951cbc7311d0f2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.903095 14809 raft_consensus.cc:383] T 293a76d0d49e4a12ab951cbc7311d0f2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.903126 14809 raft_consensus.cc:738] T 293a76d0d49e4a12ab951cbc7311d0f2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.903228 14809 consensus_queue.cc:260] T 293a76d0d49e4a12ab951cbc7311d0f2 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
W20250115 03:06:22.903339 14358 leader_election.cc:343] T 815c5e096e4d4b0cbfb7c15bac571b10 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.903498 14809 ts_tablet_manager.cc:1428] T 293a76d0d49e4a12ab951cbc7311d0f2 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.903674 14809 tablet_bootstrap.cc:492] T d140be0fab694f59b8a535392970fad0 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.903954 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "815c5e096e4d4b0cbfb7c15bac571b10" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.904505 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ed04cecd900245b7a6b4864fd816043a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.904835 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed04cecd900245b7a6b4864fd816043a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.905155 14809 tablet_bootstrap.cc:654] T d140be0fab694f59b8a535392970fad0 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.905222 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0657abe634564cec8f7f3ebc673a0ed1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:22.905539 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0657abe634564cec8f7f3ebc673a0ed1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:22.907111 14356 leader_election.cc:343] T 815c5e096e4d4b0cbfb7c15bac571b10 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.907176 14356 leader_election.cc:304] T 815c5e096e4d4b0cbfb7c15bac571b10 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.907258 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daea48ecc69d441ab66fcb40ea2e7a66" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.907456 16411 raft_consensus.cc:2743] T 815c5e096e4d4b0cbfb7c15bac571b10 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.907806 16798 consensus_queue.cc:1035] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.909411 14812 tablet_bootstrap.cc:492] T 9bb0b7a8acfd40738abcc951936b1116 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.909489 14812 ts_tablet_manager.cc:1397] T 9bb0b7a8acfd40738abcc951936b1116 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.001s
I20250115 03:06:22.909870 14812 raft_consensus.cc:357] T 9bb0b7a8acfd40738abcc951936b1116 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.909956 14812 raft_consensus.cc:383] T 9bb0b7a8acfd40738abcc951936b1116 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.909982 14812 raft_consensus.cc:738] T 9bb0b7a8acfd40738abcc951936b1116 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
W20250115 03:06:22.910033 14356 leader_election.cc:343] T daea48ecc69d441ab66fcb40ea2e7a66 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.910092 14356 leader_election.cc:304] T daea48ecc69d441ab66fcb40ea2e7a66 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.910140 14812 consensus_queue.cc:260] T 9bb0b7a8acfd40738abcc951936b1116 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.910387 14812 ts_tablet_manager.cc:1428] T 9bb0b7a8acfd40738abcc951936b1116 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:22.910591 14812 tablet_bootstrap.cc:492] T b0a7def7d9a7418381b2d5a93a9a9265 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.910950 14809 tablet_bootstrap.cc:492] T d140be0fab694f59b8a535392970fad0 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.911016 14809 ts_tablet_manager.cc:1397] T d140be0fab694f59b8a535392970fad0 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20250115 03:06:22.911038 16411 raft_consensus.cc:2743] T daea48ecc69d441ab66fcb40ea2e7a66 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.911478 14809 raft_consensus.cc:357] T d140be0fab694f59b8a535392970fad0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.911563 14809 raft_consensus.cc:383] T d140be0fab694f59b8a535392970fad0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.911587 14809 raft_consensus.cc:738] T d140be0fab694f59b8a535392970fad0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.911666 14809 consensus_queue.cc:260] T d140be0fab694f59b8a535392970fad0 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.911887 14812 tablet_bootstrap.cc:654] T b0a7def7d9a7418381b2d5a93a9a9265 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.911892 14809 ts_tablet_manager.cc:1428] T d140be0fab694f59b8a535392970fad0 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.912014 14809 tablet_bootstrap.cc:492] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.912830 14809 tablet_bootstrap.cc:654] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.913980 14809 tablet_bootstrap.cc:492] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.914045 14809 ts_tablet_manager.cc:1397] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:22.914388 14809 raft_consensus.cc:357] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.914461 14809 raft_consensus.cc:383] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.914486 14809 raft_consensus.cc:738] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.914579 14809 consensus_queue.cc:260] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.914835 14809 ts_tablet_manager.cc:1428] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.914964 14809 tablet_bootstrap.cc:492] T fe513ca696f14838b3886aac174f2cb5 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.915740 14809 tablet_bootstrap.cc:654] T fe513ca696f14838b3886aac174f2cb5 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.918068 16806 consensus_queue.cc:1035] T 5a2894e4736b4191963a61d908bc673c P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.918664 16406 raft_consensus.cc:491] T 2b0e3df3c79c40ba958b6e2453fa666f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.918754 16406 raft_consensus.cc:513] T 2b0e3df3c79c40ba958b6e2453fa666f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.919057 16406 leader_election.cc:290] T 2b0e3df3c79c40ba958b6e2453fa666f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.919512 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b0e3df3c79c40ba958b6e2453fa666f" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.919863 16798 consensus_queue.cc:1035] T 7f9818d0493547358fd5654a01d4f456 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250115 03:06:22.920364 14358 leader_election.cc:343] T 2b0e3df3c79c40ba958b6e2453fa666f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.921399 16406 raft_consensus.cc:491] T 293a76d0d49e4a12ab951cbc7311d0f2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.921468 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b0e3df3c79c40ba958b6e2453fa666f" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.921478 16406 raft_consensus.cc:513] T 293a76d0d49e4a12ab951cbc7311d0f2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.921780 16406 leader_election.cc:290] T 293a76d0d49e4a12ab951cbc7311d0f2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:22.922053 14356 leader_election.cc:343] T 2b0e3df3c79c40ba958b6e2453fa666f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.922115 14356 leader_election.cc:304] T 2b0e3df3c79c40ba958b6e2453fa666f P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.922183 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "293a76d0d49e4a12ab951cbc7311d0f2" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.922277 16406 raft_consensus.cc:2743] T 2b0e3df3c79c40ba958b6e2453fa666f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:22.922557 14358 leader_election.cc:343] T 293a76d0d49e4a12ab951cbc7311d0f2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.922720 16558 consensus_queue.cc:1035] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.923162 16798 raft_consensus.cc:491] T 60997f625d074960ad75aef5ba14b9c2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.923242 16798 raft_consensus.cc:513] T 60997f625d074960ad75aef5ba14b9c2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.923566 16798 leader_election.cc:290] T 60997f625d074960ad75aef5ba14b9c2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.923745 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "293a76d0d49e4a12ab951cbc7311d0f2" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.923892 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60997f625d074960ad75aef5ba14b9c2" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.923992 14691 raft_consensus.cc:2463] T 60997f625d074960ad75aef5ba14b9c2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.924271 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_9090fd5bbab14ac59398d8e07b2816fb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:22.924314 14493 leader_election.cc:304] T 60997f625d074960ad75aef5ba14b9c2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
W20250115 03:06:22.924522 14356 leader_election.cc:343] T 293a76d0d49e4a12ab951cbc7311d0f2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.924579 14356 leader_election.cc:304] T 293a76d0d49e4a12ab951cbc7311d0f2 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
W20250115 03:06:22.924605 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9090fd5bbab14ac59398d8e07b2816fb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.924860 16406 raft_consensus.cc:2743] T 293a76d0d49e4a12ab951cbc7311d0f2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.924914 16806 raft_consensus.cc:2798] T 60997f625d074960ad75aef5ba14b9c2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.924983 16806 raft_consensus.cc:491] T 60997f625d074960ad75aef5ba14b9c2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.925017 16806 raft_consensus.cc:3054] T 60997f625d074960ad75aef5ba14b9c2 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.925280 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60997f625d074960ad75aef5ba14b9c2" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.925259 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d3c621d275d449c1b66d9f6e2c35238e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:22.925365 14422 raft_consensus.cc:2463] T 60997f625d074960ad75aef5ba14b9c2 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
W20250115 03:06:22.925544 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d3c621d275d449c1b66d9f6e2c35238e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:22.928345 14809 tablet_bootstrap.cc:492] T fe513ca696f14838b3886aac174f2cb5 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.928421 14809 ts_tablet_manager.cc:1397] T fe513ca696f14838b3886aac174f2cb5 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20250115 03:06:22.928809 14809 raft_consensus.cc:357] T fe513ca696f14838b3886aac174f2cb5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.928995 14809 raft_consensus.cc:383] T fe513ca696f14838b3886aac174f2cb5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.929033 14809 raft_consensus.cc:738] T fe513ca696f14838b3886aac174f2cb5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.929131 14809 consensus_queue.cc:260] T fe513ca696f14838b3886aac174f2cb5 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.929435 14809 ts_tablet_manager.cc:1428] T fe513ca696f14838b3886aac174f2cb5 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.929651 14809 tablet_bootstrap.cc:492] T 192a2efd3e74463e9ea5844eaaf53c97 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.930868 14809 tablet_bootstrap.cc:654] T 192a2efd3e74463e9ea5844eaaf53c97 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.931828 14809 tablet_bootstrap.cc:492] T 192a2efd3e74463e9ea5844eaaf53c97 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.931938 14809 ts_tablet_manager.cc:1397] T 192a2efd3e74463e9ea5844eaaf53c97 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:22.932374 14809 raft_consensus.cc:357] T 192a2efd3e74463e9ea5844eaaf53c97 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.932505 14809 raft_consensus.cc:383] T 192a2efd3e74463e9ea5844eaaf53c97 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.932564 14809 raft_consensus.cc:738] T 192a2efd3e74463e9ea5844eaaf53c97 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.932729 14809 consensus_queue.cc:260] T 192a2efd3e74463e9ea5844eaaf53c97 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.933166 14809 ts_tablet_manager.cc:1428] T 192a2efd3e74463e9ea5844eaaf53c97 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.933415 14809 tablet_bootstrap.cc:492] T cc328a01837a4c21abc38c7d85557e15 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.934298 14809 tablet_bootstrap.cc:654] T cc328a01837a4c21abc38c7d85557e15 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.934916 14812 tablet_bootstrap.cc:492] T b0a7def7d9a7418381b2d5a93a9a9265 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.935000 14812 ts_tablet_manager.cc:1397] T b0a7def7d9a7418381b2d5a93a9a9265 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.002s
I20250115 03:06:22.935382 14812 raft_consensus.cc:357] T b0a7def7d9a7418381b2d5a93a9a9265 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.935474 14812 raft_consensus.cc:383] T b0a7def7d9a7418381b2d5a93a9a9265 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.935501 14812 raft_consensus.cc:738] T b0a7def7d9a7418381b2d5a93a9a9265 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.935597 14812 consensus_queue.cc:260] T b0a7def7d9a7418381b2d5a93a9a9265 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.935863 14812 ts_tablet_manager.cc:1428] T b0a7def7d9a7418381b2d5a93a9a9265 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.936018 14812 tablet_bootstrap.cc:492] T f66defba523f4df996929f98c848f95a P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.936866 14812 tablet_bootstrap.cc:654] T f66defba523f4df996929f98c848f95a P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.937381 16817 consensus_queue.cc:1035] T 72abf6acd9d54f3b983b7098aaf2691d P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:22.938576 14812 tablet_bootstrap.cc:492] T f66defba523f4df996929f98c848f95a P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.938640 14812 ts_tablet_manager.cc:1397] T f66defba523f4df996929f98c848f95a P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20250115 03:06:22.939019 14812 raft_consensus.cc:357] T f66defba523f4df996929f98c848f95a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.939095 14812 raft_consensus.cc:383] T f66defba523f4df996929f98c848f95a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.939119 14812 raft_consensus.cc:738] T f66defba523f4df996929f98c848f95a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.939219 14812 consensus_queue.cc:260] T f66defba523f4df996929f98c848f95a P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.939579 16406 raft_consensus.cc:491] T 820d5af6daa74cadb53d2a5229854a60 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.939657 16406 raft_consensus.cc:513] T 820d5af6daa74cadb53d2a5229854a60 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.939761 16817 raft_consensus.cc:491] T fdaf7c090ef642b8b339f61a5d3fb06e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.939834 16817 raft_consensus.cc:513] T fdaf7c090ef642b8b339f61a5d3fb06e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.939949 16406 leader_election.cc:290] T 820d5af6daa74cadb53d2a5229854a60 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.940104 16817 leader_election.cc:290] T fdaf7c090ef642b8b339f61a5d3fb06e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.940404 14812 ts_tablet_manager.cc:1428] T f66defba523f4df996929f98c848f95a P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:22.940553 14812 tablet_bootstrap.cc:492] T 815c5e096e4d4b0cbfb7c15bac571b10 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.941027 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdaf7c090ef642b8b339f61a5d3fb06e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.941138 14420 raft_consensus.cc:2463] T fdaf7c090ef642b8b339f61a5d3fb06e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.941331 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820d5af6daa74cadb53d2a5229854a60" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.941407 14628 leader_election.cc:304] T fdaf7c090ef642b8b339f61a5d3fb06e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.941509 14812 tablet_bootstrap.cc:654] T 815c5e096e4d4b0cbfb7c15bac571b10 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.941563 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdaf7c090ef642b8b339f61a5d3fb06e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:22.941632 14358 leader_election.cc:343] T 820d5af6daa74cadb53d2a5229854a60 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.941890 16817 raft_consensus.cc:2798] T fdaf7c090ef642b8b339f61a5d3fb06e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.941944 16817 raft_consensus.cc:491] T fdaf7c090ef642b8b339f61a5d3fb06e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.941980 16817 raft_consensus.cc:3054] T fdaf7c090ef642b8b339f61a5d3fb06e P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.942030 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820d5af6daa74cadb53d2a5229854a60" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:22.942323 14626 leader_election.cc:343] T fdaf7c090ef642b8b339f61a5d3fb06e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.942541 14809 tablet_bootstrap.cc:492] T cc328a01837a4c21abc38c7d85557e15 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.942610 14809 ts_tablet_manager.cc:1397] T cc328a01837a4c21abc38c7d85557e15 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
W20250115 03:06:22.942734 14356 leader_election.cc:343] T 820d5af6daa74cadb53d2a5229854a60 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.942799 14356 leader_election.cc:304] T 820d5af6daa74cadb53d2a5229854a60 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.942968 16406 raft_consensus.cc:2743] T 820d5af6daa74cadb53d2a5229854a60 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.943049 14809 raft_consensus.cc:357] T cc328a01837a4c21abc38c7d85557e15 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.943215 14809 raft_consensus.cc:383] T cc328a01837a4c21abc38c7d85557e15 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.943291 14809 raft_consensus.cc:738] T cc328a01837a4c21abc38c7d85557e15 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.943290 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_3ca292c29f184579b5fd9fe897150035" } new_table_name: "default.Table_e991beb7e41d4a8891dc78ef21b17634" modify_external_catalogs: true
I20250115 03:06:22.943509 14809 consensus_queue.cc:260] T cc328a01837a4c21abc38c7d85557e15 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.943912 14809 ts_tablet_manager.cc:1428] T cc328a01837a4c21abc38c7d85557e15 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.944121 14809 tablet_bootstrap.cc:492] T c8ac40736c74479dab284ffc7a394883 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.944607 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_91a3670ada684454977bfbc2bf08a599" } modify_external_catalogs: true
I20250115 03:06:22.944707 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_91a3670ada684454977bfbc2bf08a599" } modify_external_catalogs: true
I20250115 03:06:22.945113 14809 tablet_bootstrap.cc:654] T c8ac40736c74479dab284ffc7a394883 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.946023 14812 tablet_bootstrap.cc:492] T 815c5e096e4d4b0cbfb7c15bac571b10 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:22.946089 14812 ts_tablet_manager.cc:1397] T 815c5e096e4d4b0cbfb7c15bac571b10 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250115 03:06:22.946349 16558 raft_consensus.cc:491] T bf6199cdd6e940d1970dd43d8090100d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.946419 16558 raft_consensus.cc:513] T bf6199cdd6e940d1970dd43d8090100d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.946763 16558 leader_election.cc:290] T bf6199cdd6e940d1970dd43d8090100d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.947161 14809 tablet_bootstrap.cc:492] T c8ac40736c74479dab284ffc7a394883 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.947268 14809 ts_tablet_manager.cc:1397] T c8ac40736c74479dab284ffc7a394883 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20250115 03:06:22.947151 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6199cdd6e940d1970dd43d8090100d" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.946432 14812 raft_consensus.cc:357] T 815c5e096e4d4b0cbfb7c15bac571b10 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.947578 14420 raft_consensus.cc:2463] T bf6199cdd6e940d1970dd43d8090100d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.947626 14812 raft_consensus.cc:383] T 815c5e096e4d4b0cbfb7c15bac571b10 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.947654 14812 raft_consensus.cc:738] T 815c5e096e4d4b0cbfb7c15bac571b10 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.947661 14809 raft_consensus.cc:357] T c8ac40736c74479dab284ffc7a394883 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.947754 14809 raft_consensus.cc:383] T c8ac40736c74479dab284ffc7a394883 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.947786 14809 raft_consensus.cc:738] T c8ac40736c74479dab284ffc7a394883 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.947751 14812 consensus_queue.cc:260] T 815c5e096e4d4b0cbfb7c15bac571b10 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.947770 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6199cdd6e940d1970dd43d8090100d" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.947856 14552 raft_consensus.cc:2463] T bf6199cdd6e940d1970dd43d8090100d P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.947885 14809 consensus_queue.cc:260] T c8ac40736c74479dab284ffc7a394883 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.947994 14812 ts_tablet_manager.cc:1428] T 815c5e096e4d4b0cbfb7c15bac571b10 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:22.948117 14809 ts_tablet_manager.cc:1428] T c8ac40736c74479dab284ffc7a394883 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:22.948138 14812 tablet_bootstrap.cc:492] T 820d5af6daa74cadb53d2a5229854a60 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:22.948259 14809 tablet_bootstrap.cc:492] T d3728e87eecc4bc48bc1944907ceb8bd P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.948995 14812 tablet_bootstrap.cc:654] T 820d5af6daa74cadb53d2a5229854a60 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.949188 14809 tablet_bootstrap.cc:654] T d3728e87eecc4bc48bc1944907ceb8bd P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.949278 16411 raft_consensus.cc:491] T fe513ca696f14838b3886aac174f2cb5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.949342 16411 raft_consensus.cc:513] T fe513ca696f14838b3886aac174f2cb5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.949654 16411 leader_election.cc:290] T fe513ca696f14838b3886aac174f2cb5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.949697 14628 leader_election.cc:304] T bf6199cdd6e940d1970dd43d8090100d P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.950068 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe513ca696f14838b3886aac174f2cb5" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.950289 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe513ca696f14838b3886aac174f2cb5" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.950320 16558 raft_consensus.cc:2798] T bf6199cdd6e940d1970dd43d8090100d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.950361 16558 raft_consensus.cc:491] T bf6199cdd6e940d1970dd43d8090100d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.950400 16558 raft_consensus.cc:3054] T bf6199cdd6e940d1970dd43d8090100d P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:22.950510 14358 leader_election.cc:343] T fe513ca696f14838b3886aac174f2cb5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:22.950733 14356 leader_election.cc:343] T fe513ca696f14838b3886aac174f2cb5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.950795 14356 leader_election.cc:304] T fe513ca696f14838b3886aac174f2cb5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.950911 16406 raft_consensus.cc:2743] T fe513ca696f14838b3886aac174f2cb5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.951555 14809 tablet_bootstrap.cc:492] T d3728e87eecc4bc48bc1944907ceb8bd P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.951622 14809 ts_tablet_manager.cc:1397] T d3728e87eecc4bc48bc1944907ceb8bd P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20250115 03:06:22.951982 14809 raft_consensus.cc:357] T d3728e87eecc4bc48bc1944907ceb8bd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.952059 14809 raft_consensus.cc:383] T d3728e87eecc4bc48bc1944907ceb8bd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.952082 14809 raft_consensus.cc:738] T d3728e87eecc4bc48bc1944907ceb8bd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.952170 14809 consensus_queue.cc:260] T d3728e87eecc4bc48bc1944907ceb8bd P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.952368 14809 ts_tablet_manager.cc:1428] T d3728e87eecc4bc48bc1944907ceb8bd P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:22.952486 14809 tablet_bootstrap.cc:492] T 2f0779261368409c9e7b12088f88cb16 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.952502 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 84c592ced5b443869240eff55fe9d549
I20250115 03:06:22.953323 14809 tablet_bootstrap.cc:654] T 2f0779261368409c9e7b12088f88cb16 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.954846 14809 tablet_bootstrap.cc:492] T 2f0779261368409c9e7b12088f88cb16 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.954911 14809 ts_tablet_manager.cc:1397] T 2f0779261368409c9e7b12088f88cb16 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20250115 03:06:22.955258 14809 raft_consensus.cc:357] T 2f0779261368409c9e7b12088f88cb16 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.955335 14809 raft_consensus.cc:383] T 2f0779261368409c9e7b12088f88cb16 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.955360 14809 raft_consensus.cc:738] T 2f0779261368409c9e7b12088f88cb16 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.955462 14809 consensus_queue.cc:260] T 2f0779261368409c9e7b12088f88cb16 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.955672 14809 ts_tablet_manager.cc:1428] T 2f0779261368409c9e7b12088f88cb16 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:22.955801 14809 tablet_bootstrap.cc:492] T 7ac903c81a1f481ab587e6772c306a96 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.956563 14809 tablet_bootstrap.cc:654] T 7ac903c81a1f481ab587e6772c306a96 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:22.960422 16189 consensus_meta.cc:287] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent flushing consensus metadata: real 0.577s user 0.001s sys 0.000s
I20250115 03:06:22.960487 16189 raft_consensus.cc:513] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.960831 16189 leader_election.cc:290] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.961045 14416 raft_consensus.cc:2463] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 1.
I20250115 03:06:22.961603 14628 leader_election.cc:304] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
W20250115 03:06:22.961650 16557 consensus_meta.cc:287] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770: Time spent flushing consensus metadata: real 0.634s user 0.000s sys 0.001s
I20250115 03:06:22.961730 16557 raft_consensus.cc:513] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.961778 16840 raft_consensus.cc:2798] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:22.961855 16840 raft_consensus.cc:695] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770 [term 1 LEADER]: Becoming Leader. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Running, Role: LEADER
I20250115 03:06:22.961963 16840 consensus_queue.cc:237] T 0b5fd91a0ea64b3b85b7571e5daa08b2 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.962028 16557 leader_election.cc:290] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.963183 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet f66defba523f4df996929f98c848f95a with delete_type TABLET_DATA_DELETED (replaced by tablet b9979224e3874a7982e1bada8395c921) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.963289 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c4ce5c2921341ce837ee4f5f7d131b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.963352 14396 tablet_service.cc:1514] Processing DeleteTablet for tablet 015ec42caa1b468ea10f25c8f26e278a with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.963439 14552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "398b93f1db0741a9b54837bca43b1bad" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:22.963536 14552 raft_consensus.cc:3054] T 398b93f1db0741a9b54837bca43b1bad P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.963551 14396 tablet_service.cc:1467] Processing CreateTablet for tablet fc4c45cf04854df5af831417623f14f1 (DEFAULT_TABLE table=default.Table_48bca7fdfb0544a59c84fc547c19e499 [id=59dba0a633d2416eb65151bc64e41490]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.964015 14809 tablet_bootstrap.cc:492] T 7ac903c81a1f481ab587e6772c306a96 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.964102 14809 ts_tablet_manager.cc:1397] T 7ac903c81a1f481ab587e6772c306a96 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20250115 03:06:22.964562 14809 raft_consensus.cc:357] T 7ac903c81a1f481ab587e6772c306a96 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.964648 14809 raft_consensus.cc:383] T 7ac903c81a1f481ab587e6772c306a96 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.964679 14809 raft_consensus.cc:738] T 7ac903c81a1f481ab587e6772c306a96 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.964805 14809 consensus_queue.cc:260] T 7ac903c81a1f481ab587e6772c306a96 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.965072 14809 ts_tablet_manager.cc:1428] T 7ac903c81a1f481ab587e6772c306a96 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:22.965219 14809 tablet_bootstrap.cc:492] T 695927c78bb74b669f2aeca1abf1b648 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.966055 14809 tablet_bootstrap.cc:654] T 695927c78bb74b669f2aeca1abf1b648 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.966719 16406 raft_consensus.cc:491] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.966786 16406 raft_consensus.cc:513] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.967095 16406 leader_election.cc:290] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.967383 14402 tablet_service.cc:1467] Processing CreateTablet for tablet 47fed237bfc94d7e819557e9fce0ce95 (DEFAULT_TABLE table=default.Table_3f9adcb39f2945e88044d6f0ed9797ad [id=5fefdddd34b441ed8d5c93eb06fcda0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.968087 14809 tablet_bootstrap.cc:492] T 695927c78bb74b669f2aeca1abf1b648 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.968152 14809 ts_tablet_manager.cc:1397] T 695927c78bb74b669f2aeca1abf1b648 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20250115 03:06:22.968461 14394 tablet_service.cc:1467] Processing CreateTablet for tablet a3ddba7630904f16a36698c1e5e7e9cf (DEFAULT_TABLE table=default.Table_1288c4fed9bb4bee8a2a567a7b16990d [id=342cd63e28b34a4c8cf552111b3325e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.968545 14550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66defba523f4df996929f98c848f95a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.968530 14809 raft_consensus.cc:357] T 695927c78bb74b669f2aeca1abf1b648 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.968617 14809 raft_consensus.cc:383] T 695927c78bb74b669f2aeca1abf1b648 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.968647 14809 raft_consensus.cc:738] T 695927c78bb74b669f2aeca1abf1b648 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.968776 14809 consensus_queue.cc:260] T 695927c78bb74b669f2aeca1abf1b648 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.969015 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c436975bd034068a6d58d5831db1896 with delete_type TABLET_DATA_DELETED (replaced by tablet 802a007b8d154198b291aaf9530161b9) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.969069 14809 ts_tablet_manager.cc:1428] T 695927c78bb74b669f2aeca1abf1b648 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:22.969101 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet b9f67c35591f4fdeb8a926a085171b49 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.969158 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet edf17b41198b4f8fbe5db22c58aa6bb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.969198 14809 tablet_bootstrap.cc:492] T c127b5ac695a456aa38479a5f9a8fe82 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.969218 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 85fcafa24fa1448c995d30dd93b0737f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.969271 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 15263a3b32be418288770ce348c082d7 with delete_type TABLET_DATA_DELETED (replaced by tablet d1daa457581a48ceb18bf617a60a0802) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.969341 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b0e3df3c79c40ba958b6e2453fa666f with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.969404 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 12b78b3abfe84714a97ecc89f2fb9e33 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:22.969578 16840 raft_consensus.cc:491] T cef4442bfcfd4ad4b343a2785c3a9a74 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.969595 14393 tablet_service.cc:1467] Processing CreateTablet for tablet 1a179a7c7aab4831a42e9f25f22163ad (DEFAULT_TABLE table=default.Table_1f46782fd85640939416bcf0de0784e0 [id=02c20c94390c4ba1a7bc2a0c1611fb62]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.969655 16840 raft_consensus.cc:513] T cef4442bfcfd4ad4b343a2785c3a9a74 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.969964 14809 tablet_bootstrap.cc:654] T c127b5ac695a456aa38479a5f9a8fe82 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:22.969962 16840 leader_election.cc:290] T cef4442bfcfd4ad4b343a2785c3a9a74 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.970645 14388 tablet_service.cc:1467] Processing CreateTablet for tablet 1822e678e3a44f24a6f4d4b13e86e8aa (DEFAULT_TABLE table=default.Table_0fb325f01dbc4e24990e73ac9d846313 [id=c32d1b3b38d2471092547a983a7ce6f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.968461 14401 tablet_service.cc:1467] Processing CreateTablet for tablet ad5ada0afc2d4a4d94595fa951fcda1f (DEFAULT_TABLE table=default.Table_55bbd621f94d45fc8b93a7211619f08a [id=8ad688c2aa5048db9a71e6b7611e98ed]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:22.971024 14356 leader_election.cc:343] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.971263 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 84c592ced5b443869240eff55fe9d549 (DEFAULT_TABLE table=default.Table_91a3670ada684454977bfbc2bf08a599 [id=fca39b0eb2374166b5f78b1a94c66451]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.971276 14550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef4442bfcfd4ad4b343a2785c3a9a74" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:22.971357 14628 leader_election.cc:343] T 398b93f1db0741a9b54837bca43b1bad P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 398b93f1db0741a9b54837bca43b1bad
I20250115 03:06:22.971390 14550 raft_consensus.cc:2463] T cef4442bfcfd4ad4b343a2785c3a9a74 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.971645 14626 leader_election.cc:304] T cef4442bfcfd4ad4b343a2785c3a9a74 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:22.971952 16840 raft_consensus.cc:491] T d643bd0947c54bcfa79f15d88e1e53e4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.972015 16840 raft_consensus.cc:513] T d643bd0947c54bcfa79f15d88e1e53e4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.972263 16840 leader_election.cc:290] T d643bd0947c54bcfa79f15d88e1e53e4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.972326 16840 raft_consensus.cc:2798] T cef4442bfcfd4ad4b343a2785c3a9a74 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.972354 16840 raft_consensus.cc:491] T cef4442bfcfd4ad4b343a2785c3a9a74 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.972340 14390 tablet_service.cc:1467] Processing CreateTablet for tablet dbde1733e0b94b1cbc8ea63182aab272 (DEFAULT_TABLE table=default.Table_b898f210bcf24098907e4c59c1278e80 [id=47a8e27516914e01bac481b02ae022c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.972384 16840 raft_consensus.cc:3054] T cef4442bfcfd4ad4b343a2785c3a9a74 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.972805 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d643bd0947c54bcfa79f15d88e1e53e4" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.972898 14422 raft_consensus.cc:2463] T d643bd0947c54bcfa79f15d88e1e53e4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.973111 14551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d643bd0947c54bcfa79f15d88e1e53e4" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.973132 14628 leader_election.cc:304] T d643bd0947c54bcfa79f15d88e1e53e4 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:22.973201 14551 raft_consensus.cc:2463] T d643bd0947c54bcfa79f15d88e1e53e4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.973575 16406 raft_consensus.cc:491] T 9ab3c40051ad42f692cebbd96144b60a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.973644 16406 raft_consensus.cc:513] T 9ab3c40051ad42f692cebbd96144b60a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.973913 16406 leader_election.cc:290] T 9ab3c40051ad42f692cebbd96144b60a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.973932 14397 tablet_service.cc:1467] Processing CreateTablet for tablet ceb349810698473da852b16ab4060137 (DEFAULT_TABLE table=default.Table_16f20fbeaf6c4ddf9712b913666e5a62 [id=6b4d53a42dbf4338ad3eb5574a6ce61c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:22.974321 14551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ab3c40051ad42f692cebbd96144b60a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.974535 14686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ab3c40051ad42f692cebbd96144b60a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.974840 14356 leader_election.cc:343] T 9ab3c40051ad42f692cebbd96144b60a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.974944 14387 tablet_service.cc:1467] Processing CreateTablet for tablet f2b18131a2e74ab891eb725c1c441cc6 (DEFAULT_TABLE table=default.Table_6d27f634a64f4390a1b36e24e867106b [id=606af32f91fa485b9ebea12bdfe5c233]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:22.975065 14358 leader_election.cc:343] T 9ab3c40051ad42f692cebbd96144b60a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.975111 14358 leader_election.cc:304] T 9ab3c40051ad42f692cebbd96144b60a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.975235 16406 raft_consensus.cc:491] T d140be0fab694f59b8a535392970fad0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.975287 16406 raft_consensus.cc:513] T d140be0fab694f59b8a535392970fad0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.975531 16406 leader_election.cc:290] T d140be0fab694f59b8a535392970fad0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.975596 16406 raft_consensus.cc:2743] T 9ab3c40051ad42f692cebbd96144b60a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.975697 14416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cef4442bfcfd4ad4b343a2785c3a9a74" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.975793 14416 raft_consensus.cc:2463] T cef4442bfcfd4ad4b343a2785c3a9a74 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.976007 14551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d140be0fab694f59b8a535392970fad0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.976565 14683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d140be0fab694f59b8a535392970fad0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.976815 14686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca14a7d574842ee94a93afb85ca1afd" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:22.976918 14686 raft_consensus.cc:3054] T fca14a7d574842ee94a93afb85ca1afd P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.977099 14391 tablet_service.cc:1467] Processing CreateTablet for tablet f97f9a033e284ce4ac4307cbae846e38 (DEFAULT_TABLE table=default.Table_f9c13724577e4bbbb8c20649b1cc6eb8 [id=d99617d91dcd406884530612717f45cd]), partition=RANGE (key) PARTITION UNBOUNDED
W20250115 03:06:22.977200 14356 leader_election.cc:343] T d140be0fab694f59b8a535392970fad0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:22.977409 14358 leader_election.cc:343] T d140be0fab694f59b8a535392970fad0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.977470 14358 leader_election.cc:304] T d140be0fab694f59b8a535392970fad0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.977720 16406 raft_consensus.cc:2743] T d140be0fab694f59b8a535392970fad0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.978313 16557 raft_consensus.cc:2798] T d643bd0947c54bcfa79f15d88e1e53e4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.978374 16557 raft_consensus.cc:491] T d643bd0947c54bcfa79f15d88e1e53e4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.978430 16557 raft_consensus.cc:3054] T d643bd0947c54bcfa79f15d88e1e53e4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.978790 14809 tablet_bootstrap.cc:492] T c127b5ac695a456aa38479a5f9a8fe82 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:22.978817 16189 raft_consensus.cc:491] T 78cc96de8b414ac19285338ccb1024e0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.978857 14809 ts_tablet_manager.cc:1397] T c127b5ac695a456aa38479a5f9a8fe82 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20250115 03:06:22.978894 16189 raft_consensus.cc:513] T 78cc96de8b414ac19285338ccb1024e0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.979214 16189 leader_election.cc:290] T 78cc96de8b414ac19285338ccb1024e0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.979218 14809 raft_consensus.cc:357] T c127b5ac695a456aa38479a5f9a8fe82 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.979293 14809 raft_consensus.cc:383] T c127b5ac695a456aa38479a5f9a8fe82 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:22.979331 14809 raft_consensus.cc:738] T c127b5ac695a456aa38479a5f9a8fe82 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:22.979426 14809 consensus_queue.cc:260] T c127b5ac695a456aa38479a5f9a8fe82 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.979638 14809 ts_tablet_manager.cc:1428] T c127b5ac695a456aa38479a5f9a8fe82 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:22.979761 14809 tablet_bootstrap.cc:492] T d8f54e9ccd04475f826ee851b0f079c0 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:22.980358 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cc96de8b414ac19285338ccb1024e0" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.980468 14422 raft_consensus.cc:2463] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.980510 14683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cc96de8b414ac19285338ccb1024e0" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.980368 16406 raft_consensus.cc:491] T 4c4ce5c2921341ce837ee4f5f7d131b7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.980599 14683 raft_consensus.cc:2463] T 78cc96de8b414ac19285338ccb1024e0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:22.980671 16406 raft_consensus.cc:513] T 4c4ce5c2921341ce837ee4f5f7d131b7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.980983 16406 leader_election.cc:290] T 4c4ce5c2921341ce837ee4f5f7d131b7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.981238 14493 leader_election.cc:304] T 78cc96de8b414ac19285338ccb1024e0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:22.981298 14551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c4ce5c2921341ce837ee4f5f7d131b7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:22.981572 14356 leader_election.cc:343] T 4c4ce5c2921341ce837ee4f5f7d131b7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.981716 14683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c4ce5c2921341ce837ee4f5f7d131b7" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.982000 14358 leader_election.cc:343] T 4c4ce5c2921341ce837ee4f5f7d131b7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.982017 16189 raft_consensus.cc:2798] T 78cc96de8b414ac19285338ccb1024e0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.982064 16189 raft_consensus.cc:491] T 78cc96de8b414ac19285338ccb1024e0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.982069 14358 leader_election.cc:304] T 4c4ce5c2921341ce837ee4f5f7d131b7 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.982093 16189 raft_consensus.cc:3054] T 78cc96de8b414ac19285338ccb1024e0 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.982304 16406 raft_consensus.cc:2743] T 4c4ce5c2921341ce837ee4f5f7d131b7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.985059 14416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca14a7d574842ee94a93afb85ca1afd" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:22.985162 14416 raft_consensus.cc:3054] T fca14a7d574842ee94a93afb85ca1afd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.985525 16406 raft_consensus.cc:491] T 7340ac89e5524c35948f3acc6c941cb5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.985603 16406 raft_consensus.cc:513] T 7340ac89e5524c35948f3acc6c941cb5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.985914 16406 leader_election.cc:290] T 7340ac89e5524c35948f3acc6c941cb5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.986315 14551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7340ac89e5524c35948f3acc6c941cb5" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.986315 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66defba523f4df996929f98c848f95a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:22.986440 14691 raft_consensus.cc:2463] T f66defba523f4df996929f98c848f95a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8df7247ee9c43e0aa18216da2661f75 in term 0.
I20250115 03:06:22.987027 14809 tablet_bootstrap.cc:654] T d8f54e9ccd04475f826ee851b0f079c0 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:22.987638 14356 leader_election.cc:343] T 7340ac89e5524c35948f3acc6c941cb5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.987823 14358 leader_election.cc:304] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.987993 16411 raft_consensus.cc:2798] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:22.988059 16411 raft_consensus.cc:491] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:22.988122 16411 raft_consensus.cc:3054] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:22.988504 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7340ac89e5524c35948f3acc6c941cb5" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.988973 14358 leader_election.cc:343] T 7340ac89e5524c35948f3acc6c941cb5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.989046 14358 leader_election.cc:304] T 7340ac89e5524c35948f3acc6c941cb5 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.989250 16406 raft_consensus.cc:2743] T 7340ac89e5524c35948f3acc6c941cb5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:22.990659 16849 raft_consensus.cc:491] T b0a7def7d9a7418381b2d5a93a9a9265 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.990803 16849 raft_consensus.cc:513] T b0a7def7d9a7418381b2d5a93a9a9265 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.991174 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_16f20fbeaf6c4ddf9712b913666e5a62" } new_table_name: "default.Table_fc8dca6b0f184858888644ec41e6c9bf" modify_external_catalogs: true
I20250115 03:06:22.991510 14551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0a7def7d9a7418381b2d5a93a9a9265" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:22.991896 14626 leader_election.cc:343] T b0a7def7d9a7418381b2d5a93a9a9265 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.992147 16406 raft_consensus.cc:491] T 9c436975bd034068a6d58d5831db1896 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.992228 16406 raft_consensus.cc:513] T 9c436975bd034068a6d58d5831db1896 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:22.992534 16406 leader_election.cc:290] T 9c436975bd034068a6d58d5831db1896 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:22.992834 14551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c436975bd034068a6d58d5831db1896" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.993196 16849 leader_election.cc:290] T b0a7def7d9a7418381b2d5a93a9a9265 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
W20250115 03:06:22.993279 14356 leader_election.cc:343] T 9c436975bd034068a6d58d5831db1896 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.993379 16849 raft_consensus.cc:491] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:22.993446 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0a7def7d9a7418381b2d5a93a9a9265" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:22.993546 14420 raft_consensus.cc:2463] T b0a7def7d9a7418381b2d5a93a9a9265 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.993803 14628 leader_election.cc:304] T b0a7def7d9a7418381b2d5a93a9a9265 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.993477 16849 raft_consensus.cc:513] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:22.994645 14551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6132d15b17ab4d0aaa10a7ea04cda7ac" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:22.994755 14551 raft_consensus.cc:2463] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:22.995055 14626 leader_election.cc:304] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:22.997014 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c436975bd034068a6d58d5831db1896" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:22.997502 14358 leader_election.cc:343] T 9c436975bd034068a6d58d5831db1896 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:22.997570 14358 leader_election.cc:304] T 9c436975bd034068a6d58d5831db1896 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:22.997771 16849 leader_election.cc:290] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:22.997922 16406 raft_consensus.cc:2743] T 9c436975bd034068a6d58d5831db1896 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20250115 03:06:22.997987 14628 leader_election.cc:343] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 6132d15b17ab4d0aaa10a7ea04cda7ac
W20250115 03:06:23.000090 14165 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ceb349810698473da852b16ab4060137 consensus state has no leader
I20250115 03:06:23.000255 16849 raft_consensus.cc:2798] T b0a7def7d9a7418381b2d5a93a9a9265 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.000306 16849 raft_consensus.cc:491] T b0a7def7d9a7418381b2d5a93a9a9265 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.000348 16849 raft_consensus.cc:3054] T b0a7def7d9a7418381b2d5a93a9a9265 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.000634 16848 raft_consensus.cc:2798] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.000682 16848 raft_consensus.cc:491] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.000721 16848 raft_consensus.cc:3054] T 6132d15b17ab4d0aaa10a7ea04cda7ac P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.002543 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_af4c136524d54a0a9a31eabc483319fa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:23.003003 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_af4c136524d54a0a9a31eabc483319fa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:23.006330 14163 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_12e083a49c774f2eab3c48fb2dd2cd0f" } modify_external_catalogs: true
I20250115 03:06:23.006426 14163 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_12e083a49c774f2eab3c48fb2dd2cd0f" } modify_external_catalogs: true
I20250115 03:06:23.011679 14163 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet f118a169019d4a32aa86b4fe9314f747
I20250115 03:06:23.013718 14164 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_2eec7fe42b894a729d365bc75c060236 [id=b298dce7ab664327893633f821ed4d89] alter complete (version 4)
I20250115 03:06:23.015188 14812 tablet_bootstrap.cc:492] T 820d5af6daa74cadb53d2a5229854a60 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.015278 14812 ts_tablet_manager.cc:1397] T 820d5af6daa74cadb53d2a5229854a60 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.067s user 0.000s sys 0.002s
I20250115 03:06:23.015779 14812 raft_consensus.cc:357] T 820d5af6daa74cadb53d2a5229854a60 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.015882 14812 raft_consensus.cc:383] T 820d5af6daa74cadb53d2a5229854a60 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.015913 14812 raft_consensus.cc:738] T 820d5af6daa74cadb53d2a5229854a60 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.016021 14812 consensus_queue.cc:260] T 820d5af6daa74cadb53d2a5229854a60 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.016302 14812 ts_tablet_manager.cc:1428] T 820d5af6daa74cadb53d2a5229854a60 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:23.016464 14812 tablet_bootstrap.cc:492] T 015ec42caa1b468ea10f25c8f26e278a P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:23.017479 14812 tablet_bootstrap.cc:654] T 015ec42caa1b468ea10f25c8f26e278a P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.017789 14809 tablet_bootstrap.cc:492] T d8f54e9ccd04475f826ee851b0f079c0 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:23.017867 14809 ts_tablet_manager.cc:1397] T d8f54e9ccd04475f826ee851b0f079c0 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.038s user 0.001s sys 0.000s
I20250115 03:06:23.018265 14809 raft_consensus.cc:357] T d8f54e9ccd04475f826ee851b0f079c0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.018361 14809 raft_consensus.cc:383] T d8f54e9ccd04475f826ee851b0f079c0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.018396 14809 raft_consensus.cc:738] T d8f54e9ccd04475f826ee851b0f079c0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.018489 14809 consensus_queue.cc:260] T d8f54e9ccd04475f826ee851b0f079c0 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.018798 14809 ts_tablet_manager.cc:1428] T d8f54e9ccd04475f826ee851b0f079c0 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.018973 14809 tablet_bootstrap.cc:492] T 4febfae2f2794669b5997a1fa7ef7412 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:23.019709 14809 tablet_bootstrap.cc:654] T 4febfae2f2794669b5997a1fa7ef7412 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.020979 14811 tablet_bootstrap.cc:492] T fdaf7c090ef642b8b339f61a5d3fb06e P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:23.021071 14811 ts_tablet_manager.cc:1397] T fdaf7c090ef642b8b339f61a5d3fb06e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.121s user 0.000s sys 0.002s
I20250115 03:06:23.021518 14811 raft_consensus.cc:357] T fdaf7c090ef642b8b339f61a5d3fb06e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.021631 14811 raft_consensus.cc:383] T fdaf7c090ef642b8b339f61a5d3fb06e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.021679 14811 raft_consensus.cc:738] T fdaf7c090ef642b8b339f61a5d3fb06e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.021799 14811 consensus_queue.cc:260] T fdaf7c090ef642b8b339f61a5d3fb06e P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.022141 14811 ts_tablet_manager.cc:1428] T fdaf7c090ef642b8b339f61a5d3fb06e P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:23.022286 14811 tablet_bootstrap.cc:492] T 820d5af6daa74cadb53d2a5229854a60 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:23.022311 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b3c0e0a301084e008804b31efbcbca00"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:23.023061 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_bbfcc61694f44215aa5736bcc075ec0e" } new_table_name: "default.Table_bdc74493a1b9469da9c4bf1d21f75684" modify_external_catalogs: true
I20250115 03:06:23.023169 14811 tablet_bootstrap.cc:654] T 820d5af6daa74cadb53d2a5229854a60 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.032008 14811 tablet_bootstrap.cc:492] T 820d5af6daa74cadb53d2a5229854a60 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:23.032092 14811 ts_tablet_manager.cc:1397] T 820d5af6daa74cadb53d2a5229854a60 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20250115 03:06:23.032538 14811 raft_consensus.cc:357] T 820d5af6daa74cadb53d2a5229854a60 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.032634 14811 raft_consensus.cc:383] T 820d5af6daa74cadb53d2a5229854a60 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.032665 14811 raft_consensus.cc:738] T 820d5af6daa74cadb53d2a5229854a60 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.032775 14811 consensus_queue.cc:260] T 820d5af6daa74cadb53d2a5229854a60 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.033067 14811 ts_tablet_manager.cc:1428] T 820d5af6daa74cadb53d2a5229854a60 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.033234 14811 tablet_bootstrap.cc:492] T b0a7def7d9a7418381b2d5a93a9a9265 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:23.034201 14811 tablet_bootstrap.cc:654] T b0a7def7d9a7418381b2d5a93a9a9265 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.037793 14809 tablet_bootstrap.cc:492] T 4febfae2f2794669b5997a1fa7ef7412 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:23.037881 14809 ts_tablet_manager.cc:1397] T 4febfae2f2794669b5997a1fa7ef7412 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.000s
I20250115 03:06:23.038313 14809 raft_consensus.cc:357] T 4febfae2f2794669b5997a1fa7ef7412 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.038420 14809 raft_consensus.cc:383] T 4febfae2f2794669b5997a1fa7ef7412 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.038462 14809 raft_consensus.cc:738] T 4febfae2f2794669b5997a1fa7ef7412 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.038571 14809 consensus_queue.cc:260] T 4febfae2f2794669b5997a1fa7ef7412 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.038865 14809 ts_tablet_manager.cc:1428] T 4febfae2f2794669b5997a1fa7ef7412 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.039037 14809 tablet_bootstrap.cc:492] T d1c4e127c99241da950af20754e6667c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:23.039597 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6b1f81ee12e04acda4bcf7d7f289a67a" } modify_external_catalogs: true
I20250115 03:06:23.039719 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_6b1f81ee12e04acda4bcf7d7f289a67a" } modify_external_catalogs: true
I20250115 03:06:23.039746 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_10e94765088044309cd6f627ab7a293b" } modify_external_catalogs: true
I20250115 03:06:23.039809 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_10e94765088044309cd6f627ab7a293b" } modify_external_catalogs: true
I20250115 03:06:23.039880 14809 tablet_bootstrap.cc:654] T d1c4e127c99241da950af20754e6667c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.041009 14812 tablet_bootstrap.cc:492] T 015ec42caa1b468ea10f25c8f26e278a P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.041101 14812 ts_tablet_manager.cc:1397] T 015ec42caa1b468ea10f25c8f26e278a P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.002s
I20250115 03:06:23.041567 14812 raft_consensus.cc:357] T 015ec42caa1b468ea10f25c8f26e278a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.041684 14812 raft_consensus.cc:383] T 015ec42caa1b468ea10f25c8f26e278a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.041718 14812 raft_consensus.cc:738] T 015ec42caa1b468ea10f25c8f26e278a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.041836 14812 consensus_queue.cc:260] T 015ec42caa1b468ea10f25c8f26e278a P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.042116 14812 ts_tablet_manager.cc:1428] T 015ec42caa1b468ea10f25c8f26e278a P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:23.042284 14812 tablet_bootstrap.cc:492] T 7340ac89e5524c35948f3acc6c941cb5 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:23.042630 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_1ae3ab02b9eb4ac98b3f5ea457bab6d1" } modify_external_catalogs: true
I20250115 03:06:23.042722 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_1ae3ab02b9eb4ac98b3f5ea457bab6d1" } modify_external_catalogs: true
I20250115 03:06:23.043202 14812 tablet_bootstrap.cc:654] T 7340ac89e5524c35948f3acc6c941cb5 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:23.043524 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b3c0e0a301084e008804b31efbcbca00 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:23.044055 14171 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 0fa5ce72e20b4e0092b39c64160f0995
I20250115 03:06:23.044206 14809 tablet_bootstrap.cc:492] T d1c4e127c99241da950af20754e6667c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:23.044335 14809 ts_tablet_manager.cc:1397] T d1c4e127c99241da950af20754e6667c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20250115 03:06:23.044962 16846 raft_consensus.cc:491] T cc01ad473e404e8ba26fd4cf7b0799f6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.045302 16798 raft_consensus.cc:491] T b3334f58b90b4947af52150546ccaa28 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.045380 16798 raft_consensus.cc:513] T b3334f58b90b4947af52150546ccaa28 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.045727 16798 leader_election.cc:290] T b3334f58b90b4947af52150546ccaa28 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.045050 14809 raft_consensus.cc:357] T d1c4e127c99241da950af20754e6667c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.045912 14809 raft_consensus.cc:383] T d1c4e127c99241da950af20754e6667c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.045094 16846 raft_consensus.cc:513] T cc01ad473e404e8ba26fd4cf7b0799f6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.046043 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3334f58b90b4947af52150546ccaa28" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:23.046159 14690 raft_consensus.cc:2463] T b3334f58b90b4947af52150546ccaa28 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:23.045984 14809 raft_consensus.cc:738] T d1c4e127c99241da950af20754e6667c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.046365 14809 consensus_queue.cc:260] T d1c4e127c99241da950af20754e6667c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.046490 14493 leader_election.cc:304] T b3334f58b90b4947af52150546ccaa28 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:23.046640 16798 raft_consensus.cc:2798] T b3334f58b90b4947af52150546ccaa28 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.046644 14809 ts_tablet_manager.cc:1428] T d1c4e127c99241da950af20754e6667c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:23.046691 16798 raft_consensus.cc:491] T b3334f58b90b4947af52150546ccaa28 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.046741 16798 raft_consensus.cc:3054] T b3334f58b90b4947af52150546ccaa28 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.046808 14809 tablet_bootstrap.cc:492] T 520574b17c68472e8b624c2ec02759db P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:23.047426 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3334f58b90b4947af52150546ccaa28" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.047567 14420 raft_consensus.cc:2463] T b3334f58b90b4947af52150546ccaa28 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:23.047685 14809 tablet_bootstrap.cc:654] T 520574b17c68472e8b624c2ec02759db P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.047855 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet ad0e9216bd154f4a8a37874d3d8a08c2
I20250115 03:06:23.048331 16846 leader_election.cc:290] T cc01ad473e404e8ba26fd4cf7b0799f6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.048624 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc01ad473e404e8ba26fd4cf7b0799f6" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:23.048812 16852 raft_consensus.cc:491] T 815c5e096e4d4b0cbfb7c15bac571b10 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.048885 16852 raft_consensus.cc:513] T 815c5e096e4d4b0cbfb7c15bac571b10 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:23.048970 14358 leader_election.cc:343] T cc01ad473e404e8ba26fd4cf7b0799f6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.049321 16852 leader_election.cc:290] T 815c5e096e4d4b0cbfb7c15bac571b10 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:23.049660 14551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "815c5e096e4d4b0cbfb7c15bac571b10" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.049886 14812 tablet_bootstrap.cc:492] T 7340ac89e5524c35948f3acc6c941cb5 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
W20250115 03:06:23.049986 14626 leader_election.cc:343] T 815c5e096e4d4b0cbfb7c15bac571b10 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.050032 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc01ad473e404e8ba26fd4cf7b0799f6" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.050208 14812 ts_tablet_manager.cc:1397] T 7340ac89e5524c35948f3acc6c941cb5 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
W20250115 03:06:23.050316 14356 leader_election.cc:343] T cc01ad473e404e8ba26fd4cf7b0799f6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.050386 14356 leader_election.cc:304] T cc01ad473e404e8ba26fd4cf7b0799f6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.050675 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "815c5e096e4d4b0cbfb7c15bac571b10" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.050796 14420 raft_consensus.cc:2463] T 815c5e096e4d4b0cbfb7c15bac571b10 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.051105 14812 raft_consensus.cc:357] T 7340ac89e5524c35948f3acc6c941cb5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.051383 14812 raft_consensus.cc:383] T 7340ac89e5524c35948f3acc6c941cb5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.051455 14812 raft_consensus.cc:738] T 7340ac89e5524c35948f3acc6c941cb5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.051654 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5a2894e4736b4191963a61d908bc673c (table default.Table_df79ed42f26f4b68b197877c445b9665 [id=bb6453f2ab97404cbd3ba32a87976e52]). Expected version 1 got 0
I20250115 03:06:23.051676 14628 leader_election.cc:304] T 815c5e096e4d4b0cbfb7c15bac571b10 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.051657 14812 consensus_queue.cc:260] T 7340ac89e5524c35948f3acc6c941cb5 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.051838 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet fe513ca696f14838b3886aac174f2cb5 (table default.Table_1e82291a1f76499cbe13faaf6248ec82 [id=e54709f6a5194c01a2a57ac4c21b86b0]). Expected version 1 got 0
I20250115 03:06:23.051927 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 695927c78bb74b669f2aeca1abf1b648 (table default.Table_d9796d81c05d4f50bc62f2f73fefc5d8 [id=ccb0d57971b94cf3b5ccaed448fe3223]). Expected version 1 got 0
I20250115 03:06:23.051928 16850 raft_consensus.cc:2798] T 815c5e096e4d4b0cbfb7c15bac571b10 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.051995 16850 raft_consensus.cc:491] T 815c5e096e4d4b0cbfb7c15bac571b10 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.052075 16850 raft_consensus.cc:3054] T 815c5e096e4d4b0cbfb7c15bac571b10 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.052055 14163 catalog_manager.cc:5526] T 72abf6acd9d54f3b983b7098aaf2691d P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195). New cstate: current_term: 1 leader_uuid: "5c0dae7aff4b44c699275b95ba639770" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } }
I20250115 03:06:23.052196 14163 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 72abf6acd9d54f3b983b7098aaf2691d (table default.Table_5e72c1c96c004205bf7304be210f76f1 [id=5ca0a4e7179649d3870fdadc05eaa427]). Expected version 1 got 0
I20250115 03:06:23.052750 14812 ts_tablet_manager.cc:1428] T 7340ac89e5524c35948f3acc6c941cb5 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:23.053088 14812 tablet_bootstrap.cc:492] T 85fcafa24fa1448c995d30dd93b0737f P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:23.053295 16846 raft_consensus.cc:2743] T cc01ad473e404e8ba26fd4cf7b0799f6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:23.054711 14812 tablet_bootstrap.cc:654] T 85fcafa24fa1448c995d30dd93b0737f P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.055617 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet af712cc03750455087ecd381cc97d464
I20250115 03:06:23.055920 14811 tablet_bootstrap.cc:492] T b0a7def7d9a7418381b2d5a93a9a9265 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:23.056002 14811 ts_tablet_manager.cc:1397] T b0a7def7d9a7418381b2d5a93a9a9265 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.000s
I20250115 03:06:23.056413 14811 raft_consensus.cc:357] T b0a7def7d9a7418381b2d5a93a9a9265 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.056504 14811 raft_consensus.cc:383] T b0a7def7d9a7418381b2d5a93a9a9265 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.056533 14811 raft_consensus.cc:738] T b0a7def7d9a7418381b2d5a93a9a9265 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.056640 14811 consensus_queue.cc:260] T b0a7def7d9a7418381b2d5a93a9a9265 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.057096 14811 ts_tablet_manager.cc:1428] T b0a7def7d9a7418381b2d5a93a9a9265 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.057264 14811 tablet_bootstrap.cc:492] T 015ec42caa1b468ea10f25c8f26e278a P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
W20250115 03:06:23.057837 16620 consensus_meta.cc:287] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent flushing consensus metadata: real 0.553s user 0.001s sys 0.000s
I20250115 03:06:23.057891 16620 raft_consensus.cc:513] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.058203 16620 leader_election.cc:290] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
W20250115 03:06:23.058921 16556 consensus_meta.cc:287] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770: Time spent flushing consensus metadata: real 0.679s user 0.001s sys 0.000s
I20250115 03:06:23.058976 16556 raft_consensus.cc:513] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.059309 16556 leader_election.cc:290] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:23.059541 14416 raft_consensus.cc:2463] T fca14a7d574842ee94a93afb85ca1afd P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:23.060627 14395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8b721500ba146d4a97fa52c86814a35. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.061002 14399 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9099a50c9e9f487dba708e5cbdff329c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.061345 14386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea1ff51dc99e4cf5aa310cbaa5cb16c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.061681 14385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f75507ac77a4a458e2394d3c9a16a91. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.061997 14384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3149d4e2744f7fa1032663ea2b46e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.062283 14383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c1a685ffa324f56be0d8e3c2827974b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.062522 14389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a97b4ad4327446cbae405f3d43a17781. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.062736 14392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eee4e25e788646c9abaf9ad9ccb0a92c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.062844 14812 tablet_bootstrap.cc:492] T 85fcafa24fa1448c995d30dd93b0737f P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.062913 14812 ts_tablet_manager.cc:1397] T 85fcafa24fa1448c995d30dd93b0737f P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20250115 03:06:23.062983 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc4c45cf04854df5af831417623f14f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.063205 14402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47fed237bfc94d7e819557e9fce0ce95. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.063297 14812 raft_consensus.cc:357] T 85fcafa24fa1448c995d30dd93b0737f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.063382 14812 raft_consensus.cc:383] T 85fcafa24fa1448c995d30dd93b0737f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.063416 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3ddba7630904f16a36698c1e5e7e9cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.063409 14812 raft_consensus.cc:738] T 85fcafa24fa1448c995d30dd93b0737f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.063583 14812 consensus_queue.cc:260] T 85fcafa24fa1448c995d30dd93b0737f P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.063661 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a179a7c7aab4831a42e9f25f22163ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.063874 14388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1822e678e3a44f24a6f4d4b13e86e8aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.063990 14812 ts_tablet_manager.cc:1428] T 85fcafa24fa1448c995d30dd93b0737f P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.064105 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad5ada0afc2d4a4d94595fa951fcda1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.064199 14812 tablet_bootstrap.cc:492] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:23.064306 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84c592ced5b443869240eff55fe9d549. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.064518 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbde1733e0b94b1cbc8ea63182aab272. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.064723 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ceb349810698473da852b16ab4060137. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.065528 14812 tablet_bootstrap.cc:654] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.065727 14387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2b18131a2e74ab891eb725c1c441cc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.065995 14391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f97f9a033e284ce4ac4307cbae846e38. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.063895 16620 raft_consensus.cc:491] T 9bb0b7a8acfd40738abcc951936b1116 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.069414 16620 raft_consensus.cc:513] T 9bb0b7a8acfd40738abcc951936b1116 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.069723 16620 leader_election.cc:290] T 9bb0b7a8acfd40738abcc951936b1116 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.059751 14811 tablet_bootstrap.cc:654] T 015ec42caa1b468ea10f25c8f26e278a P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.069965 14417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de14b1d525cc42808500279ae6b60e80" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.072402 14812 tablet_bootstrap.cc:492] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.072479 14812 ts_tablet_manager.cc:1397] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20250115 03:06:23.072837 14812 raft_consensus.cc:357] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.072912 14812 raft_consensus.cc:383] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.072957 14812 raft_consensus.cc:738] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.073050 14812 consensus_queue.cc:260] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.073290 14812 ts_tablet_manager.cc:1428] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.073429 14812 tablet_bootstrap.cc:492] T 9c436975bd034068a6d58d5831db1896 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:23.074219 14812 tablet_bootstrap.cc:654] T 9c436975bd034068a6d58d5831db1896 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.077782 14811 tablet_bootstrap.cc:492] T 015ec42caa1b468ea10f25c8f26e278a P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:23.077912 14811 ts_tablet_manager.cc:1397] T 015ec42caa1b468ea10f25c8f26e278a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.002s
I20250115 03:06:23.078462 14811 raft_consensus.cc:357] T 015ec42caa1b468ea10f25c8f26e278a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.078670 14811 raft_consensus.cc:383] T 015ec42caa1b468ea10f25c8f26e278a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.078761 14811 raft_consensus.cc:738] T 015ec42caa1b468ea10f25c8f26e278a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.078918 14811 consensus_queue.cc:260] T 015ec42caa1b468ea10f25c8f26e278a P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.079627 14812 tablet_bootstrap.cc:492] T 9c436975bd034068a6d58d5831db1896 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.079713 14812 ts_tablet_manager.cc:1397] T 9c436975bd034068a6d58d5831db1896 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250115 03:06:23.080082 14812 raft_consensus.cc:357] T 9c436975bd034068a6d58d5831db1896 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.080186 14812 raft_consensus.cc:383] T 9c436975bd034068a6d58d5831db1896 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.080214 14812 raft_consensus.cc:738] T 9c436975bd034068a6d58d5831db1896 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.080296 14812 consensus_queue.cc:260] T 9c436975bd034068a6d58d5831db1896 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.080546 14812 ts_tablet_manager.cc:1428] T 9c436975bd034068a6d58d5831db1896 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.080685 14812 tablet_bootstrap.cc:492] T b9f67c35591f4fdeb8a926a085171b49 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:23.081445 14811 ts_tablet_manager.cc:1428] T 015ec42caa1b468ea10f25c8f26e278a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20250115 03:06:23.081604 14811 tablet_bootstrap.cc:492] T f66defba523f4df996929f98c848f95a P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:23.082753 14812 tablet_bootstrap.cc:654] T b9f67c35591f4fdeb8a926a085171b49 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.083210 14164 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2ffd8bc4e3924dc6a952ee6b31e2b305" } new_table_name: "default.Table_c4e2942423c7424bb3d3f9cfe864ee1f" modify_external_catalogs: true
I20250115 03:06:23.085361 14811 tablet_bootstrap.cc:654] T f66defba523f4df996929f98c848f95a P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.086117 16838 raft_consensus.cc:491] T aadcec8aca2b4ece915707c17f60b8b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.086192 16838 raft_consensus.cc:513] T aadcec8aca2b4ece915707c17f60b8b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.086527 16838 leader_election.cc:290] T aadcec8aca2b4ece915707c17f60b8b5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:23.086838 16847 raft_consensus.cc:491] T 9bb0b7a8acfd40738abcc951936b1116 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.086917 16847 raft_consensus.cc:513] T 9bb0b7a8acfd40738abcc951936b1116 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.087240 16847 leader_election.cc:290] T 9bb0b7a8acfd40738abcc951936b1116 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:23.087286 14551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aadcec8aca2b4ece915707c17f60b8b5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.087301 14416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aadcec8aca2b4ece915707c17f60b8b5" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.087414 14551 raft_consensus.cc:2388] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c0dae7aff4b44c699275b95ba639770 in current term 1: Already voted for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in this term.
I20250115 03:06:23.087404 14416 raft_consensus.cc:2463] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.087456 16847 raft_consensus.cc:491] T 980b69b60a354519b0691ecb0bbd6c70 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.087522 16847 raft_consensus.cc:513] T 980b69b60a354519b0691ecb0bbd6c70 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.087800 16847 leader_election.cc:290] T 980b69b60a354519b0691ecb0bbd6c70 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:23.088471 14551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb0b7a8acfd40738abcc951936b1116" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.088574 14551 raft_consensus.cc:2463] T 9bb0b7a8acfd40738abcc951936b1116 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
W20250115 03:06:23.088742 14628 leader_election.cc:343] T b61cc4b11251486a8089ca9bf44ac91e P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: b61cc4b11251486a8089ca9bf44ac91e
I20250115 03:06:23.088764 14550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "980b69b60a354519b0691ecb0bbd6c70" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.088846 14628 leader_election.cc:304] T aadcec8aca2b4ece915707c17f60b8b5 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.088868 14550 raft_consensus.cc:2463] T 980b69b60a354519b0691ecb0bbd6c70 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.089046 16838 raft_consensus.cc:2798] T aadcec8aca2b4ece915707c17f60b8b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.089102 16838 raft_consensus.cc:491] T aadcec8aca2b4ece915707c17f60b8b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.089138 16838 raft_consensus.cc:3054] T aadcec8aca2b4ece915707c17f60b8b5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.089192 14168 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a6989a57431e4523afea72558cea88c2" } new_table_name: "default.Table_9bb447731f2f47b99a4cd198442c05c2" modify_external_catalogs: true
I20250115 03:06:23.089414 14626 leader_election.cc:304] T 9bb0b7a8acfd40738abcc951936b1116 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:23.089481 14555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b61cc4b11251486a8089ca9bf44ac91e" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
I20250115 03:06:23.089567 14555 raft_consensus.cc:3054] T b61cc4b11251486a8089ca9bf44ac91e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.089576 14626 leader_election.cc:304] T 980b69b60a354519b0691ecb0bbd6c70 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:23.089748 16556 raft_consensus.cc:2798] T 9bb0b7a8acfd40738abcc951936b1116 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.089795 16556 raft_consensus.cc:491] T 9bb0b7a8acfd40738abcc951936b1116 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.089830 16556 raft_consensus.cc:3054] T 9bb0b7a8acfd40738abcc951936b1116 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.089965 16852 raft_consensus.cc:2798] T 980b69b60a354519b0691ecb0bbd6c70 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.090003 16852 raft_consensus.cc:491] T 980b69b60a354519b0691ecb0bbd6c70 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.090036 16852 raft_consensus.cc:3054] T 980b69b60a354519b0691ecb0bbd6c70 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.089991 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_11659a810c9a4358af952c8ecdcf9d87"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:23.090101 14812 tablet_bootstrap.cc:492] T b9f67c35591f4fdeb8a926a085171b49 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.090166 14812 ts_tablet_manager.cc:1397] T b9f67c35591f4fdeb8a926a085171b49 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
W20250115 03:06:23.090302 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11659a810c9a4358af952c8ecdcf9d87 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:23.090562 14812 raft_consensus.cc:357] T b9f67c35591f4fdeb8a926a085171b49 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.090646 14812 raft_consensus.cc:383] T b9f67c35591f4fdeb8a926a085171b49 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.090673 14812 raft_consensus.cc:738] T b9f67c35591f4fdeb8a926a085171b49 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.090766 14812 consensus_queue.cc:260] T b9f67c35591f4fdeb8a926a085171b49 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.091061 14812 ts_tablet_manager.cc:1428] T b9f67c35591f4fdeb8a926a085171b49 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.091204 14812 tablet_bootstrap.cc:492] T 2b0e3df3c79c40ba958b6e2453fa666f P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:23.091868 14691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb0b7a8acfd40738abcc951936b1116" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:23.092042 14690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aadcec8aca2b4ece915707c17f60b8b5" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:23.092059 14812 tablet_bootstrap.cc:654] T 2b0e3df3c79c40ba958b6e2453fa666f P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.092517 14416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb0b7a8acfd40738abcc951936b1116" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.092619 14416 raft_consensus.cc:2463] T 9bb0b7a8acfd40738abcc951936b1116 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.092815 16620 raft_consensus.cc:491] T fdaf7c090ef642b8b339f61a5d3fb06e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.092888 16620 raft_consensus.cc:513] T fdaf7c090ef642b8b339f61a5d3fb06e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.093201 16620 leader_election.cc:290] T fdaf7c090ef642b8b339f61a5d3fb06e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.093922 14683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdaf7c090ef642b8b339f61a5d3fb06e" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:23.094728 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_0c288c0d91374c20be64d0ec492a7260"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:23.095033 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c288c0d91374c20be64d0ec492a7260 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:23.095485 14412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdaf7c090ef642b8b339f61a5d3fb06e" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.094769 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_feac21ed047342a3bd9170f367844b37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:23.095556 16406 raft_consensus.cc:491] T 12b78b3abfe84714a97ecc89f2fb9e33 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.095582 14412 raft_consensus.cc:2463] T fdaf7c090ef642b8b339f61a5d3fb06e P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:23.095629 16406 raft_consensus.cc:513] T 12b78b3abfe84714a97ecc89f2fb9e33 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
W20250115 03:06:23.095808 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_feac21ed047342a3bd9170f367844b37 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:23.095968 16406 leader_election.cc:290] T 12b78b3abfe84714a97ecc89f2fb9e33 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.096570 14417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bb0b7a8acfd40738abcc951936b1116" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.096662 14417 raft_consensus.cc:2463] T 9bb0b7a8acfd40738abcc951936b1116 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:23.096688 14812 tablet_bootstrap.cc:492] T 2b0e3df3c79c40ba958b6e2453fa666f P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.096761 14812 ts_tablet_manager.cc:1397] T 2b0e3df3c79c40ba958b6e2453fa666f P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:23.097092 14416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aadcec8aca2b4ece915707c17f60b8b5" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:23.097180 14416 raft_consensus.cc:3054] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.097182 14812 raft_consensus.cc:357] T 2b0e3df3c79c40ba958b6e2453fa666f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.097267 14812 raft_consensus.cc:383] T 2b0e3df3c79c40ba958b6e2453fa666f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.097297 14812 raft_consensus.cc:738] T 2b0e3df3c79c40ba958b6e2453fa666f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.097400 14812 consensus_queue.cc:260] T 2b0e3df3c79c40ba958b6e2453fa666f P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.097489 14682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12b78b3abfe84714a97ecc89f2fb9e33" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:23.097494 14493 leader_election.cc:304] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:23.097669 14812 ts_tablet_manager.cc:1428] T 2b0e3df3c79c40ba958b6e2453fa666f P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20250115 03:06:23.097661 14493 leader_election.cc:343] T de14b1d525cc42808500279ae6b60e80 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Illegal state: Tablet not RUNNING: SHUTDOWN
I20250115 03:06:23.097816 14812 tablet_bootstrap.cc:492] T 15263a3b32be418288770ce348c082d7 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:23.097923 14358 leader_election.cc:343] T 12b78b3abfe84714a97ecc89f2fb9e33 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.098052 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "980b69b60a354519b0691ecb0bbd6c70" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.098146 14420 raft_consensus.cc:2463] T 980b69b60a354519b0691ecb0bbd6c70 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.098627 14812 tablet_bootstrap.cc:654] T 15263a3b32be418288770ce348c082d7 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.099951 14551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12b78b3abfe84714a97ecc89f2fb9e33" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.099951 14493 leader_election.cc:304] T fdaf7c090ef642b8b339f61a5d3fb06e P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:23.100179 14493 leader_election.cc:304] T 9bb0b7a8acfd40738abcc951936b1116 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:23.100584 16808 raft_consensus.cc:2798] T fdaf7c090ef642b8b339f61a5d3fb06e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.100631 16808 raft_consensus.cc:491] T fdaf7c090ef642b8b339f61a5d3fb06e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.100666 16808 raft_consensus.cc:3054] T fdaf7c090ef642b8b339f61a5d3fb06e P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.099990 16620 raft_consensus.cc:2798] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:23.101055 16620 raft_consensus.cc:695] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:23.101176 16620 consensus_queue.cc:237] T fca14a7d574842ee94a93afb85ca1afd P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.101691 16620 raft_consensus.cc:2798] T 9bb0b7a8acfd40738abcc951936b1116 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.101735 16620 raft_consensus.cc:491] T 9bb0b7a8acfd40738abcc951936b1116 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.101765 16620 raft_consensus.cc:3054] T 9bb0b7a8acfd40738abcc951936b1116 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.103607 14812 tablet_bootstrap.cc:492] T 15263a3b32be418288770ce348c082d7 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.103682 14812 ts_tablet_manager.cc:1397] T 15263a3b32be418288770ce348c082d7 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250115 03:06:23.104044 14812 raft_consensus.cc:357] T 15263a3b32be418288770ce348c082d7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.104128 14812 raft_consensus.cc:383] T 15263a3b32be418288770ce348c082d7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.104158 14812 raft_consensus.cc:738] T 15263a3b32be418288770ce348c082d7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.104166 16406 raft_consensus.cc:491] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.104231 16406 raft_consensus.cc:513] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.104254 14812 consensus_queue.cc:260] T 15263a3b32be418288770ce348c082d7 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.104497 14812 ts_tablet_manager.cc:1428] T 15263a3b32be418288770ce348c082d7 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.104521 16406 leader_election.cc:290] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.104640 14812 tablet_bootstrap.cc:492] T 12b78b3abfe84714a97ecc89f2fb9e33 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:23.104859 14682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ae7d8e213544722bb30bfc7f0f4a9c4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:23.105190 14358 leader_election.cc:343] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:23.105468 14356 leader_election.cc:343] T 12b78b3abfe84714a97ecc89f2fb9e33 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.105547 14812 tablet_bootstrap.cc:654] T 12b78b3abfe84714a97ecc89f2fb9e33 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.105541 14356 leader_election.cc:304] T 12b78b3abfe84714a97ecc89f2fb9e33 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.105636 14163 catalog_manager.cc:6071] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: default.Table_77761cbb0c804fa4a8d4298c4db47f76 [id=22d05297fb9545aeab39b5b62248d0d4] alter complete (version 1)
I20250115 03:06:23.105666 16406 raft_consensus.cc:2743] T 12b78b3abfe84714a97ecc89f2fb9e33 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:23.107681 14551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ae7d8e213544722bb30bfc7f0f4a9c4" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.109537 14391 tablet_service.cc:1467] Processing CreateTablet for tablet 680d9a43dbb545949b03f8093702399a (DEFAULT_TABLE table=default.Table_5daefae8f3b94003a8941d6f94dda850 [id=85d57da9b6ef47c799d659910c165538]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.109798 14391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 680d9a43dbb545949b03f8093702399a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.110275 14812 tablet_bootstrap.cc:492] T 12b78b3abfe84714a97ecc89f2fb9e33 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.110347 14812 ts_tablet_manager.cc:1397] T 12b78b3abfe84714a97ecc89f2fb9e33 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20250115 03:06:23.110761 14812 raft_consensus.cc:357] T 12b78b3abfe84714a97ecc89f2fb9e33 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.110847 14812 raft_consensus.cc:383] T 12b78b3abfe84714a97ecc89f2fb9e33 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.110875 14812 raft_consensus.cc:738] T 12b78b3abfe84714a97ecc89f2fb9e33 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.110963 14812 consensus_queue.cc:260] T 12b78b3abfe84714a97ecc89f2fb9e33 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.111197 14812 ts_tablet_manager.cc:1428] T 12b78b3abfe84714a97ecc89f2fb9e33 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.111364 14812 tablet_bootstrap.cc:492] T cc01ad473e404e8ba26fd4cf7b0799f6 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:23.111464 14819 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45810 (request call id 1105) took 1320 ms. Trace:
I20250115 03:06:23.111538 14819 rpcz_store.cc:276] 0115 03:06:21.790914 (+ 0us) service_pool.cc:168] Inserting onto call queue
0115 03:06:21.792679 (+ 1765us) service_pool.cc:225] Handling call
0115 03:06:22.391854 (+599175us) ts_tablet_manager.cc:1145] Deleting tablet de14b1d525cc42808500279ae6b60e80
0115 03:06:22.391861 (+ 7us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0115 03:06:22.485420 (+ 93559us) tablet_metadata.cc:641] Metadata flushed
0115 03:06:22.570994 (+ 85574us) tablet_metadata.cc:641] Metadata flushed
0115 03:06:23.111450 (+540456us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":599105}
I20250115 03:06:23.112190 14819 tablet_replica.cc:331] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:23.112197 14812 tablet_bootstrap.cc:654] T cc01ad473e404e8ba26fd4cf7b0799f6 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.112287 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet de14b1d525cc42808500279ae6b60e80 (table default.Table_b922b874a412477f8cf1a410e26beb1d [id=76a2384a834c40a2a8ec940808a94290]) successfully deleted
I20250115 03:06:23.112322 14819 raft_consensus.cc:2238] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:23.112418 14819 raft_consensus.cc:2267] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:23.112769 14819 ts_tablet_manager.cc:1905] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
W20250115 03:06:23.114282 14164 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c9ed00b2835b46f78b3393fa40b13bcf consensus state has no leader
W20250115 03:06:23.114526 14168 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8ec7bef5519f40419f0ec4bfc7750797 consensus state has no leader
I20250115 03:06:23.115568 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_4b5c24b98b6c44ac80e91078b456cd69"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:23.115865 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4b5c24b98b6c44ac80e91078b456cd69 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:23.118355 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_216163bc6a7d4d9292b2ff6766a5397d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:23.118633 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_216163bc6a7d4d9292b2ff6766a5397d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:23.119431 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_15ff7d65ac354c86b25af14aa5881324"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:23.119537 14812 tablet_bootstrap.cc:492] T cc01ad473e404e8ba26fd4cf7b0799f6 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.119609 14812 ts_tablet_manager.cc:1397] T cc01ad473e404e8ba26fd4cf7b0799f6 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
W20250115 03:06:23.119714 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_15ff7d65ac354c86b25af14aa5881324 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:23.120057 14812 raft_consensus.cc:357] T cc01ad473e404e8ba26fd4cf7b0799f6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.120152 14812 raft_consensus.cc:383] T cc01ad473e404e8ba26fd4cf7b0799f6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.120180 14812 raft_consensus.cc:738] T cc01ad473e404e8ba26fd4cf7b0799f6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.120287 14812 consensus_queue.cc:260] T cc01ad473e404e8ba26fd4cf7b0799f6 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.120553 14812 ts_tablet_manager.cc:1428] T cc01ad473e404e8ba26fd4cf7b0799f6 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.120702 14812 tablet_bootstrap.cc:492] T ec0f3424ebcb442597a94ccdbb25eeb6 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
W20250115 03:06:23.121788 14356 leader_election.cc:343] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.121857 14356 leader_election.cc:304] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.121896 14812 tablet_bootstrap.cc:654] T ec0f3424ebcb442597a94ccdbb25eeb6 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:23.125648 16828 log.cc:926] Time spent T 5a2894e4736b4191963a61d908bc673c P e8df7247ee9c43e0aa18216da2661f75: Append to log took a long time: real 0.077s user 0.000s sys 0.000s
I20250115 03:06:23.127031 14809 tablet_bootstrap.cc:492] T 520574b17c68472e8b624c2ec02759db P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:23.127120 14809 ts_tablet_manager.cc:1397] T 520574b17c68472e8b624c2ec02759db P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.080s user 0.002s sys 0.000s
I20250115 03:06:23.127537 14809 raft_consensus.cc:357] T 520574b17c68472e8b624c2ec02759db P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.127641 14809 raft_consensus.cc:383] T 520574b17c68472e8b624c2ec02759db P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.127691 14809 raft_consensus.cc:738] T 520574b17c68472e8b624c2ec02759db P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.127696 14812 tablet_bootstrap.cc:492] T ec0f3424ebcb442597a94ccdbb25eeb6 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.127759 14812 ts_tablet_manager.cc:1397] T ec0f3424ebcb442597a94ccdbb25eeb6 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:23.127800 14809 consensus_queue.cc:260] T 520574b17c68472e8b624c2ec02759db P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.128068 14809 ts_tablet_manager.cc:1428] T 520574b17c68472e8b624c2ec02759db P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.128234 14809 tablet_bootstrap.cc:492] T f25edfc3d7c043bfa337b31fa9131f1d P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:23.128276 14812 raft_consensus.cc:357] T ec0f3424ebcb442597a94ccdbb25eeb6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.128353 14812 raft_consensus.cc:383] T ec0f3424ebcb442597a94ccdbb25eeb6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.128381 14812 raft_consensus.cc:738] T ec0f3424ebcb442597a94ccdbb25eeb6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.128490 14812 consensus_queue.cc:260] T ec0f3424ebcb442597a94ccdbb25eeb6 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.128751 14812 ts_tablet_manager.cc:1428] T ec0f3424ebcb442597a94ccdbb25eeb6 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.128908 14812 tablet_bootstrap.cc:492] T 9ab3c40051ad42f692cebbd96144b60a P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:23.129284 14811 tablet_bootstrap.cc:492] T f66defba523f4df996929f98c848f95a P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:23.129383 14811 ts_tablet_manager.cc:1397] T f66defba523f4df996929f98c848f95a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.048s user 0.000s sys 0.002s
I20250115 03:06:23.129766 14812 tablet_bootstrap.cc:654] T 9ab3c40051ad42f692cebbd96144b60a P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.129817 14811 raft_consensus.cc:357] T f66defba523f4df996929f98c848f95a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.129909 14811 raft_consensus.cc:383] T f66defba523f4df996929f98c848f95a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.129937 14811 raft_consensus.cc:738] T f66defba523f4df996929f98c848f95a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.130056 14811 consensus_queue.cc:260] T f66defba523f4df996929f98c848f95a P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.130333 14811 ts_tablet_manager.cc:1428] T f66defba523f4df996929f98c848f95a P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:23.130484 14811 tablet_bootstrap.cc:492] T 815c5e096e4d4b0cbfb7c15bac571b10 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:23.130669 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 5a2894e4736b4191963a61d908bc673c (table default.Table_df79ed42f26f4b68b197877c445b9665 [id=bb6453f2ab97404cbd3ba32a87976e52]). Expected version 1 got 0
I20250115 03:06:23.130815 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet d1c4e127c99241da950af20754e6667c (table default.Table_94c5e41f3cf24fc983cb6108737fb277 [id=fc65db087e2a4b158e2db97175d1863f]). Expected version 1 got 0
I20250115 03:06:23.131597 14811 tablet_bootstrap.cc:654] T 815c5e096e4d4b0cbfb7c15bac571b10 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.132117 16406 raft_consensus.cc:2743] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:23.132731 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b8fe10217f50463e9498565886de04c6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:23.133045 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b8fe10217f50463e9498565886de04c6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:23.134492 14165 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_8a8b70005ac8484f9117b80d600afdf7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:23.134873 14165 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a8b70005ac8484f9117b80d600afdf7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:23.137724 14809 tablet_bootstrap.cc:654] T f25edfc3d7c043bfa337b31fa9131f1d P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.139351 16882 raft_consensus.cc:491] T f66defba523f4df996929f98c848f95a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.139487 16882 raft_consensus.cc:513] T f66defba523f4df996929f98c848f95a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.139856 16882 leader_election.cc:290] T f66defba523f4df996929f98c848f95a P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.140121 16846 raft_consensus.cc:491] T 695927c78bb74b669f2aeca1abf1b648 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.140200 16846 raft_consensus.cc:513] T 695927c78bb74b669f2aeca1abf1b648 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.140515 16846 leader_election.cc:290] T 695927c78bb74b669f2aeca1abf1b648 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.140831 14682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "695927c78bb74b669f2aeca1abf1b648" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:23.141216 14358 leader_election.cc:343] T 695927c78bb74b669f2aeca1abf1b648 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.141613 14551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "695927c78bb74b669f2aeca1abf1b648" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:23.142765 14356 leader_election.cc:343] T 695927c78bb74b669f2aeca1abf1b648 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.142871 14680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66defba523f4df996929f98c848f95a" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:23.142989 14680 raft_consensus.cc:2463] T f66defba523f4df996929f98c848f95a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:23.142990 14356 leader_election.cc:304] T 695927c78bb74b669f2aeca1abf1b648 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.143420 14493 leader_election.cc:304] T f66defba523f4df996929f98c848f95a P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:23.143625 16846 raft_consensus.cc:2743] T 695927c78bb74b669f2aeca1abf1b648 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:23.143944 16876 raft_consensus.cc:2798] T f66defba523f4df996929f98c848f95a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.144095 16876 raft_consensus.cc:491] T f66defba523f4df996929f98c848f95a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.144176 16876 raft_consensus.cc:3054] T f66defba523f4df996929f98c848f95a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.142771 14417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66defba523f4df996929f98c848f95a" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.146550 16847 raft_consensus.cc:491] T 15263a3b32be418288770ce348c082d7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.146634 16847 raft_consensus.cc:513] T 15263a3b32be418288770ce348c082d7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.146878 16846 raft_consensus.cc:491] T d8f54e9ccd04475f826ee851b0f079c0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.146952 16847 leader_election.cc:290] T 15263a3b32be418288770ce348c082d7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:23.146958 16846 raft_consensus.cc:513] T d8f54e9ccd04475f826ee851b0f079c0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.147205 16846 leader_election.cc:290] T d8f54e9ccd04475f826ee851b0f079c0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.147442 14551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f54e9ccd04475f826ee851b0f079c0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.147886 14551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15263a3b32be418288770ce348c082d7" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.148151 14412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15263a3b32be418288770ce348c082d7" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.148166 14682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d8f54e9ccd04475f826ee851b0f079c0" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:23.148258 14412 raft_consensus.cc:2463] T 15263a3b32be418288770ce348c082d7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
W20250115 03:06:23.148288 14626 leader_election.cc:343] T 15263a3b32be418288770ce348c082d7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:23.148523 14358 leader_election.cc:343] T d8f54e9ccd04475f826ee851b0f079c0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.148790 14628 leader_election.cc:304] T 15263a3b32be418288770ce348c082d7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.148998 14811 tablet_bootstrap.cc:492] T 815c5e096e4d4b0cbfb7c15bac571b10 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:23.149060 16868 raft_consensus.cc:2798] T 15263a3b32be418288770ce348c082d7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.149106 16868 raft_consensus.cc:491] T 15263a3b32be418288770ce348c082d7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.149070 14811 ts_tablet_manager.cc:1397] T 815c5e096e4d4b0cbfb7c15bac571b10 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.001s
I20250115 03:06:23.149150 16868 raft_consensus.cc:3054] T 15263a3b32be418288770ce348c082d7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:23.149156 14356 leader_election.cc:343] T d8f54e9ccd04475f826ee851b0f079c0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.149214 14356 leader_election.cc:304] T d8f54e9ccd04475f826ee851b0f079c0 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.149060 16881 raft_consensus.cc:491] T 1202288f2442477badeb35667dd08dab P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.149365 16881 raft_consensus.cc:513] T 1202288f2442477badeb35667dd08dab P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.149578 16846 raft_consensus.cc:2743] T d8f54e9ccd04475f826ee851b0f079c0 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:23.149576 14811 raft_consensus.cc:357] T 815c5e096e4d4b0cbfb7c15bac571b10 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.149662 14811 raft_consensus.cc:383] T 815c5e096e4d4b0cbfb7c15bac571b10 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.149690 16881 leader_election.cc:290] T 1202288f2442477badeb35667dd08dab P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:23.149701 14811 raft_consensus.cc:738] T 815c5e096e4d4b0cbfb7c15bac571b10 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.149960 14811 consensus_queue.cc:260] T 815c5e096e4d4b0cbfb7c15bac571b10 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.150012 14682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1202288f2442477badeb35667dd08dab" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:23.150106 14682 raft_consensus.cc:2463] T 1202288f2442477badeb35667dd08dab P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:23.150151 14412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1202288f2442477badeb35667dd08dab" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.150244 14412 raft_consensus.cc:2463] T 1202288f2442477badeb35667dd08dab P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:23.150346 14493 leader_election.cc:304] T 1202288f2442477badeb35667dd08dab P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:23.150650 14811 ts_tablet_manager.cc:1428] T 815c5e096e4d4b0cbfb7c15bac571b10 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.150873 14811 tablet_bootstrap.cc:492] T 7340ac89e5524c35948f3acc6c941cb5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:23.151788 14811 tablet_bootstrap.cc:654] T 7340ac89e5524c35948f3acc6c941cb5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.154274 14811 tablet_bootstrap.cc:492] T 7340ac89e5524c35948f3acc6c941cb5 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:23.154395 14811 ts_tablet_manager.cc:1397] T 7340ac89e5524c35948f3acc6c941cb5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20250115 03:06:23.154845 14811 raft_consensus.cc:357] T 7340ac89e5524c35948f3acc6c941cb5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.154985 14811 raft_consensus.cc:383] T 7340ac89e5524c35948f3acc6c941cb5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.155055 14811 raft_consensus.cc:738] T 7340ac89e5524c35948f3acc6c941cb5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.155221 14811 consensus_queue.cc:260] T 7340ac89e5524c35948f3acc6c941cb5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.155527 14811 ts_tablet_manager.cc:1428] T 7340ac89e5524c35948f3acc6c941cb5 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.155719 14811 tablet_bootstrap.cc:492] T 9c436975bd034068a6d58d5831db1896 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:23.156813 14811 tablet_bootstrap.cc:654] T 9c436975bd034068a6d58d5831db1896 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.157433 16885 raft_consensus.cc:491] T 015ec42caa1b468ea10f25c8f26e278a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.157564 16885 raft_consensus.cc:513] T 015ec42caa1b468ea10f25c8f26e278a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:23.157672 16617 consensus_meta.cc:287] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent flushing consensus metadata: real 0.653s user 0.000s sys 0.001s
I20250115 03:06:23.157721 16617 raft_consensus.cc:513] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.158077 16617 leader_election.cc:290] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
W20250115 03:06:23.158274 14688 consensus_meta.cc:287] T ca117aca95f640be880b640c85e18cd4 P 5c0dae7aff4b44c699275b95ba639770: Time spent flushing consensus metadata: real 0.757s user 0.001s sys 0.000s
I20250115 03:06:23.158331 14688 raft_consensus.cc:2463] T ca117aca95f640be880b640c85e18cd4 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:23.158615 16411 raft_consensus.cc:513] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.158916 16411 leader_election.cc:290] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 election: Requested vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:23.159027 14659 consensus_meta.cc:287] T 67ff648cc30b4f07a944005f53494e8c P 5c0dae7aff4b44c699275b95ba639770: Time spent flushing consensus metadata: real 0.980s user 0.000s sys 0.001s
W20250115 03:06:23.159065 14657 consensus_meta.cc:287] T 9099a50c9e9f487dba708e5cbdff329c P 5c0dae7aff4b44c699275b95ba639770: Time spent flushing consensus metadata: real 0.678s user 0.001s sys 0.000s
I20250115 03:06:23.159226 14659 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36830 (request call id 1059) took 1218 ms. Trace:
I20250115 03:06:23.159334 14657 tablet_service.cc:1514] Processing DeleteTablet for tablet b6175f691fec45bf9853ee90959a1c98 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:23.159595 14657 tablet_service.cc:1467] Processing CreateTablet for tablet 3f75507ac77a4a458e2394d3c9a16a91 (DEFAULT_TABLE table=default.Table_9d40428922614f03985ba101c29255a9 [id=b28583f60a0c4535a26249841af49923]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.159680 14493 leader_election.cc:304] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:23.159824 14657 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f75507ac77a4a458e2394d3c9a16a91. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.159900 16617 raft_consensus.cc:2798] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:23.159987 16617 raft_consensus.cc:695] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:23.160097 16617 consensus_queue.cc:237] T ca117aca95f640be880b640c85e18cd4 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.161407 14682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e170ced1612442ebcfdaff6d9826ea3" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:23.161513 14682 raft_consensus.cc:3054] T 3e170ced1612442ebcfdaff6d9826ea3 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:23.161662 14662 consensus_meta.cc:287] T 306e4240c66647bc85d862252518f499 P 5c0dae7aff4b44c699275b95ba639770: Time spent flushing consensus metadata: real 0.982s user 0.001s sys 0.000s
I20250115 03:06:23.161854 14662 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36830 (request call id 1068) took 1194 ms. Trace:
I20250115 03:06:23.161898 14662 rpcz_store.cc:276] 0115 03:06:21.967364 (+ 0us) service_pool.cc:168] Inserting onto call queue
0115 03:06:21.967444 (+ 80us) service_pool.cc:225] Handling call
0115 03:06:21.967692 (+ 248us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0115 03:06:21.967704 (+ 12us) ts_tablet_manager.cc:676] Creating new metadata...
0115 03:06:22.179402 (+211698us) tablet_metadata.cc:641] Metadata flushed
0115 03:06:23.161748 (+982346us) tablet_replica.cc:206] Creating consensus instance
0115 03:06:23.161845 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250115 03:06:23.162166 14662 tablet_service.cc:1467] Processing CreateTablet for tablet 1d3149d4e2744f7fa1032663ea2b46e7 (DEFAULT_TABLE table=default.Table_2faf003f0c414f759fe0c2e2115b3d85 [id=6b2ce97872bb414e96a4823cde9ab442]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.162366 14662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d3149d4e2744f7fa1032663ea2b46e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.163770 14391 tablet_service.cc:1514] Processing DeleteTablet for tablet ec0f3424ebcb442597a94ccdbb25eeb6 with delete_type TABLET_DATA_DELETED (replaced by tablet 0c5f5dfdf8234b93a878accd42c946f2) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.163882 14391 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f88a8cc2995427eb2082117e553812c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.163960 14391 tablet_service.cc:1514] Processing DeleteTablet for tablet 29c72f33a2034c4eae81c7dfcca27ca4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.164032 14391 tablet_service.cc:1514] Processing DeleteTablet for tablet 8efa998709034fe9bb37c7c608c99e76 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.164121 14391 tablet_service.cc:1514] Processing DeleteTablet for tablet 824fe68c383d4483991841a9f4c1b4ab with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.164199 14391 tablet_service.cc:1514] Processing DeleteTablet for tablet 84c592ced5b443869240eff55fe9d549 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.164255 16885 leader_election.cc:290] T 015ec42caa1b468ea10f25c8f26e278a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:23.164419 14391 tablet_service.cc:1467] Processing CreateTablet for tablet 9c2d3b2a4c7143af8dff42cb38e18e11 (DEFAULT_TABLE table=default.Table_2f4d77d2fd6d4dcb8bcee6f7214324cd [id=8e0361ded9dc4a399eafd156cdb018ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.164628 14417 raft_consensus.cc:2388] T f66defba523f4df996929f98c848f95a P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in current term 1: Already voted for candidate e8df7247ee9c43e0aa18216da2661f75 in this term.
I20250115 03:06:23.164784 14412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015ec42caa1b468ea10f25c8f26e278a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.164878 14412 raft_consensus.cc:2463] T 015ec42caa1b468ea10f25c8f26e278a P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.159293 14659 rpcz_store.cc:276] 0115 03:06:21.941215 (+ 0us) service_pool.cc:168] Inserting onto call queue
0115 03:06:21.941918 (+ 703us) service_pool.cc:225] Handling call
0115 03:06:21.942194 (+ 276us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0115 03:06:21.942210 (+ 16us) ts_tablet_manager.cc:676] Creating new metadata...
0115 03:06:22.178550 (+236340us) tablet_metadata.cc:641] Metadata flushed
0115 03:06:23.159130 (+980580us) tablet_replica.cc:206] Creating consensus instance
0115 03:06:23.159219 (+ 89us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250115 03:06:23.165285 14659 tablet_service.cc:1467] Processing CreateTablet for tablet 3c1a685ffa324f56be0d8e3c2827974b (DEFAULT_TABLE table=default.Table_9091a95606104776ab8c2479b53af477 [id=3707a028ba604e7db8dc77c059a60f4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.165481 14659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c1a685ffa324f56be0d8e3c2827974b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.166265 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 4114f1b9321f4618b32da6bb341cd5cf (DEFAULT_TABLE table=default.Table_82eeb56321174250b56cbfba4d9d1744 [id=f93028a86db549408ba35a13a474ddde]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.166703 14551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "015ec42caa1b468ea10f25c8f26e278a" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.166810 14551 raft_consensus.cc:2463] T 015ec42caa1b468ea10f25c8f26e278a P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.167333 14626 leader_election.cc:304] T 015ec42caa1b468ea10f25c8f26e278a P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:23.167660 16847 raft_consensus.cc:2798] T 015ec42caa1b468ea10f25c8f26e278a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.167896 16847 raft_consensus.cc:491] T 015ec42caa1b468ea10f25c8f26e278a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.167963 16847 raft_consensus.cc:3054] T 015ec42caa1b468ea10f25c8f26e278a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.168530 14688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66defba523f4df996929f98c848f95a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:23.168635 14688 raft_consensus.cc:3054] T f66defba523f4df996929f98c848f95a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.168888 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_a0656bab78454d5aa134b66287b01eb0" } new_table_name: "default.Table_3f3e7d7a59c24b1096b08267b575f7ff" modify_external_catalogs: true
I20250115 03:06:23.169009 16855 raft_consensus.cc:2798] T 1202288f2442477badeb35667dd08dab P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.169186 16855 raft_consensus.cc:491] T 1202288f2442477badeb35667dd08dab P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.169286 16855 raft_consensus.cc:3054] T 1202288f2442477badeb35667dd08dab P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.169783 14811 tablet_bootstrap.cc:492] T 9c436975bd034068a6d58d5831db1896 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:23.169911 14811 ts_tablet_manager.cc:1397] T 9c436975bd034068a6d58d5831db1896 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20250115 03:06:23.170377 14811 raft_consensus.cc:357] T 9c436975bd034068a6d58d5831db1896 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.170526 14811 raft_consensus.cc:383] T 9c436975bd034068a6d58d5831db1896 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.170601 14811 raft_consensus.cc:738] T 9c436975bd034068a6d58d5831db1896 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.170795 14811 consensus_queue.cc:260] T 9c436975bd034068a6d58d5831db1896 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.171125 14811 ts_tablet_manager.cc:1428] T 9c436975bd034068a6d58d5831db1896 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.171350 14811 tablet_bootstrap.cc:492] T 15263a3b32be418288770ce348c082d7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:23.172681 14811 tablet_bootstrap.cc:654] T 15263a3b32be418288770ce348c082d7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:23.173184 14165 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2ae7d8e213544722bb30bfc7f0f4a9c4 consensus state has no leader
I20250115 03:06:23.174360 14551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66defba523f4df996929f98c848f95a" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1"
W20250115 03:06:23.174425 14493 leader_election.cc:343] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 3e170ced1612442ebcfdaff6d9826ea3
I20250115 03:06:23.174959 16411 raft_consensus.cc:491] T d3728e87eecc4bc48bc1944907ceb8bd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.175036 16411 raft_consensus.cc:513] T d3728e87eecc4bc48bc1944907ceb8bd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.175330 16411 leader_election.cc:290] T d3728e87eecc4bc48bc1944907ceb8bd P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.175655 14680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3728e87eecc4bc48bc1944907ceb8bd" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:23.176733 14809 tablet_bootstrap.cc:492] T f25edfc3d7c043bfa337b31fa9131f1d P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:23.176821 14809 ts_tablet_manager.cc:1397] T f25edfc3d7c043bfa337b31fa9131f1d P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.049s user 0.002s sys 0.000s
I20250115 03:06:23.177062 16874 consensus_queue.cc:1035] T 781f6a9148cd4bd684f24c294c0b019b P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:23.177331 14809 raft_consensus.cc:357] T f25edfc3d7c043bfa337b31fa9131f1d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.177426 14809 raft_consensus.cc:383] T f25edfc3d7c043bfa337b31fa9131f1d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.177459 14809 raft_consensus.cc:738] T f25edfc3d7c043bfa337b31fa9131f1d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.177536 14550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3728e87eecc4bc48bc1944907ceb8bd" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.177570 14809 consensus_queue.cc:260] T f25edfc3d7c043bfa337b31fa9131f1d P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.177881 14809 ts_tablet_manager.cc:1428] T f25edfc3d7c043bfa337b31fa9131f1d P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20250115 03:06:23.177927 14358 leader_election.cc:343] T d3728e87eecc4bc48bc1944907ceb8bd P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.178049 14809 tablet_bootstrap.cc:492] T 1fd348b3ae2a45dfa05c52f02536a201 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
W20250115 03:06:23.178148 14356 leader_election.cc:343] T d3728e87eecc4bc48bc1944907ceb8bd P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.178220 14356 leader_election.cc:304] T d3728e87eecc4bc48bc1944907ceb8bd P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.178478 16411 raft_consensus.cc:2743] T d3728e87eecc4bc48bc1944907ceb8bd P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:23.178972 14809 tablet_bootstrap.cc:654] T 1fd348b3ae2a45dfa05c52f02536a201 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.181383 16406 raft_consensus.cc:491] T 4535ac88f78f48da83d9d2f951afdfa6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.181452 16406 raft_consensus.cc:513] T 4535ac88f78f48da83d9d2f951afdfa6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.181684 14812 tablet_bootstrap.cc:492] T 9ab3c40051ad42f692cebbd96144b60a P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.181763 16406 leader_election.cc:290] T 4535ac88f78f48da83d9d2f951afdfa6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.181775 14812 ts_tablet_manager.cc:1397] T 9ab3c40051ad42f692cebbd96144b60a P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.053s user 0.000s sys 0.002s
I20250115 03:06:23.182191 14812 raft_consensus.cc:357] T 9ab3c40051ad42f692cebbd96144b60a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.182279 14812 raft_consensus.cc:383] T 9ab3c40051ad42f692cebbd96144b60a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.182307 14812 raft_consensus.cc:738] T 9ab3c40051ad42f692cebbd96144b60a P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.182418 14812 consensus_queue.cc:260] T 9ab3c40051ad42f692cebbd96144b60a P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.182451 14680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4535ac88f78f48da83d9d2f951afdfa6" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:23.182693 14812 ts_tablet_manager.cc:1428] T 9ab3c40051ad42f692cebbd96144b60a P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20250115 03:06:23.182823 14358 leader_election.cc:343] T 4535ac88f78f48da83d9d2f951afdfa6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.182855 14812 tablet_bootstrap.cc:492] T b9430861457247d9a324b43bd83fbae0 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:23.183780 14812 tablet_bootstrap.cc:654] T b9430861457247d9a324b43bd83fbae0 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.184381 14550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4535ac88f78f48da83d9d2f951afdfa6" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:23.184826 14356 leader_election.cc:343] T 4535ac88f78f48da83d9d2f951afdfa6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.184897 14356 leader_election.cc:304] T 4535ac88f78f48da83d9d2f951afdfa6 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.185103 16846 raft_consensus.cc:2743] T 4535ac88f78f48da83d9d2f951afdfa6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:23.185905 14812 tablet_bootstrap.cc:492] T b9430861457247d9a324b43bd83fbae0 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.185974 14812 ts_tablet_manager.cc:1397] T b9430861457247d9a324b43bd83fbae0 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20250115 03:06:23.186347 14812 raft_consensus.cc:357] T b9430861457247d9a324b43bd83fbae0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.186434 14812 raft_consensus.cc:383] T b9430861457247d9a324b43bd83fbae0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.186465 14812 raft_consensus.cc:738] T b9430861457247d9a324b43bd83fbae0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.186565 14812 consensus_queue.cc:260] T b9430861457247d9a324b43bd83fbae0 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.186846 14812 ts_tablet_manager.cc:1428] T b9430861457247d9a324b43bd83fbae0 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.186995 14812 tablet_bootstrap.cc:492] T 4535ac88f78f48da83d9d2f951afdfa6 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:23.187945 14812 tablet_bootstrap.cc:654] T 4535ac88f78f48da83d9d2f951afdfa6 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.189432 14812 tablet_bootstrap.cc:492] T 4535ac88f78f48da83d9d2f951afdfa6 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.189469 14809 tablet_bootstrap.cc:492] T 1fd348b3ae2a45dfa05c52f02536a201 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:23.189507 14812 ts_tablet_manager.cc:1397] T 4535ac88f78f48da83d9d2f951afdfa6 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:23.189539 14809 ts_tablet_manager.cc:1397] T 1fd348b3ae2a45dfa05c52f02536a201 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20250115 03:06:23.189862 14812 raft_consensus.cc:357] T 4535ac88f78f48da83d9d2f951afdfa6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.189941 14812 raft_consensus.cc:383] T 4535ac88f78f48da83d9d2f951afdfa6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.189965 14812 raft_consensus.cc:738] T 4535ac88f78f48da83d9d2f951afdfa6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.189940 14809 raft_consensus.cc:357] T 1fd348b3ae2a45dfa05c52f02536a201 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.190027 14809 raft_consensus.cc:383] T 1fd348b3ae2a45dfa05c52f02536a201 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.190060 14809 raft_consensus.cc:738] T 1fd348b3ae2a45dfa05c52f02536a201 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.190057 14812 consensus_queue.cc:260] T 4535ac88f78f48da83d9d2f951afdfa6 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.190165 14809 consensus_queue.cc:260] T 1fd348b3ae2a45dfa05c52f02536a201 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.190296 14812 ts_tablet_manager.cc:1428] T 4535ac88f78f48da83d9d2f951afdfa6 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:23.190374 14809 ts_tablet_manager.cc:1428] T 1fd348b3ae2a45dfa05c52f02536a201 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.190429 14812 tablet_bootstrap.cc:492] T b142fa43a7734d1fb6e3d04bc720a74f P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:23.190529 14809 tablet_bootstrap.cc:492] T 00aad81f4ca3427cb683de71148cff5c P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:23.191241 14812 tablet_bootstrap.cc:654] T b142fa43a7734d1fb6e3d04bc720a74f P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.193300 14165 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_77761cbb0c804fa4a8d4298c4db47f76" } modify_external_catalogs: true
I20250115 03:06:23.193413 14165 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_77761cbb0c804fa4a8d4298c4db47f76" } modify_external_catalogs: true
I20250115 03:06:23.193706 14809 tablet_bootstrap.cc:654] T 00aad81f4ca3427cb683de71148cff5c P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.200147 14165 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 21524aef80f74a6c899ea66b9b022389
I20250115 03:06:23.202374 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_4479fcf9dae14e2b82d8400944206205" } new_table_name: "default.Table_5d0aace607fb49c8b751e34a6425be42" modify_external_catalogs: true
I20250115 03:06:23.203699 14811 tablet_bootstrap.cc:492] T 15263a3b32be418288770ce348c082d7 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:23.203786 14811 ts_tablet_manager.cc:1397] T 15263a3b32be418288770ce348c082d7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.032s user 0.001s sys 0.001s
I20250115 03:06:23.204299 14811 raft_consensus.cc:357] T 15263a3b32be418288770ce348c082d7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.204392 14811 raft_consensus.cc:383] T 15263a3b32be418288770ce348c082d7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.204435 14811 raft_consensus.cc:738] T 15263a3b32be418288770ce348c082d7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.204540 14811 consensus_queue.cc:260] T 15263a3b32be418288770ce348c082d7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.204807 14811 ts_tablet_manager.cc:1428] T 15263a3b32be418288770ce348c082d7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:23.204984 14811 tablet_bootstrap.cc:492] T b9f67c35591f4fdeb8a926a085171b49 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:23.205835 14811 tablet_bootstrap.cc:654] T b9f67c35591f4fdeb8a926a085171b49 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:23.210582 14165 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c8c73857d8184dc8a535b562b966ff4f consensus state has no leader
I20250115 03:06:23.211081 16885 raft_consensus.cc:491] T d5e4743e016e49ed8ba26b1ac96b1470 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.211305 16885 raft_consensus.cc:513] T d5e4743e016e49ed8ba26b1ac96b1470 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.212275 16406 raft_consensus.cc:491] T 7ac903c81a1f481ab587e6772c306a96 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.212352 16406 raft_consensus.cc:513] T 7ac903c81a1f481ab587e6772c306a96 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.212528 14550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e4743e016e49ed8ba26b1ac96b1470" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.212669 16406 leader_election.cc:290] T 7ac903c81a1f481ab587e6772c306a96 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.213119 14681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ac903c81a1f481ab587e6772c306a96" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:23.213446 14811 tablet_bootstrap.cc:492] T b9f67c35591f4fdeb8a926a085171b49 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:23.213289 16885 leader_election.cc:290] T d5e4743e016e49ed8ba26b1ac96b1470 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:23.213447 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5e4743e016e49ed8ba26b1ac96b1470" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.213522 14811 ts_tablet_manager.cc:1397] T b9f67c35591f4fdeb8a926a085171b49 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s
I20250115 03:06:23.213567 14420 raft_consensus.cc:2463] T d5e4743e016e49ed8ba26b1ac96b1470 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.213578 16885 raft_consensus.cc:491] T ef0994b1e51b4c688354998730e7b743 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.213761 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ac903c81a1f481ab587e6772c306a96" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.213848 14628 leader_election.cc:304] T d5e4743e016e49ed8ba26b1ac96b1470 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:23.213938 14811 raft_consensus.cc:357] T b9f67c35591f4fdeb8a926a085171b49 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.214025 14811 raft_consensus.cc:383] T b9f67c35591f4fdeb8a926a085171b49 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.214053 14811 raft_consensus.cc:738] T b9f67c35591f4fdeb8a926a085171b49 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.214028 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0c288c0d91374c20be64d0ec492a7260" } modify_external_catalogs: true
I20250115 03:06:23.214061 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_dc41872d048942e7851fd8d74fce612b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:23.214120 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_0c288c0d91374c20be64d0ec492a7260" } modify_external_catalogs: true
I20250115 03:06:23.214176 14811 consensus_queue.cc:260] T b9f67c35591f4fdeb8a926a085171b49 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:23.214367 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dc41872d048942e7851fd8d74fce612b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:23.214424 14811 ts_tablet_manager.cc:1428] T b9f67c35591f4fdeb8a926a085171b49 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.214570 14811 tablet_bootstrap.cc:492] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
W20250115 03:06:23.213465 14358 leader_election.cc:343] T 7ac903c81a1f481ab587e6772c306a96 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.215605 14811 tablet_bootstrap.cc:654] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.216015 16885 raft_consensus.cc:513] T ef0994b1e51b4c688354998730e7b743 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.216841 14549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef0994b1e51b4c688354998730e7b743" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.216883 16900 raft_consensus.cc:2798] T d5e4743e016e49ed8ba26b1ac96b1470 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.216962 16900 raft_consensus.cc:491] T d5e4743e016e49ed8ba26b1ac96b1470 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.217013 16900 raft_consensus.cc:3054] T d5e4743e016e49ed8ba26b1ac96b1470 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.216989 14549 raft_consensus.cc:2463] T ef0994b1e51b4c688354998730e7b743 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.216902 14165 catalog_manager.cc:5526] T 781f6a9148cd4bd684f24c294c0b019b P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } }
I20250115 03:06:23.217084 14165 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 781f6a9148cd4bd684f24c294c0b019b (table default.Table_d64c7e1e71a3478687247bda551f3fe2 [id=b9f32421ef8a4c549dab58ffb0f71fa7]). Expected version 1 got 0
I20250115 03:06:23.217674 14626 leader_election.cc:304] T ef0994b1e51b4c688354998730e7b743 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
W20250115 03:06:23.218689 14356 leader_election.cc:343] T 7ac903c81a1f481ab587e6772c306a96 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.218773 14356 leader_election.cc:304] T 7ac903c81a1f481ab587e6772c306a96 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.218955 16846 raft_consensus.cc:2743] T 7ac903c81a1f481ab587e6772c306a96 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:23.220409 16890 raft_consensus.cc:2798] T ef0994b1e51b4c688354998730e7b743 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.220479 16890 raft_consensus.cc:491] T ef0994b1e51b4c688354998730e7b743 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.220525 16890 raft_consensus.cc:3054] T ef0994b1e51b4c688354998730e7b743 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.220625 14167 catalog_manager.cc:5808] Not sending DeleteTablet requests; no consensus state for tablet a82a3f274c4e409db668884da51c06c1
I20250115 03:06:23.221165 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef0994b1e51b4c688354998730e7b743" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.221266 14420 raft_consensus.cc:2463] T ef0994b1e51b4c688354998730e7b743 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.221383 14811 tablet_bootstrap.cc:492] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:23.221454 14811 ts_tablet_manager.cc:1397] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20250115 03:06:23.221534 14809 tablet_bootstrap.cc:492] T 00aad81f4ca3427cb683de71148cff5c P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:23.221607 14809 ts_tablet_manager.cc:1397] T 00aad81f4ca3427cb683de71148cff5c P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.000s
I20250115 03:06:23.221858 14811 raft_consensus.cc:357] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.221951 14811 raft_consensus.cc:383] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.221980 14811 raft_consensus.cc:738] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.221995 14809 raft_consensus.cc:357] T 00aad81f4ca3427cb683de71148cff5c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.222074 14809 raft_consensus.cc:383] T 00aad81f4ca3427cb683de71148cff5c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.222102 14809 raft_consensus.cc:738] T 00aad81f4ca3427cb683de71148cff5c P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.222110 14811 consensus_queue.cc:260] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.222198 14809 consensus_queue.cc:260] T 00aad81f4ca3427cb683de71148cff5c P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.222364 14811 ts_tablet_manager.cc:1428] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.222510 14811 tablet_bootstrap.cc:492] T 2b0e3df3c79c40ba958b6e2453fa666f P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:23.223341 14811 tablet_bootstrap.cc:654] T 2b0e3df3c79c40ba958b6e2453fa666f P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.224692 14809 ts_tablet_manager.cc:1428] T 00aad81f4ca3427cb683de71148cff5c P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:23.224857 14809 tablet_bootstrap.cc:492] T ef7cd35eb1614a29a9197e9c03185f7d P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:23.225762 14809 tablet_bootstrap.cc:654] T ef7cd35eb1614a29a9197e9c03185f7d P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.225821 14812 tablet_bootstrap.cc:492] T b142fa43a7734d1fb6e3d04bc720a74f P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.225890 14812 ts_tablet_manager.cc:1397] T b142fa43a7734d1fb6e3d04bc720a74f P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.035s user 0.001s sys 0.000s
I20250115 03:06:23.226279 14812 raft_consensus.cc:357] T b142fa43a7734d1fb6e3d04bc720a74f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.226367 14812 raft_consensus.cc:383] T b142fa43a7734d1fb6e3d04bc720a74f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.226397 14812 raft_consensus.cc:738] T b142fa43a7734d1fb6e3d04bc720a74f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.226724 14812 consensus_queue.cc:260] T b142fa43a7734d1fb6e3d04bc720a74f P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.219286 16885 leader_election.cc:290] T ef0994b1e51b4c688354998730e7b743 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:23.228257 14811 tablet_bootstrap.cc:492] T 2b0e3df3c79c40ba958b6e2453fa666f P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:23.228367 14811 ts_tablet_manager.cc:1397] T 2b0e3df3c79c40ba958b6e2453fa666f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20250115 03:06:23.228775 14811 raft_consensus.cc:357] T 2b0e3df3c79c40ba958b6e2453fa666f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.228914 14811 raft_consensus.cc:383] T 2b0e3df3c79c40ba958b6e2453fa666f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.228737 14812 ts_tablet_manager.cc:1428] T b142fa43a7734d1fb6e3d04bc720a74f P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20250115 03:06:23.229009 14811 raft_consensus.cc:738] T 2b0e3df3c79c40ba958b6e2453fa666f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.229140 14811 consensus_queue.cc:260] T 2b0e3df3c79c40ba958b6e2453fa666f P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.229467 14811 ts_tablet_manager.cc:1428] T 2b0e3df3c79c40ba958b6e2453fa666f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:23.229768 14812 tablet_bootstrap.cc:492] T daea48ecc69d441ab66fcb40ea2e7a66 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:23.229830 14811 tablet_bootstrap.cc:492] T 85fcafa24fa1448c995d30dd93b0737f P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:23.230796 14811 tablet_bootstrap.cc:654] T 85fcafa24fa1448c995d30dd93b0737f P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.234102 16846 raft_consensus.cc:491] T cc328a01837a4c21abc38c7d85557e15 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.234184 16846 raft_consensus.cc:513] T cc328a01837a4c21abc38c7d85557e15 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.234515 16846 leader_election.cc:290] T cc328a01837a4c21abc38c7d85557e15 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.234745 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_39b71235b2df40ec88411bb02d370d7b" } new_table_name: "default.Table_e9b688cf9aa84f4c929250bf0e7c8227" modify_external_catalogs: true
I20250115 03:06:23.235103 14681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc328a01837a4c21abc38c7d85557e15" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:23.235415 14358 leader_election.cc:343] T cc328a01837a4c21abc38c7d85557e15 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.235546 14812 tablet_bootstrap.cc:654] T daea48ecc69d441ab66fcb40ea2e7a66 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.236507 14549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc328a01837a4c21abc38c7d85557e15" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:23.236804 14356 leader_election.cc:343] T cc328a01837a4c21abc38c7d85557e15 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.236876 14356 leader_election.cc:304] T cc328a01837a4c21abc38c7d85557e15 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.237087 16846 raft_consensus.cc:2743] T cc328a01837a4c21abc38c7d85557e15 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:23.237520 14809 tablet_bootstrap.cc:492] T ef7cd35eb1614a29a9197e9c03185f7d P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:23.237615 14809 ts_tablet_manager.cc:1397] T ef7cd35eb1614a29a9197e9c03185f7d P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20250115 03:06:23.238010 14809 raft_consensus.cc:357] T ef7cd35eb1614a29a9197e9c03185f7d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.238197 14809 raft_consensus.cc:383] T ef7cd35eb1614a29a9197e9c03185f7d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.238242 14809 raft_consensus.cc:738] T ef7cd35eb1614a29a9197e9c03185f7d P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.238428 14809 consensus_queue.cc:260] T ef7cd35eb1614a29a9197e9c03185f7d P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.238485 16885 raft_consensus.cc:491] T ec0f3424ebcb442597a94ccdbb25eeb6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.239152 14809 ts_tablet_manager.cc:1428] T ef7cd35eb1614a29a9197e9c03185f7d P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.239318 14809 tablet_bootstrap.cc:492] T 3724ff14f535416e859339a137c05050 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
W20250115 03:06:23.239457 14167 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a8a5743475814401ac2c0a51cfd70207 consensus state has no leader
I20250115 03:06:23.240427 14809 tablet_bootstrap.cc:654] T 3724ff14f535416e859339a137c05050 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.239748 16885 raft_consensus.cc:513] T ec0f3424ebcb442597a94ccdbb25eeb6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.242676 14549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec0f3424ebcb442597a94ccdbb25eeb6" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:23.243064 14626 leader_election.cc:343] T ec0f3424ebcb442597a94ccdbb25eeb6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.243770 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec0f3424ebcb442597a94ccdbb25eeb6" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.243875 14420 raft_consensus.cc:2463] T ec0f3424ebcb442597a94ccdbb25eeb6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.243806 16885 leader_election.cc:290] T ec0f3424ebcb442597a94ccdbb25eeb6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:23.244123 14628 leader_election.cc:304] T ec0f3424ebcb442597a94ccdbb25eeb6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.245002 16846 raft_consensus.cc:491] T 4febfae2f2794669b5997a1fa7ef7412 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.245082 16846 raft_consensus.cc:513] T 4febfae2f2794669b5997a1fa7ef7412 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.244362 16885 raft_consensus.cc:2798] T ec0f3424ebcb442597a94ccdbb25eeb6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.245397 16846 leader_election.cc:290] T 4febfae2f2794669b5997a1fa7ef7412 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.245882 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e7b6ace2c7c04dbfa9e87c6a4697a243" } modify_external_catalogs: true
I20250115 03:06:23.245983 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_e7b6ace2c7c04dbfa9e87c6a4697a243" } modify_external_catalogs: true
I20250115 03:06:23.246829 14549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4febfae2f2794669b5997a1fa7ef7412" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:23.247447 14356 leader_election.cc:343] T 4febfae2f2794669b5997a1fa7ef7412 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.245434 16885 raft_consensus.cc:491] T ec0f3424ebcb442597a94ccdbb25eeb6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.245959 14681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4febfae2f2794669b5997a1fa7ef7412" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:23.247737 16885 raft_consensus.cc:3054] T ec0f3424ebcb442597a94ccdbb25eeb6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:23.248067 14358 leader_election.cc:343] T 4febfae2f2794669b5997a1fa7ef7412 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.248138 14358 leader_election.cc:304] T 4febfae2f2794669b5997a1fa7ef7412 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.248315 16846 raft_consensus.cc:2743] T 4febfae2f2794669b5997a1fa7ef7412 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:23.252108 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet 836541bcc82d4644b4c5eb2f3f25c030
I20250115 03:06:23.253201 14167 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_93bd5dbdd2344875ba2d8d8a3d0a7591" } modify_external_catalogs: true
I20250115 03:06:23.253355 14167 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_93bd5dbdd2344875ba2d8d8a3d0a7591" } modify_external_catalogs: true
I20250115 03:06:23.254446 14819 ts_tablet_manager.cc:1918] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20250115 03:06:23.254521 14819 log.cc:1198] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75: Deleting WAL directory at /tmp/dist-test-taskE2XgYg/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1736910369671392-14059-0/minicluster-data/ts-0/wal/wals/78cc96de8b414ac19285338ccb1024e0
I20250115 03:06:23.254760 14819 ts_tablet_manager.cc:1939] T 78cc96de8b414ac19285338ccb1024e0 P e8df7247ee9c43e0aa18216da2661f75: Deleting consensus metadata
W20250115 03:06:23.254916 14858 consensus_meta.cc:287] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent flushing consensus metadata: real 0.726s user 0.001s sys 0.000s
I20250115 03:06:23.254974 14858 raft_consensus.cc:513] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.255311 14858 leader_election.cc:290] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:23.255523 14682 raft_consensus.cc:2463] T 3e170ced1612442ebcfdaff6d9826ea3 P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:23.255779 14416 raft_consensus.cc:2463] T aadcec8aca2b4ece915707c17f60b8b5 P e8df7247ee9c43e0aa18216da2661f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:23.256184 14493 leader_election.cc:304] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:23.258318 14391 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c2d3b2a4c7143af8dff42cb38e18e11. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.258484 16871 raft_consensus.cc:2798] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:23.258565 16871 raft_consensus.cc:695] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:23.258671 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4114f1b9321f4618b32da6bb341cd5cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.258703 16871 consensus_queue.cc:237] T aadcec8aca2b4ece915707c17f60b8b5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.259506 14168 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_47fd03c657214fea893d4881cbbff5f1" } modify_external_catalogs: true
I20250115 03:06:23.259594 14168 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_47fd03c657214fea893d4881cbbff5f1" } modify_external_catalogs: true
I20250115 03:06:23.261246 14681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c580cdf3a854df988df32a9294c83bc" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:23.261345 14681 raft_consensus.cc:3054] T 0c580cdf3a854df988df32a9294c83bc P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
W20250115 03:06:23.261971 14672 consensus_meta.cc:287] T db486cc5a4f84ab6ac6d2c6aa02eca3d P 5c0dae7aff4b44c699275b95ba639770: Time spent flushing consensus metadata: real 1.082s user 0.001s sys 0.000s
I20250115 03:06:23.262164 14672 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36830 (request call id 1067) took 1295 ms. Trace:
I20250115 03:06:23.262210 14672 rpcz_store.cc:276] 0115 03:06:21.966359 (+ 0us) service_pool.cc:168] Inserting onto call queue
0115 03:06:21.966438 (+ 79us) service_pool.cc:225] Handling call
0115 03:06:21.966688 (+ 250us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0115 03:06:21.966703 (+ 15us) ts_tablet_manager.cc:676] Creating new metadata...
0115 03:06:22.180260 (+213557us) tablet_metadata.cc:641] Metadata flushed
0115 03:06:23.262075 (+1081815us) tablet_replica.cc:206] Creating consensus instance
0115 03:06:23.262157 (+ 82us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250115 03:06:23.262359 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b4eb3b20bb4908884e5716e7b2a369 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:23.262475 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ba7e498bcf4359a1a588c1daea3fb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:23.262552 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet b2d42233d2234f7da262753c3187be43 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:23.262619 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet cd07196be4b54a90865aecea55493aa1 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:23.262836 14672 tablet_service.cc:1467] Processing CreateTablet for tablet a97b4ad4327446cbae405f3d43a17781 (DEFAULT_TABLE table=default.Table_4f962cfd7b5b45629185faf7991c8bd3 [id=a2abece3dc784cb8a7c503b92a39bbf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.263059 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a97b4ad4327446cbae405f3d43a17781. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.265300 16889 raft_consensus.cc:491] T b142fa43a7734d1fb6e3d04bc720a74f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.265372 16889 raft_consensus.cc:513] T b142fa43a7734d1fb6e3d04bc720a74f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.265592 14493 leader_election.cc:304] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:23.267194 16871 raft_consensus.cc:2798] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:23.267194 16889 leader_election.cc:290] T b142fa43a7734d1fb6e3d04bc720a74f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:23.267270 16871 raft_consensus.cc:695] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:23.267395 16871 consensus_queue.cc:237] T 3e170ced1612442ebcfdaff6d9826ea3 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.269199 14167 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet e179d1fd2d614a0e97112256babaf2a3
I20250115 03:06:23.269588 14168 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 147c86b98f604d518e71f2ba3121fc21: Sending DeleteTablet for 3 replicas of tablet a142f9f7321a484ba7fde981ecc417e4
I20250115 03:06:23.273711 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_3a2da4c077894d14b522cb51f6623b4d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:23.274016 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a2da4c077894d14b522cb51f6623b4d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:23.275502 14812 tablet_bootstrap.cc:492] T daea48ecc69d441ab66fcb40ea2e7a66 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.275684 14812 ts_tablet_manager.cc:1397] T daea48ecc69d441ab66fcb40ea2e7a66 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.046s user 0.002s sys 0.001s
I20250115 03:06:23.277408 14812 raft_consensus.cc:357] T daea48ecc69d441ab66fcb40ea2e7a66 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.278890 14812 raft_consensus.cc:383] T daea48ecc69d441ab66fcb40ea2e7a66 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.278959 14812 raft_consensus.cc:738] T daea48ecc69d441ab66fcb40ea2e7a66 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.279161 14812 consensus_queue.cc:260] T daea48ecc69d441ab66fcb40ea2e7a66 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.279626 14812 ts_tablet_manager.cc:1428] T daea48ecc69d441ab66fcb40ea2e7a66 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20250115 03:06:23.280002 14812 tablet_bootstrap.cc:492] T 293a76d0d49e4a12ab951cbc7311d0f2 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:23.282606 16889 raft_consensus.cc:491] T 820d5af6daa74cadb53d2a5229854a60 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.282721 16889 raft_consensus.cc:513] T 820d5af6daa74cadb53d2a5229854a60 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.284001 16889 leader_election.cc:290] T 820d5af6daa74cadb53d2a5229854a60 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:23.285231 14812 tablet_bootstrap.cc:654] T 293a76d0d49e4a12ab951cbc7311d0f2 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.285840 16889 raft_consensus.cc:491] T 12b78b3abfe84714a97ecc89f2fb9e33 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.286146 16883 raft_consensus.cc:491] T 7340ac89e5524c35948f3acc6c941cb5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.285953 16889 raft_consensus.cc:513] T 12b78b3abfe84714a97ecc89f2fb9e33 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.286234 16883 raft_consensus.cc:513] T 7340ac89e5524c35948f3acc6c941cb5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.286552 16883 leader_election.cc:290] T 7340ac89e5524c35948f3acc6c941cb5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.287256 16889 leader_election.cc:290] T 12b78b3abfe84714a97ecc89f2fb9e33 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:23.288847 14549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b142fa43a7734d1fb6e3d04bc720a74f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.289337 14547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820d5af6daa74cadb53d2a5229854a60" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.289448 14547 raft_consensus.cc:2463] T 820d5af6daa74cadb53d2a5229854a60 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.290019 14549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12b78b3abfe84714a97ecc89f2fb9e33" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:23.290112 14493 leader_election.cc:343] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): Not found: Tablet not found: 0c580cdf3a854df988df32a9294c83bc
W20250115 03:06:23.290977 14626 leader_election.cc:343] T b142fa43a7734d1fb6e3d04bc720a74f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.291213 14676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7340ac89e5524c35948f3acc6c941cb5" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:23.291329 14676 raft_consensus.cc:2463] T 7340ac89e5524c35948f3acc6c941cb5 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:23.294127 14812 tablet_bootstrap.cc:492] T 293a76d0d49e4a12ab951cbc7311d0f2 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.294252 14812 ts_tablet_manager.cc:1397] T 293a76d0d49e4a12ab951cbc7311d0f2 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
I20250115 03:06:23.295003 14812 raft_consensus.cc:357] T 293a76d0d49e4a12ab951cbc7311d0f2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.295130 14812 raft_consensus.cc:383] T 293a76d0d49e4a12ab951cbc7311d0f2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.295179 14812 raft_consensus.cc:738] T 293a76d0d49e4a12ab951cbc7311d0f2 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.295385 14812 consensus_queue.cc:260] T 293a76d0d49e4a12ab951cbc7311d0f2 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.296424 14812 ts_tablet_manager.cc:1428] T 293a76d0d49e4a12ab951cbc7311d0f2 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 03:06:23.296697 14812 tablet_bootstrap.cc:492] T d140be0fab694f59b8a535392970fad0 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:23.297186 14626 leader_election.cc:304] T 820d5af6daa74cadb53d2a5229854a60 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
W20250115 03:06:23.297360 14626 leader_election.cc:343] T 12b78b3abfe84714a97ecc89f2fb9e33 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.297788 14493 leader_election.cc:304] T 7340ac89e5524c35948f3acc6c941cb5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters:
I20250115 03:06:23.298081 16871 consensus_queue.cc:1035] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:23.298435 16871 raft_consensus.cc:2798] T 7340ac89e5524c35948f3acc6c941cb5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.298485 16871 raft_consensus.cc:491] T 7340ac89e5524c35948f3acc6c941cb5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.298520 16871 raft_consensus.cc:3054] T 7340ac89e5524c35948f3acc6c941cb5 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.298765 16883 consensus_queue.cc:1035] T 68a04d57feea40eeb8e06cb6a2c61119 P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:23.299059 14812 tablet_bootstrap.cc:654] T d140be0fab694f59b8a535392970fad0 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.300871 16904 raft_consensus.cc:2798] T 820d5af6daa74cadb53d2a5229854a60 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.260914 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet f118a169019d4a32aa86b4fe9314f747 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:23 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.263072 14809 tablet_bootstrap.cc:492] T 3724ff14f535416e859339a137c05050 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:23.301126 14809 ts_tablet_manager.cc:1397] T 3724ff14f535416e859339a137c05050 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.062s user 0.002s sys 0.000s
I20250115 03:06:23.301134 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 21524aef80f74a6c899ea66b9b022389 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:23 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.301215 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fd348b3ae2a45dfa05c52f02536a201 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.301278 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 00aad81f4ca3427cb683de71148cff5c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:21 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.301335 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet 836541bcc82d4644b4c5eb2f3f25c030 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:23 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.301517 14398 tablet_service.cc:1467] Processing CreateTablet for tablet 770a1f9cdf6b427a9e4e59b7eee855eb (DEFAULT_TABLE table=default.Table_b3c0e0a301084e008804b31efbcbca00 [id=13ee55966c0047bab1b07f7060a363f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.301743 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 770a1f9cdf6b427a9e4e59b7eee855eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.303589 14398 tablet_service.cc:1514] Processing DeleteTablet for tablet ef7cd35eb1614a29a9197e9c03185f7d with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.303786 14398 tablet_service.cc:1467] Processing CreateTablet for tablet f920f74ae5474937981bbd9f7c666ef8 (DEFAULT_TABLE table=default.Table_af4c136524d54a0a9a31eabc483319fa [id=f0c68406fe8e4dcd98032ce49c5df5d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.303978 14398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f920f74ae5474937981bbd9f7c666ef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.268806 14384 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fa5ce72e20b4e0092b39c64160f0995 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:23 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.305437 14384 tablet_service.cc:1514] Processing DeleteTablet for tablet e179d1fd2d614a0e97112256babaf2a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:23 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.305519 14384 tablet_service.cc:1514] Processing DeleteTablet for tablet a142f9f7321a484ba7fde981ecc417e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:23 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.269750 14383 tablet_service.cc:1514] Processing DeleteTablet for tablet ad0e9216bd154f4a8a37874d3d8a08c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:23 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.270639 14389 tablet_service.cc:1467] Processing CreateTablet for tablet 1672330a6a3443c9a4fd6455c60e7245 (DEFAULT_TABLE table=default.Table_9090fd5bbab14ac59398d8e07b2816fb [id=6983d22b28744258b18cd9bc77d4f13b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.305920 14389 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1672330a6a3443c9a4fd6455c60e7245. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.265081 14386 tablet_service.cc:1514] Processing DeleteTablet for tablet 4535ac88f78f48da83d9d2f951afdfa6 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a3964469dbd460c8416828043f862c4) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.271946 14392 tablet_service.cc:1467] Processing CreateTablet for tablet 33b1e6a1825e48c891b85a9e2d58ec75 (DEFAULT_TABLE table=default.Table_0657abe634564cec8f7f3ebc673a0ed1 [id=c6660e1d2a394ed99f8566afec3811e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.307857 14392 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33b1e6a1825e48c891b85a9e2d58ec75. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.273838 16846 raft_consensus.cc:491] T 192a2efd3e74463e9ea5844eaaf53c97 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.307921 16846 raft_consensus.cc:513] T 192a2efd3e74463e9ea5844eaaf53c97 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.308296 16846 leader_election.cc:290] T 192a2efd3e74463e9ea5844eaaf53c97 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.274467 14402 tablet_service.cc:1514] Processing DeleteTablet for tablet af712cc03750455087ecd381cc97d464 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:23 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.309510 14548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "192a2efd3e74463e9ea5844eaaf53c97" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
W20250115 03:06:23.309871 14356 leader_election.cc:343] T 192a2efd3e74463e9ea5844eaaf53c97 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.277685 14394 tablet_service.cc:1467] Processing CreateTablet for tablet ee7378fde6094f90ab8a35fd2feb45b8 (DEFAULT_TABLE table=default.Table_d3c621d275d449c1b66d9f6e2c35238e [id=c315d59bf8c84a04ae46401543ad5bd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.310173 14394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee7378fde6094f90ab8a35fd2feb45b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.280120 14401 tablet_service.cc:1467] Processing CreateTablet for tablet 81f59552c0414dcaa5aa5e81aeaec94b (DEFAULT_TABLE table=default.Table_3a10e88b551746deb92961d0cc131814 [id=61e6913e435f49428b764d934ad8c049]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.311574 14401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81f59552c0414dcaa5aa5e81aeaec94b. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.312490 14168 catalog_manager.cc:5526] T 68a04d57feea40eeb8e06cb6a2c61119 P e8df7247ee9c43e0aa18216da2661f75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194). New cstate: current_term: 1 leader_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } }
I20250115 03:06:23.281898 14390 tablet_service.cc:1467] Processing CreateTablet for tablet b0fd66f08783480594a24da777f3e493 (DEFAULT_TABLE table=default.Table_78fb2ed5aedf41f18778eb7a136fd9b0 [id=d96d9893ed5144c68a3b283621aacdef]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.312870 14390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0fd66f08783480594a24da777f3e493. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.282883 14387 tablet_service.cc:1467] Processing CreateTablet for tablet 0db7ad17d6274966a5000a07e71bce17 (DEFAULT_TABLE table=default.Table_ed04cecd900245b7a6b4864fd816043a [id=013718a7768a483595c29a1e7722125b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.314229 14387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0db7ad17d6274966a5000a07e71bce17. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.283826 14399 tablet_service.cc:1514] Processing DeleteTablet for tablet c127b5ac695a456aa38479a5f9a8fe82 with delete_type TABLET_DATA_DELETED (replaced by tablet a5eda11ab1a24b5dbe7b177a49671cf5) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.283839 14412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "820d5af6daa74cadb53d2a5229854a60" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.267037 14385 tablet_service.cc:1514] Processing DeleteTablet for tablet daea48ecc69d441ab66fcb40ea2e7a66 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:18 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.315454 14412 raft_consensus.cc:2463] T 820d5af6daa74cadb53d2a5229854a60 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.286051 14395 tablet_service.cc:1514] Processing DeleteTablet for tablet 293a76d0d49e4a12ab951cbc7311d0f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.288609 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b142fa43a7734d1fb6e3d04bc720a74f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.315670 14420 raft_consensus.cc:2463] T b142fa43a7734d1fb6e3d04bc720a74f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.284667 14393 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ac903c81a1f481ab587e6772c306a96 with delete_type TABLET_DATA_DELETED (replaced by tablet 607fc76ce73744c4b8ac5975515e1fa3) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.295950 14416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7340ac89e5524c35948f3acc6c941cb5" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.315994 14416 raft_consensus.cc:2463] T 7340ac89e5524c35948f3acc6c941cb5 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:23.279135 14388 tablet_service.cc:1467] Processing CreateTablet for tablet fc8f98137bae40c0a0e992f163e6c558 (DEFAULT_TABLE table=default.Table_1ba1ad97ed8d4607882ced8381790a9e [id=15b1073c624649448e1a8fd2a0ab5219]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.272980 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 143879565b6244b280bf4b21f6fc38be (DEFAULT_TABLE table=default.Table_91c8242231ef4fc3ba7adbd6a1de8689 [id=e0ee6f5f0c324b218bdf608a3aaa7ec4]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.287220 14417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12b78b3abfe84714a97ecc89f2fb9e33" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.301519 14809 raft_consensus.cc:357] T 3724ff14f535416e859339a137c05050 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.316268 14388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc8f98137bae40c0a0e992f163e6c558. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.316298 14809 raft_consensus.cc:383] T 3724ff14f535416e859339a137c05050 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.316339 14809 raft_consensus.cc:738] T 3724ff14f535416e859339a137c05050 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.316442 14809 consensus_queue.cc:260] T 3724ff14f535416e859339a137c05050 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.316725 14809 ts_tablet_manager.cc:1428] T 3724ff14f535416e859339a137c05050 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.016s user 0.001s sys 0.000s
I20250115 03:06:23.316273 14417 raft_consensus.cc:2463] T 12b78b3abfe84714a97ecc89f2fb9e33 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.299094 14400 tablet_service.cc:1514] Processing DeleteTablet for tablet 4febfae2f2794669b5997a1fa7ef7412 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:19 UTC) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.285581 14397 tablet_service.cc:1514] Processing DeleteTablet for tablet d140be0fab694f59b8a535392970fad0 with delete_type TABLET_DATA_DELETED (replaced by tablet eb787122e6674201bf62d6bc787e8c20) from {username='slave'} at 127.0.0.1:45810
I20250115 03:06:23.320996 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_20058f3c37f84990b4ea6a66260d5db6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:23.321094 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_b97baa78114e4eab889110d44a2da354" } new_table_name: "default.Table_b2cd344400c045fea6186580b3b9d03b" modify_external_catalogs: true
W20250115 03:06:23.321318 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_20058f3c37f84990b4ea6a66260d5db6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:23.321751 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_ce2812714c014ef283a1ef551f750414"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:23.321836 14164 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b9787d4b06224df483f9c5938c896f99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:23.321995 14165 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_888e1854ea244e68914b76bc41d90137" } new_table_name: "default.Table_137c5200ef264ea580bc8f4d33b18bec" modify_external_catalogs: true
W20250115 03:06:23.322028 14164 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b9787d4b06224df483f9c5938c896f99 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:23.292814 14391 tablet_service.cc:1514] Processing DeleteTablet for tablet d8f54e9ccd04475f826ee851b0f079c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:20 UTC) from {username='slave'} at 127.0.0.1:45810
W20250115 03:06:23.323194 14491 service_pool.cc:137] CreateTablet request on kudu.tserver.TabletServerAdminService from 127.0.0.1:47222 dropped due to backpressure: service queue is full with 50 items
I20250115 03:06:23.323344 16904 raft_consensus.cc:491] T 820d5af6daa74cadb53d2a5229854a60 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.323408 16904 raft_consensus.cc:3054] T 820d5af6daa74cadb53d2a5229854a60 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.323244 14491 service_pool.cc:138] CreateTablet request on kudu.tserver.TabletServerAdminService from 127.0.0.1:47222 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1119)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1120)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1121)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1122)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1123)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1124)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1125)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1126)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1127)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1128)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1129)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1130)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1131)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1132)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1133)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1134)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1135)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1136)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1137)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1138)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1139)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1140)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1141)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1142)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1143)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1144)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1145)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1146)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1147)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1148)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1149)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1150)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1151)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1152)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1153)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1154)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1155)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1156)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1157)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1158)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1159)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1160)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1161)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1162)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1163)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1164)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1165)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1166)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1167)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1168)
I20250115 03:06:23.308945 14678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "192a2efd3e74463e9ea5844eaaf53c97" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:23.324376 14809 tablet_bootstrap.cc:492] T c07e0a0d439041cdbe089b3de4597098 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:23.324456 14393 tablet_service.cc:1467] Processing CreateTablet for tablet bddd45757a0a4e858fea442909427be3 (DEFAULT_TABLE table=default.Table_4b5c24b98b6c44ac80e91078b456cd69 [id=c113d95d41d84f2386d56102781e73a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.325296 14393 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bddd45757a0a4e858fea442909427be3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.325317 14809 tablet_bootstrap.cc:654] T c07e0a0d439041cdbe089b3de4597098 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:23.322036 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ce2812714c014ef283a1ef551f750414 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:23.326598 14395 tablet_service.cc:1467] Processing CreateTablet for tablet db66e69735af40768a787f45910d4dd9 (DEFAULT_TABLE table=default.Table_216163bc6a7d4d9292b2ff6766a5397d [id=9688a1abfd92403b9f109478c67c1cbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.326848 14395 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db66e69735af40768a787f45910d4dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.327328 14163 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_d4c68c853b8f47f4b1ed8208268593ee"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:23.327630 14163 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d4c68c853b8f47f4b1ed8208268593ee in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:23.328250 14162 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_76cc65ff3f7b40be8d0ad0047434e58b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250115 03:06:23.328428 16406 raft_consensus.cc:491] T c8ac40736c74479dab284ffc7a394883 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.328500 16406 raft_consensus.cc:513] T c8ac40736c74479dab284ffc7a394883 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
W20250115 03:06:23.328526 14162 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_76cc65ff3f7b40be8d0ad0047434e58b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:23.328804 16406 leader_election.cc:290] T c8ac40736c74479dab284ffc7a394883 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:23.328872 14167 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 224fbd2c521f4de99fb53d961736a5b5 consensus state has no leader
I20250115 03:06:23.330096 14397 tablet_service.cc:1467] Processing CreateTablet for tablet ec3843ab792a4420bb5f6c55af9d981e (DEFAULT_TABLE table=default.Table_b8fe10217f50463e9498565886de04c6 [id=a7b11960de7f4eb0a3c379bf8ee96f2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.330353 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 545738d072b34d21a82d095de8586d4f (DEFAULT_TABLE table=default.Table_15ff7d65ac354c86b25af14aa5881324 [id=4d1d15e2e47e4b16b00463af4c5ab0aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.330430 14548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ac40736c74479dab284ffc7a394883" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.331079 14391 tablet_service.cc:1467] Processing CreateTablet for tablet a0cb81f7feb94d6c939b69a0ce51c6bc (DEFAULT_TABLE table=default.Table_dc41872d048942e7851fd8d74fce612b [id=38e8dc20bfd14687bcaa8589ee797404]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.332693 14397 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec3843ab792a4420bb5f6c55af9d981e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.334733 16406 raft_consensus.cc:491] T 3724ff14f535416e859339a137c05050 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.334822 16406 raft_consensus.cc:513] T 3724ff14f535416e859339a137c05050 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.335168 16406 leader_election.cc:290] T 3724ff14f535416e859339a137c05050 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
W20250115 03:06:23.337160 14356 leader_election.cc:343] T c8ac40736c74479dab284ffc7a394883 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.312908 14812 tablet_bootstrap.cc:492] T d140be0fab694f59b8a535392970fad0 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.315981 14628 leader_election.cc:304] T b142fa43a7734d1fb6e3d04bc720a74f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.337380 14812 ts_tablet_manager.cc:1397] T d140be0fab694f59b8a535392970fad0 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.041s user 0.002s sys 0.000s
I20250115 03:06:23.337868 14628 leader_election.cc:304] T 12b78b3abfe84714a97ecc89f2fb9e33 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.337841 14812 raft_consensus.cc:357] T d140be0fab694f59b8a535392970fad0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.337944 14812 raft_consensus.cc:383] T d140be0fab694f59b8a535392970fad0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.337977 14812 raft_consensus.cc:738] T d140be0fab694f59b8a535392970fad0 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.338028 14676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ac40736c74479dab284ffc7a394883" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:23.338097 14358 leader_election.cc:343] T 192a2efd3e74463e9ea5844eaaf53c97 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.338085 14812 consensus_queue.cc:260] T d140be0fab694f59b8a535392970fad0 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.338157 14358 leader_election.cc:304] T 192a2efd3e74463e9ea5844eaaf53c97 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.338243 14679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3724ff14f535416e859339a137c05050" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:23.338352 14812 ts_tablet_manager.cc:1428] T d140be0fab694f59b8a535392970fad0 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20250115 03:06:23.338508 14812 tablet_bootstrap.cc:492] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:23.338693 14548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3724ff14f535416e859339a137c05050" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.338801 14168 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_4f2f4f63c42b49c3a90d4979a6057a0a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:23.339057 14358 leader_election.cc:343] T c8ac40736c74479dab284ffc7a394883 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.339116 14358 leader_election.cc:304] T c8ac40736c74479dab284ffc7a394883 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
W20250115 03:06:23.339115 14168 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f2f4f63c42b49c3a90d4979a6057a0a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:23.339323 14358 leader_election.cc:343] T 3724ff14f535416e859339a137c05050 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.339432 16846 raft_consensus.cc:2743] T 192a2efd3e74463e9ea5844eaaf53c97 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:23.339469 14812 tablet_bootstrap.cc:654] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.339527 16846 raft_consensus.cc:2743] T c8ac40736c74479dab284ffc7a394883 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:23.339813 16905 raft_consensus.cc:2798] T b142fa43a7734d1fb6e3d04bc720a74f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.339865 16905 raft_consensus.cc:491] T b142fa43a7734d1fb6e3d04bc720a74f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.339920 16905 raft_consensus.cc:3054] T b142fa43a7734d1fb6e3d04bc720a74f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.340849 16889 raft_consensus.cc:2798] T 12b78b3abfe84714a97ecc89f2fb9e33 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.340905 16889 raft_consensus.cc:491] T 12b78b3abfe84714a97ecc89f2fb9e33 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.340965 16889 raft_consensus.cc:3054] T 12b78b3abfe84714a97ecc89f2fb9e33 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.342365 14164 catalog_manager.cc:5618] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107) does not have the latest schema for tablet 3724ff14f535416e859339a137c05050 (table default.Table_f6d9aee213514fd9a38318e1fc091215 [id=adfd7e86aa0f48e5b5e36b4113517dba]). Expected version 2 got 0
I20250115 03:06:23.342728 14396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 143879565b6244b280bf4b21f6fc38be. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:23.342723 14165 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 917a0c0014a741ba879b7fbda8c9cd3d consensus state has no leader
I20250115 03:06:23.345072 16881 raft_consensus.cc:491] T 9c436975bd034068a6d58d5831db1896 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.345177 16881 raft_consensus.cc:513] T 9c436975bd034068a6d58d5831db1896 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.345398 14400 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 545738d072b34d21a82d095de8586d4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.345528 16881 leader_election.cc:290] T 9c436975bd034068a6d58d5831db1896 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.346454 14167 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_b5ca4985598a4b6a815e04fa64705190"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:23.346743 14167 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5ca4985598a4b6a815e04fa64705190 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20250115 03:06:23.347915 16739 consensus_meta.cc:287] T a8a5743475814401ac2c0a51cfd70207 P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent flushing consensus metadata: real 0.792s user 0.001s sys 0.000s
I20250115 03:06:23.347918 14681 raft_consensus.cc:2463] T 0c580cdf3a854df988df32a9294c83bc P 5c0dae7aff4b44c699275b95ba639770 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 1.
I20250115 03:06:23.347971 16739 raft_consensus.cc:513] T a8a5743475814401ac2c0a51cfd70207 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.348258 16739 leader_election.cc:290] T a8a5743475814401ac2c0a51cfd70207 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Requested vote from peers 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:23.348598 14820 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1042) took 1578 ms. Trace:
W20250115 03:06:23.348821 14670 consensus_meta.cc:287] T f118a169019d4a32aa86b4fe9314f747 P 5c0dae7aff4b44c699275b95ba639770: Time spent flushing consensus metadata: real 1.168s user 0.001s sys 0.000s
I20250115 03:06:23.348647 14820 rpcz_store.cc:276] 0115 03:06:21.770342 (+ 0us) service_pool.cc:168] Inserting onto call queue
0115 03:06:21.770443 (+ 101us) service_pool.cc:225] Handling call
0115 03:06:22.179872 (+409429us) ts_tablet_manager.cc:1145] Deleting tablet 1016cf85491d4ceda132ee57325b48a4
0115 03:06:22.179876 (+ 4us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0115 03:06:22.287830 (+107954us) tablet_metadata.cc:641] Metadata flushed
0115 03:06:22.567795 (+279965us) tablet_metadata.cc:641] Metadata flushed
0115 03:06:23.348591 (+780796us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":409249}
I20250115 03:06:23.349386 14820 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1043) took 1578 ms. Trace:
I20250115 03:06:23.349431 14820 rpcz_store.cc:276] 0115 03:06:21.770409 (+ 0us) service_pool.cc:168] Inserting onto call queue
0115 03:06:21.770441 (+ 32us) service_pool.cc:225] Handling call
0115 03:06:23.349295 (+1578854us) ts_tablet_manager.cc:1145] Deleting tablet ec0f3424ebcb442597a94ccdbb25eeb6
0115 03:06:23.349302 (+ 7us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0115 03:06:23.349361 (+ 59us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1578618}
I20250115 03:06:23.349576 14820 tablet_replica.cc:331] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1: stopping tablet replica
W20250115 03:06:23.349637 14666 consensus_meta.cc:287] T cd07196be4b54a90865aecea55493aa1 P 5c0dae7aff4b44c699275b95ba639770: Time spent flushing consensus metadata: real 1.054s user 0.000s sys 0.000s
I20250115 03:06:23.349689 14820 raft_consensus.cc:2238] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Raft consensus shutting down.
I20250115 03:06:23.349798 14820 pending_rounds.cc:62] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1: Trying to abort 1 pending ops.
I20250115 03:06:23.349818 14820 pending_rounds.cc:69] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7114384928251641856 op_type: NO_OP noop_request { }
I20250115 03:06:23.349834 14666 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36830 (request call id 1073) took 1308 ms. Trace:
I20250115 03:06:23.349861 14820 raft_consensus.cc:2883] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250115 03:06:23.349902 14820 raft_consensus.cc:2267] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:23.349881 14666 rpcz_store.cc:276] 0115 03:06:22.041156 (+ 0us) service_pool.cc:168] Inserting onto call queue
0115 03:06:22.041203 (+ 47us) service_pool.cc:225] Handling call
0115 03:06:22.041464 (+ 261us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0115 03:06:22.041480 (+ 16us) ts_tablet_manager.cc:676] Creating new metadata...
0115 03:06:22.295831 (+254351us) tablet_metadata.cc:641] Metadata flushed
0115 03:06:23.349729 (+1053898us) tablet_replica.cc:206] Creating consensus instance
0115 03:06:23.349826 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250115 03:06:23.350160 14666 tablet_service.cc:1467] Processing CreateTablet for tablet eee4e25e788646c9abaf9ad9ccb0a92c (DEFAULT_TABLE table=default.Table_6770369286d841e39d362e0111c8733c [id=69bc7ddbc89844ebb1471556cd9130af]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.350380 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eee4e25e788646c9abaf9ad9ccb0a92c. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:23.352663 14653 consensus_meta.cc:287] T 26fa8ff146834d32b97943e14f170e67 P 5c0dae7aff4b44c699275b95ba639770: Time spent flushing consensus metadata: real 1.166s user 0.000s sys 0.001s
I20250115 03:06:23.352861 14653 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36830 (request call id 1060) took 1407 ms. Trace:
I20250115 03:06:23.352908 14653 rpcz_store.cc:276] 0115 03:06:21.945794 (+ 0us) service_pool.cc:168] Inserting onto call queue
0115 03:06:21.945833 (+ 39us) service_pool.cc:225] Handling call
0115 03:06:21.946219 (+ 386us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0115 03:06:21.946236 (+ 17us) ts_tablet_manager.cc:676] Creating new metadata...
0115 03:06:22.186503 (+240267us) tablet_metadata.cc:641] Metadata flushed
0115 03:06:23.352762 (+1166259us) tablet_replica.cc:206] Creating consensus instance
0115 03:06:23.352854 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250115 03:06:23.353147 16739 raft_consensus.cc:491] T 15263a3b32be418288770ce348c082d7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.353191 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 2742a2d5a8244400b15c7c85f51b4bc7 (DEFAULT_TABLE table=default.Table_8b8c2a1323c94760a450e447b9a49789 [id=0411ce1e2061458dbc42348a839e6c70]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.353216 16739 raft_consensus.cc:513] T 15263a3b32be418288770ce348c082d7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.353426 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2742a2d5a8244400b15c7c85f51b4bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.353523 16739 leader_election.cc:290] T 15263a3b32be418288770ce348c082d7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.353510 14666 tablet_service.cc:1467] Processing CreateTablet for tablet fc4c45cf04854df5af831417623f14f1 (DEFAULT_TABLE table=default.Table_48bca7fdfb0544a59c84fc547c19e499 [id=59dba0a633d2416eb65151bc64e41490]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.353776 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc4c45cf04854df5af831417623f14f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.354280 14820 ts_tablet_manager.cc:1905] T d891020ed0bb4fc49155bd9f2f6ed35c P 84b1408700ef479cbb5a7b4d2f0bebf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:23.355273 14653 tablet_service.cc:1467] Processing CreateTablet for tablet 47fed237bfc94d7e819557e9fce0ce95 (DEFAULT_TABLE table=default.Table_3f9adcb39f2945e88044d6f0ed9797ad [id=5fefdddd34b441ed8d5c93eb06fcda0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.355473 14653 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47fed237bfc94d7e819557e9fce0ce95. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.355545 14666 tablet_service.cc:1467] Processing CreateTablet for tablet a3ddba7630904f16a36698c1e5e7e9cf (DEFAULT_TABLE table=default.Table_1288c4fed9bb4bee8a2a567a7b16990d [id=342cd63e28b34a4c8cf552111b3325e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.355772 14666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3ddba7630904f16a36698c1e5e7e9cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.352861 14491 service_pool.cc:138] CreateTablet request on kudu.tserver.TabletServerAdminService from 127.0.0.1:47222 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1119)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1120)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1121)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1122)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1123)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1124)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1125)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1126)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1127)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1128)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1129)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1130)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1131)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1132)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1133)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1134)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1135)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1136)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1137)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1138)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1139)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1140)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1141)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1142)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1143)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1144)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1145)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1146)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1147)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1148)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1149)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1150)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1151)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1152)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1153)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1154)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1155)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1156)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1157)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1158)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1159)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1160)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1161)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1162)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1163)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1164)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1165)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1166)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1167)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1168)
I20250115 03:06:23.356181 14491 service_pool.cc:138] CreateTablet request on kudu.tserver.TabletServerAdminService from 127.0.0.1:47222 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1119)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1120)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1121)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1122)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1123)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1124)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1125)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1126)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1127)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1128)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1129)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1130)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1131)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1132)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1133)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1134)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1135)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1136)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1137)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1138)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1139)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1140)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1141)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1142)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1143)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1144)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1145)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1146)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1147)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1148)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1149)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1150)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1151)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1152)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1153)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1154)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1155)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1156)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1157)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1158)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1159)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1160)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1161)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1162)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1163)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1164)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1165)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1166)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1167)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1168)
I20250115 03:06:23.357041 14670 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36830 (request call id 1061) took 1410 ms. Trace:
I20250115 03:06:23.357097 14670 rpcz_store.cc:276] 0115 03:06:21.946645 (+ 0us) service_pool.cc:168] Inserting onto call queue
0115 03:06:21.946768 (+ 123us) service_pool.cc:225] Handling call
0115 03:06:21.946998 (+ 230us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0115 03:06:21.947011 (+ 13us) ts_tablet_manager.cc:676] Creating new metadata...
0115 03:06:22.181140 (+234129us) tablet_metadata.cc:641] Metadata flushed
0115 03:06:23.348903 (+1167763us) tablet_replica.cc:206] Creating consensus instance
0115 03:06:23.357031 (+ 8128us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250115 03:06:23.357348 14670 tablet_service.cc:1467] Processing CreateTablet for tablet ad5ada0afc2d4a4d94595fa951fcda1f (DEFAULT_TABLE table=default.Table_55bbd621f94d45fc8b93a7211619f08a [id=8ad688c2aa5048db9a71e6b7611e98ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.357422 14809 tablet_bootstrap.cc:492] T c07e0a0d439041cdbe089b3de4597098 P e8df7247ee9c43e0aa18216da2661f75: No bootstrap required, opened a new log
I20250115 03:06:23.357509 14809 ts_tablet_manager.cc:1397] T c07e0a0d439041cdbe089b3de4597098 P e8df7247ee9c43e0aa18216da2661f75: Time spent bootstrapping tablet: real 0.033s user 0.000s sys 0.002s
I20250115 03:06:23.357551 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad5ada0afc2d4a4d94595fa951fcda1f. 1 dirs total, 0 dirs full, 0 dirs failed
W20250115 03:06:23.357800 14660 consensus_meta.cc:287] T ee9ef489671c454993d67632766ab1d7 P 5c0dae7aff4b44c699275b95ba639770: Time spent flushing consensus metadata: real 1.170s user 0.001s sys 0.000s
I20250115 03:06:23.357957 14660 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:36830 (request call id 1064) took 1393 ms. Trace:
I20250115 03:06:23.357991 14809 raft_consensus.cc:357] T c07e0a0d439041cdbe089b3de4597098 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.358026 14660 rpcz_store.cc:276] 0115 03:06:21.964322 (+ 0us) service_pool.cc:168] Inserting onto call queue
0115 03:06:21.964363 (+ 41us) service_pool.cc:225] Handling call
0115 03:06:21.964629 (+ 266us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0115 03:06:21.964644 (+ 15us) ts_tablet_manager.cc:676] Creating new metadata...
0115 03:06:22.187322 (+222678us) tablet_metadata.cc:641] Metadata flushed
0115 03:06:23.357873 (+1170551us) tablet_replica.cc:206] Creating consensus instance
0115 03:06:23.357950 (+ 77us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20250115 03:06:23.358083 14809 raft_consensus.cc:383] T c07e0a0d439041cdbe089b3de4597098 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.358111 14809 raft_consensus.cc:738] T c07e0a0d439041cdbe089b3de4597098 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8df7247ee9c43e0aa18216da2661f75, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.358151 14660 tablet_service.cc:1514] Processing DeleteTablet for tablet edf17b41198b4f8fbe5db22c58aa6bb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-01-15 03:06:22 UTC) from {username='slave'} at 127.0.0.1:36830
I20250115 03:06:23.358217 14809 consensus_queue.cc:260] T c07e0a0d439041cdbe089b3de4597098 P e8df7247ee9c43e0aa18216da2661f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.358354 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 1a179a7c7aab4831a42e9f25f22163ad (DEFAULT_TABLE table=default.Table_1f46782fd85640939416bcf0de0784e0 [id=02c20c94390c4ba1a7bc2a0c1611fb62]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.358562 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a179a7c7aab4831a42e9f25f22163ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.358687 14809 ts_tablet_manager.cc:1428] T c07e0a0d439041cdbe089b3de4597098 P e8df7247ee9c43e0aa18216da2661f75: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:23.358847 14809 tablet_bootstrap.cc:492] T 29c72f33a2034c4eae81c7dfcca27ca4 P e8df7247ee9c43e0aa18216da2661f75: Bootstrap starting.
I20250115 03:06:23.359385 14670 tablet_service.cc:1467] Processing CreateTablet for tablet 1822e678e3a44f24a6f4d4b13e86e8aa (DEFAULT_TABLE table=default.Table_0fb325f01dbc4e24990e73ac9d846313 [id=c32d1b3b38d2471092547a983a7ce6f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.359591 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1822e678e3a44f24a6f4d4b13e86e8aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.360507 14660 tablet_service.cc:1467] Processing CreateTablet for tablet 84c592ced5b443869240eff55fe9d549 (DEFAULT_TABLE table=default.Table_91a3670ada684454977bfbc2bf08a599 [id=fca39b0eb2374166b5f78b1a94c66451]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.360707 14660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84c592ced5b443869240eff55fe9d549. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 03:06:23.360348 14809 tablet_bootstrap.cc:654] T 29c72f33a2034c4eae81c7dfcca27ca4 P e8df7247ee9c43e0aa18216da2661f75: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.362239 14819 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45810 (request call id 1115) took 1502 ms. Trace:
I20250115 03:06:23.362298 14819 rpcz_store.cc:276] 0115 03:06:21.860033 (+ 0us) service_pool.cc:168] Inserting onto call queue
0115 03:06:21.860426 (+ 393us) service_pool.cc:225] Handling call
0115 03:06:23.112162 (+1251736us) ts_tablet_manager.cc:1145] Deleting tablet 78cc96de8b414ac19285338ccb1024e0
0115 03:06:23.112169 (+ 7us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0115 03:06:23.159090 (+ 46921us) tablet_metadata.cc:641] Metadata flushed
0115 03:06:23.254370 (+ 95280us) tablet_metadata.cc:641] Metadata flushed
0115 03:06:23.362229 (+107859us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1251615}
I20250115 03:06:23.362471 14819 tablet_replica.cc:331] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75: stopping tablet replica
I20250115 03:06:23.362577 14819 raft_consensus.cc:2238] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250115 03:06:23.362563 14158 catalog_manager.cc:4872] TS e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107): tablet 78cc96de8b414ac19285338ccb1024e0 (table default.Table_e6dfaa9c84d144e68a2d047436d0ecd2 [id=63ea052eebb34ba4a66eded622ccd4a4]) successfully deleted
I20250115 03:06:23.362664 14819 raft_consensus.cc:2267] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250115 03:06:23.362993 14819 ts_tablet_manager.cc:1905] T 78b5e2e16442407880b75d256e6b2614 P e8df7247ee9c43e0aa18216da2661f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 03:06:23.363052 16739 raft_consensus.cc:491] T 2b0e3df3c79c40ba958b6e2453fa666f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.363118 16739 raft_consensus.cc:513] T 2b0e3df3c79c40ba958b6e2453fa666f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.363413 16739 leader_election.cc:290] T 2b0e3df3c79c40ba958b6e2453fa666f P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.363724 14416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c436975bd034068a6d58d5831db1896" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.363828 14416 raft_consensus.cc:2463] T 9c436975bd034068a6d58d5831db1896 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
W20250115 03:06:23.363881 14356 leader_election.cc:343] T 3724ff14f535416e859339a137c05050 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.363945 14356 leader_election.cc:304] T 3724ff14f535416e859339a137c05050 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.364151 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15263a3b32be418288770ce348c082d7" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.364234 16406 raft_consensus.cc:2743] T 3724ff14f535416e859339a137c05050 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:23.364306 14417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a5743475814401ac2c0a51cfd70207" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75"
I20250115 03:06:23.364243 14420 raft_consensus.cc:2463] T 15263a3b32be418288770ce348c082d7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:23.364375 14417 raft_consensus.cc:3054] T a8a5743475814401ac2c0a51cfd70207 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.364672 14493 leader_election.cc:304] T 9c436975bd034068a6d58d5831db1896 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:23.364837 16739 raft_consensus.cc:2798] T 9c436975bd034068a6d58d5831db1896 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.364893 16739 raft_consensus.cc:491] T 9c436975bd034068a6d58d5831db1896 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.364954 16739 raft_consensus.cc:3054] T 9c436975bd034068a6d58d5831db1896 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.365319 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b0e3df3c79c40ba958b6e2453fa666f" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.365409 14420 raft_consensus.cc:2463] T 2b0e3df3c79c40ba958b6e2453fa666f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:23.365609 14493 leader_election.cc:304] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1; no voters: e8df7247ee9c43e0aa18216da2661f75
I20250115 03:06:23.365816 14493 leader_election.cc:304] T 15263a3b32be418288770ce348c082d7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:23.365856 14679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c436975bd034068a6d58d5831db1896" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:23.365955 14679 raft_consensus.cc:2463] T 9c436975bd034068a6d58d5831db1896 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:23.366048 16874 raft_consensus.cc:2798] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 03:06:23.366101 14676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8a5743475814401ac2c0a51cfd70207" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770"
I20250115 03:06:23.366119 16874 raft_consensus.cc:695] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 1 LEADER]: Becoming Leader. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Running, Role: LEADER
I20250115 03:06:23.366173 14676 raft_consensus.cc:3054] T a8a5743475814401ac2c0a51cfd70207 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.366242 16874 consensus_queue.cc:237] T 0c580cdf3a854df988df32a9294c83bc P 84b1408700ef479cbb5a7b4d2f0bebf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.366335 14156 catalog_manager.cc:4872] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): tablet 1016cf85491d4ceda132ee57325b48a4 (table default.Table_2e115074bf7049bfbb1239b4d9c1ca2e [id=6cf622de00cb490bb794837223aa7bcb]) successfully deleted
I20250115 03:06:23.366365 14678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15263a3b32be418288770ce348c082d7" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:23.366459 14156 catalog_manager.cc:4852] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): delete failed for tablet ec0f3424ebcb442597a94ccdbb25eeb6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ec0f3424ebcb442597a94ccdbb25eeb6 already in progress: creating tablet
I20250115 03:06:23.366536 14681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b0e3df3c79c40ba958b6e2453fa666f" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:23.366605 14681 raft_consensus.cc:2463] T 2b0e3df3c79c40ba958b6e2453fa666f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:23.366833 16874 raft_consensus.cc:2798] T 15263a3b32be418288770ce348c082d7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.366881 16874 raft_consensus.cc:491] T 15263a3b32be418288770ce348c082d7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.366914 16874 raft_consensus.cc:3054] T 15263a3b32be418288770ce348c082d7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.367334 14171 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60432:
name: "default.Table_8db976fec3854583b01162b3c90b9a23"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 03:06:23.367674 14171 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8db976fec3854583b01162b3c90b9a23 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 03:06:23.368093 14491 service_pool.cc:138] CreateTablet request on kudu.tserver.TabletServerAdminService from 127.0.0.1:47222 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1119)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1120)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1121)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1122)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1123)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1124)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1125)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1126)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1127)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1128)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1129)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1130)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1131)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1132)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1133)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1134)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1135)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1136)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1137)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1138)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1139)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1140)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1141)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1142)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1143)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1144)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1145)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1146)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1147)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1148)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1149)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1150)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1151)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1152)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1153)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1154)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1155)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1156)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1157)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1158)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1159)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1160)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1161)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1162)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1163)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1164)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1165)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1166)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1167)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1168)
I20250115 03:06:23.369616 14392 tablet_service.cc:1467] Processing CreateTablet for tablet aaab3c37346840729df83740afe3fa3c (DEFAULT_TABLE table=default.Table_11659a810c9a4358af952c8ecdcf9d87 [id=74c3bc2e0f9b49a0be46ac80bf3a123b]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.370302 14396 tablet_service.cc:1467] Processing CreateTablet for tablet 6bc1b466818f4fb785b8be09ff84aabb (DEFAULT_TABLE table=default.Table_feac21ed047342a3bd9170f367844b37 [id=635feff9f5e549a9b834ff10731af7fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.371709 14400 tablet_service.cc:1467] Processing CreateTablet for tablet 11dba4635db74eef873d459f0a8597e7 (DEFAULT_TABLE table=default.Table_8a8b70005ac8484f9117b80d600afdf7 [id=55e96efa25c34ca5b1a9f26e5e65233c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250115 03:06:23.377252 14493 leader_election.cc:304] T 2b0e3df3c79c40ba958b6e2453fa666f P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:23.377228 14167 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_8a8b70005ac8484f9117b80d600afdf7" } new_table_name: "default.Table_db489690a62e4219a499f588f3827743" modify_external_catalogs: true
I20250115 03:06:23.377600 14858 raft_consensus.cc:2798] T 2b0e3df3c79c40ba958b6e2453fa666f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.377651 14858 raft_consensus.cc:491] T 2b0e3df3c79c40ba958b6e2453fa666f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.377686 14858 raft_consensus.cc:3054] T 2b0e3df3c79c40ba958b6e2453fa666f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.378070 14812 tablet_bootstrap.cc:492] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.378151 14812 ts_tablet_manager.cc:1397] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.040s user 0.000s sys 0.002s
I20250115 03:06:23.378083 14491 service_pool.cc:138] CreateTablet request on kudu.tserver.TabletServerAdminService from 127.0.0.1:47222 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1119)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1120)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1121)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1122)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1123)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1124)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1125)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1126)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1127)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1128)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1129)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1130)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1131)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1132)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1133)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1134)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1135)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1136)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1137)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1138)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1139)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1140)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1141)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1142)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1143)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1144)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1145)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1146)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1147)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1148)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1149)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1150)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1151)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1152)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1153)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1154)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1155)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1156)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1157)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1158)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1159)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1160)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1161)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1162)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1163)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1164)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1165)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1166)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1167)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1168)
I20250115 03:06:23.378662 14812 raft_consensus.cc:357] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.378764 14812 raft_consensus.cc:383] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.378793 14812 raft_consensus.cc:738] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.378485 14491 service_pool.cc:138] CreateTablet request on kudu.tserver.TabletServerAdminService from 127.0.0.1:47222 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1119)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1120)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1121)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1122)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1123)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1124)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1125)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1126)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1127)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1128)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1129)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1130)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1131)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1132)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1133)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1134)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1135)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1136)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1137)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1138)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1139)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1140)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1141)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1142)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1143)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1144)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1145)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1146)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1147)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1148)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1149)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1150)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1151)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1152)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1153)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1154)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1155)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1156)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1157)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1158)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1159)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1160)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1161)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1162)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1163)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1164)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1165)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1166)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1167)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1168)
I20250115 03:06:23.378898 14812 consensus_queue.cc:260] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.378965 14491 service_pool.cc:138] CreateTablet request on kudu.tserver.TabletServerAdminService from 127.0.0.1:47222 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1119)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1120)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1121)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1122)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1123)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1124)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1125)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1126)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1127)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1128)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1129)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1130)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1131)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1132)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1133)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1134)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1135)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1136)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1137)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1138)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1139)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1140)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1141)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1142)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1143)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1144)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1145)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1146)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1147)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1148)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1149)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1150)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1151)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1152)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1153)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1154)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1155)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1156)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1157)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1158)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1159)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1160)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1161)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1162)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1163)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1164)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1165)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1166)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1167)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1168)
I20250115 03:06:23.379251 14812 ts_tablet_manager.cc:1428] T 2ae7d8e213544722bb30bfc7f0f4a9c4 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:23.379412 14812 tablet_bootstrap.cc:492] T 192a2efd3e74463e9ea5844eaaf53c97 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:23.379354 14491 service_pool.cc:138] CreateTablet request on kudu.tserver.TabletServerAdminService from 127.0.0.1:47222 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1119)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1120)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1121)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1122)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1123)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1124)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1125)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1126)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1127)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1128)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1129)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1130)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1131)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1132)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1133)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1134)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1135)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1136)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1137)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1138)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1139)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1140)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1141)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1142)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1143)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1144)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1145)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1146)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1147)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1148)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1149)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1150)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1151)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1152)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1153)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1154)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1155)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1156)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1157)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1158)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1159)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1160)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1161)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1162)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1163)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1164)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1165)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1166)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1167)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1168)
W20250115 03:06:23.380023 14156 catalog_manager.cc:4615] TS 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): CreateTablet RPC failed for tablet db66e69735af40768a787f45910d4dd9: Remote error: Service unavailable: CreateTablet request on kudu.tserver.TabletServerAdminService from 127.0.0.1:47222 dropped due to backpressure: service queue is full with 50 items
I20250115 03:06:23.380609 14812 tablet_bootstrap.cc:654] T 192a2efd3e74463e9ea5844eaaf53c97 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
I20250115 03:06:23.380838 16899 raft_consensus.cc:491] T 85fcafa24fa1448c995d30dd93b0737f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.380920 16899 raft_consensus.cc:513] T 85fcafa24fa1448c995d30dd93b0737f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.381567 16899 leader_election.cc:290] T 85fcafa24fa1448c995d30dd93b0737f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:23.381749 14416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85fcafa24fa1448c995d30dd93b0737f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.381870 14416 raft_consensus.cc:2463] T 85fcafa24fa1448c995d30dd93b0737f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.382063 14554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85fcafa24fa1448c995d30dd93b0737f" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.384771 14811 tablet_bootstrap.cc:492] T 85fcafa24fa1448c995d30dd93b0737f P 84b1408700ef479cbb5a7b4d2f0bebf1: No bootstrap required, opened a new log
I20250115 03:06:23.384847 14811 ts_tablet_manager.cc:1397] T 85fcafa24fa1448c995d30dd93b0737f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent bootstrapping tablet: real 0.155s user 0.000s sys 0.002s
I20250115 03:06:23.385284 14811 raft_consensus.cc:357] T 85fcafa24fa1448c995d30dd93b0737f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.385372 14811 raft_consensus.cc:383] T 85fcafa24fa1448c995d30dd93b0737f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.385397 14811 raft_consensus.cc:738] T 85fcafa24fa1448c995d30dd93b0737f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b1408700ef479cbb5a7b4d2f0bebf1, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.385499 14811 consensus_queue.cc:260] T 85fcafa24fa1448c995d30dd93b0737f P 84b1408700ef479cbb5a7b4d2f0bebf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.385731 14811 ts_tablet_manager.cc:1428] T 85fcafa24fa1448c995d30dd93b0737f P 84b1408700ef479cbb5a7b4d2f0bebf1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20250115 03:06:23.385880 14811 tablet_bootstrap.cc:492] T 12b78b3abfe84714a97ecc89f2fb9e33 P 84b1408700ef479cbb5a7b4d2f0bebf1: Bootstrap starting.
I20250115 03:06:23.386754 14811 tablet_bootstrap.cc:654] T 12b78b3abfe84714a97ecc89f2fb9e33 P 84b1408700ef479cbb5a7b4d2f0bebf1: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:23.389518 14167 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 11dba4635db74eef873d459f0a8597e7 consensus state has no leader
W20250115 03:06:23.389793 14626 leader_election.cc:343] T 85fcafa24fa1448c995d30dd93b0737f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.390017 14628 leader_election.cc:304] T 85fcafa24fa1448c995d30dd93b0737f P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.390278 16899 raft_consensus.cc:491] T b9f67c35591f4fdeb8a926a085171b49 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.390342 16899 raft_consensus.cc:513] T b9f67c35591f4fdeb8a926a085171b49 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.390635 16899 leader_election.cc:290] T b9f67c35591f4fdeb8a926a085171b49 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693)
I20250115 03:06:23.390645 16921 raft_consensus.cc:491] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.390792 16899 consensus_queue.cc:1035] T a8e41e31ffae4bb092cb08d83eda16b5 P 5c0dae7aff4b44c699275b95ba639770 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 03:06:23.390841 16921 raft_consensus.cc:513] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.391011 16899 raft_consensus.cc:2798] T 85fcafa24fa1448c995d30dd93b0737f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.391058 16899 raft_consensus.cc:491] T 85fcafa24fa1448c995d30dd93b0737f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.391095 16899 raft_consensus.cc:3054] T 85fcafa24fa1448c995d30dd93b0737f P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.391142 16921 leader_election.cc:290] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:23.391880 14416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9f67c35591f4fdeb8a926a085171b49" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.391983 14416 raft_consensus.cc:2463] T b9f67c35591f4fdeb8a926a085171b49 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.392160 14412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c4ce5c2921341ce837ee4f5f7d131b7" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.392246 14412 raft_consensus.cc:2463] T 4c4ce5c2921341ce837ee4f5f7d131b7 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.392486 14628 leader_election.cc:304] T b9f67c35591f4fdeb8a926a085171b49 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:23.392633 16923 raft_consensus.cc:2798] T b9f67c35591f4fdeb8a926a085171b49 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.392701 14628 leader_election.cc:304] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:23.392735 16883 raft_consensus.cc:491] T 815c5e096e4d4b0cbfb7c15bac571b10 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.392680 16923 raft_consensus.cc:491] T b9f67c35591f4fdeb8a926a085171b49 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.392835 16923 raft_consensus.cc:3054] T b9f67c35591f4fdeb8a926a085171b49 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.392800 16883 raft_consensus.cc:513] T 815c5e096e4d4b0cbfb7c15bac571b10 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.393107 16921 raft_consensus.cc:2798] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.393378 16921 raft_consensus.cc:491] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.393424 16921 raft_consensus.cc:3054] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.393695 16883 leader_election.cc:290] T 815c5e096e4d4b0cbfb7c15bac571b10 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.395038 14546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c4ce5c2921341ce837ee4f5f7d131b7" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.395144 14546 raft_consensus.cc:2463] T 4c4ce5c2921341ce837ee4f5f7d131b7 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.395282 14548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9f67c35591f4fdeb8a926a085171b49" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.395359 14548 raft_consensus.cc:2463] T b9f67c35591f4fdeb8a926a085171b49 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.402606 16406 raft_consensus.cc:491] T c127b5ac695a456aa38479a5f9a8fe82 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.402594 14171 catalog_manager.cc:2462] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_9fffa7d4b0834e3e85a07968b118ac3a" } modify_external_catalogs: true
I20250115 03:06:23.402698 14171 catalog_manager.cc:2646] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_9fffa7d4b0834e3e85a07968b118ac3a" } modify_external_catalogs: true
I20250115 03:06:23.402694 16406 raft_consensus.cc:513] T c127b5ac695a456aa38479a5f9a8fe82 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.402999 16406 leader_election.cc:290] T c127b5ac695a456aa38479a5f9a8fe82 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.403363 14546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c127b5ac695a456aa38479a5f9a8fe82" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.403363 14682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c127b5ac695a456aa38479a5f9a8fe82" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
W20250115 03:06:23.403663 14358 leader_election.cc:343] T c127b5ac695a456aa38479a5f9a8fe82 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215): Illegal state: must be running to vote when last-logged opid is not known
W20250115 03:06:23.403856 14356 leader_election.cc:343] T c127b5ac695a456aa38479a5f9a8fe82 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.403914 14356 leader_election.cc:304] T c127b5ac695a456aa38479a5f9a8fe82 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e8df7247ee9c43e0aa18216da2661f75; no voters: 5c0dae7aff4b44c699275b95ba639770, 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.404014 16406 raft_consensus.cc:2743] T c127b5ac695a456aa38479a5f9a8fe82 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250115 03:06:23.404544 14682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "815c5e096e4d4b0cbfb7c15bac571b10" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:23.405081 16617 raft_consensus.cc:491] T 85fcafa24fa1448c995d30dd93b0737f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.405150 16617 raft_consensus.cc:513] T 85fcafa24fa1448c995d30dd93b0737f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.405408 16617 leader_election.cc:290] T 85fcafa24fa1448c995d30dd93b0737f P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.405984 14679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85fcafa24fa1448c995d30dd93b0737f" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c0dae7aff4b44c699275b95ba639770" is_pre_election: true
I20250115 03:06:23.406472 14420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "815c5e096e4d4b0cbfb7c15bac571b10" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.406574 14420 raft_consensus.cc:2463] T 815c5e096e4d4b0cbfb7c15bac571b10 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:23.406620 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85fcafa24fa1448c995d30dd93b0737f" candidate_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.406697 14422 raft_consensus.cc:2463] T 85fcafa24fa1448c995d30dd93b0737f P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84b1408700ef479cbb5a7b4d2f0bebf1 in term 0.
I20250115 03:06:23.410684 14493 leader_election.cc:304] T 815c5e096e4d4b0cbfb7c15bac571b10 P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:23.410813 16883 raft_consensus.cc:2798] T 815c5e096e4d4b0cbfb7c15bac571b10 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.410847 16883 raft_consensus.cc:491] T 815c5e096e4d4b0cbfb7c15bac571b10 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.410879 16883 raft_consensus.cc:3054] T 815c5e096e4d4b0cbfb7c15bac571b10 P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.411266 14493 leader_election.cc:304] T 85fcafa24fa1448c995d30dd93b0737f P 84b1408700ef479cbb5a7b4d2f0bebf1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b1408700ef479cbb5a7b4d2f0bebf1, e8df7247ee9c43e0aa18216da2661f75; no voters:
I20250115 03:06:23.411418 16617 raft_consensus.cc:2798] T 85fcafa24fa1448c995d30dd93b0737f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.411473 16617 raft_consensus.cc:491] T 85fcafa24fa1448c995d30dd93b0737f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.411504 16617 raft_consensus.cc:3054] T 85fcafa24fa1448c995d30dd93b0737f P 84b1408700ef479cbb5a7b4d2f0bebf1 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.411854 14163 catalog_manager.cc:3344] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60432:
table { table_name: "default.Table_2ad8dffd9f274a8184b5a06c7254b1b5" } new_table_name: "default.Table_98455e726b87436490ad15e6da085bd6" modify_external_catalogs: true
I20250115 03:06:23.413414 14812 tablet_bootstrap.cc:492] T 192a2efd3e74463e9ea5844eaaf53c97 P 5c0dae7aff4b44c699275b95ba639770: No bootstrap required, opened a new log
I20250115 03:06:23.413497 14812 ts_tablet_manager.cc:1397] T 192a2efd3e74463e9ea5844eaaf53c97 P 5c0dae7aff4b44c699275b95ba639770: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.000s
I20250115 03:06:23.413942 14812 raft_consensus.cc:357] T 192a2efd3e74463e9ea5844eaaf53c97 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.414032 14812 raft_consensus.cc:383] T 192a2efd3e74463e9ea5844eaaf53c97 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 03:06:23.414062 14812 raft_consensus.cc:738] T 192a2efd3e74463e9ea5844eaaf53c97 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c0dae7aff4b44c699275b95ba639770, State: Initialized, Role: FOLLOWER
I20250115 03:06:23.414167 14812 consensus_queue.cc:260] T 192a2efd3e74463e9ea5844eaaf53c97 P 5c0dae7aff4b44c699275b95ba639770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.414237 16928 raft_consensus.cc:491] T 4535ac88f78f48da83d9d2f951afdfa6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.414294 16928 raft_consensus.cc:513] T 4535ac88f78f48da83d9d2f951afdfa6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } }
I20250115 03:06:23.414443 14812 ts_tablet_manager.cc:1428] T 192a2efd3e74463e9ea5844eaaf53c97 P 5c0dae7aff4b44c699275b95ba639770: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20250115 03:06:23.414620 14812 tablet_bootstrap.cc:492] T fe513ca696f14838b3886aac174f2cb5 P 5c0dae7aff4b44c699275b95ba639770: Bootstrap starting.
I20250115 03:06:23.415252 14546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4535ac88f78f48da83d9d2f951afdfa6" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" is_pre_election: true
I20250115 03:06:23.415586 14812 tablet_bootstrap.cc:654] T fe513ca696f14838b3886aac174f2cb5 P 5c0dae7aff4b44c699275b95ba639770: Neither blocks nor log segments found. Creating new log.
W20250115 03:06:23.415612 14626 leader_election.cc:343] T 4535ac88f78f48da83d9d2f951afdfa6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693): Illegal state: must be running to vote when last-logged opid is not known
I20250115 03:06:23.415973 14491 service_pool.cc:138] CreateTablet request on kudu.tserver.TabletServerAdminService from 127.0.0.1:47222 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1119)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1120)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1121)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1122)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1123)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1124)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1125)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1126)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1127)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1128)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1129)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1130)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1131)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1132)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1133)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1134)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1135)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1136)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1137)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1138)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1139)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1140)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1141)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1142)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1143)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1144)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1145)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1146)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1147)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1148)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1149)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1150)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1151)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1152)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1153)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1154)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1155)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1156)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1157)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1158)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1159)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1160)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1161)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1162)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1163)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1164)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1165)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1166)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1177)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1167)
I20250115 03:06:23.414618 16928 leader_election.cc:290] T 4535ac88f78f48da83d9d2f951afdfa6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), e8df7247ee9c43e0aa18216da2661f75 (127.13.186.193:45107)
I20250115 03:06:23.416378 14422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4535ac88f78f48da83d9d2f951afdfa6" candidate_uuid: "5c0dae7aff4b44c699275b95ba639770" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8df7247ee9c43e0aa18216da2661f75" is_pre_election: true
I20250115 03:06:23.416477 14422 raft_consensus.cc:2463] T 4535ac88f78f48da83d9d2f951afdfa6 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c0dae7aff4b44c699275b95ba639770 in term 0.
I20250115 03:06:23.416740 14628 leader_election.cc:304] T 4535ac88f78f48da83d9d2f951afdfa6 P 5c0dae7aff4b44c699275b95ba639770 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c0dae7aff4b44c699275b95ba639770, e8df7247ee9c43e0aa18216da2661f75; no voters: 84b1408700ef479cbb5a7b4d2f0bebf1
I20250115 03:06:23.416945 16928 raft_consensus.cc:2798] T 4535ac88f78f48da83d9d2f951afdfa6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 03:06:23.416989 16928 raft_consensus.cc:491] T 4535ac88f78f48da83d9d2f951afdfa6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 03:06:23.417022 16928 raft_consensus.cc:3054] T 4535ac88f78f48da83d9d2f951afdfa6 P 5c0dae7aff4b44c699275b95ba639770 [term 0 FOLLOWER]: Advancing to term 1
I20250115 03:06:23.417346 14491 service_pool.cc:138] CreateTablet request on kudu.tserver.TabletServerAdminService from 127.0.0.1:47222 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1119)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1120)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1121)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1122)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1123)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1124)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1125)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1126)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1127)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1128)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1129)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1130)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1131)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1132)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1133)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1134)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1135)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1136)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1137)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1138)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1139)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1140)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1141)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1142)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1143)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1144)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1145)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1146)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1147)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1148)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1149)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1150)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1151)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1152)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1153)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1154)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1155)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1156)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1157)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1158)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1159)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1160)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1161)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1162)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1163)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1164)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1165)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1166)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1177)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1167)
W20250115 03:06:23.418972 14163 catalog_manager.cc:5786] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b9430861457247d9a324b43bd83fbae0 consensus state has no leader
I20250115 03:06:23.419448 16406 raft_consensus.cc:491] T 2f0779261368409c9e7b12088f88cb16 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 03:06:23.419539 16406 raft_consensus.cc:513] T 2f0779261368409c9e7b12088f88cb16 P e8df7247ee9c43e0aa18216da2661f75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84b1408700ef479cbb5a7b4d2f0bebf1" member_type: VOTER last_known_addr { host: "127.13.186.194" port: 34693 } } peers { permanent_uuid: "e8df7247ee9c43e0aa18216da2661f75" member_type: VOTER last_known_addr { host: "127.13.186.193" port: 45107 } } peers { permanent_uuid: "5c0dae7aff4b44c699275b95ba639770" member_type: VOTER last_known_addr { host: "127.13.186.195" port: 40215 } }
I20250115 03:06:23.419853 16406 leader_election.cc:290] T 2f0779261368409c9e7b12088f88cb16 P e8df7247ee9c43e0aa18216da2661f75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84b1408700ef479cbb5a7b4d2f0bebf1 (127.13.186.194:34693), 5c0dae7aff4b44c699275b95ba639770 (127.13.186.195:40215)
I20250115 03:06:23.419732 14491 service_pool.cc:138] CreateTablet request on kudu.tserver.TabletServerAdminService from 127.0.0.1:47222 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1119)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1120)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1121)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1122)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1123)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1124)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1125)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1126)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1127)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1128)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1129)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1130)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1131)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1132)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1133)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1134)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1135)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1136)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1137)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1138)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1139)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1140)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1141)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1142)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1143)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1144)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1145)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1146)
Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47222 (request call id 1147)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1148)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1149)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1150)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1151)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1152)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1153)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1154)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1155)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1156)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1157)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1158)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1159)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1160)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1161)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1162)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1163)
Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47222 (request call id 1164)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1165)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1166)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1177)
Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47222 (request call id 1167)
I20250115 03:06:23.420128 14546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f0779261368409c9e7b12088f88cb16" candidate_uuid: "e8df7247ee9c43e0aa18216da2661f75"